Encap FAQ

  1. Introduction
    1. What's wrong with traditional Unix package management?
    2. What is Encap?
    3. How does Encap work?
    4. What are the advantages of Encap?
    5. What does Encap not do?
    6. Encap Terminology
    7. History of Encap
  2. Using epkg, the Encap Package Manager
    1. How do I install epkg?
    2. How do I install an Encap package?
    3. How does epkg handle multiple versions of a package?
    4. Does epkg extract package archives?
    5. Can epkg download package archives?
    6. How does epkg handle filename conflicts?
    7. How do I remove an Encap package?
    8. What about dangling symlinks?
    9. How can I transition to using Encap slowly?
  3. Creating Encap Packages
    1. How do I create an Encap package for a given piece of software?
    2. Can I add a postinstall script to the package?
    3. Do all files in the package directory get linked in?
    4. How do Encap packages interact with each other?
    5. What about local customizations of Encap packages?
    6. How can I create Encap packages of Perl modules?
  4. Relocatable Packages and Other Tricks
    1. Why would I want to create a relocatable Encap package?
    2. How can I create a relocatable Encap package?
    3. How can I process all of my .in files at once?
    4. What if my Encap package is also my source distribution?
    5. How can I keep my Encap package in CVS?
    6. How can I distribute default config files with my package?
    7. How can I ensure that my Perl modules will be found?
    8. Can I test for prerequisite packages if they're not installed in the same Encap tree?
  5. More Information
    1. Where can I find more information on Encap?

Part I: Introduction

  1. What's wrong with traditional Unix package management?
  2. The traditional way of installing free software under Unix has always been to install all packages into the same subdirectories under /usr/local: binaries go in /usr/local/bin, manpages go in /usr/local/man, and so on. This makes users' lives easier, since they need only to add one directory to their PATH and MANPATH to access all of the installed software, but it can be very difficult for the system administrator to separate out all of the files which are part of a given package when it's time to upgrade to a new version.

    Software maintenance would be much easier if each package was installed in its own directory, but that would greatly complicate things for the users because they would need to list a large number of directories in their PATH. It would also cause complications when copying software from another Unix system, since the use of /usr/local is a well-established convention.

    Encap is an attempt to solve this dilemma.

    Back

  3. What is Encap?
  4. Encap is a package management system for handling third-party package management on a Unix system. Through the magic of symbolic links, it allows you to install each package in its own directory, but still have everything accessible via the traditional location in /usr/local.

    Back

  5. How does Encap work?
  6. When you install an Encap package, the files are placed in their own subdirectory, usually under /usr/local/encap. For example, if you install GNU sed version 3.02, the following files will be included:
    /usr/local/encap/sed-3.02/bin/sed
    /usr/local/encap/sed-3.02/man/man1/sed.1
    
    Once these files have been installed, the Encap package manager will create the following symlinks:
    /usr/local/bin/sed -> ../encap/sed-3.02/bin/sed
    /usr/local/man/man1/sed.1 -> ../../encap/sed-3.02/man/man1/sed.1
    
    The normal user will have /usr/local/bin in his PATH and /usr/local/man in his MANPATH, so he will not even know that the Encap system is being used.

    Back

  7. What are the advantages of Encap?
  8. Encap has the following advantages:

    Back

  9. What does Encap not do?
  10. Encap is not intended to:

    Back

  11. Encap Terminology
  12. Encap Package
    A package which has been created in Encap format. Often referred to simply as an "Encap", as in "Do you have an Encap of emacs I could copy?"
    Encap Source Directory
    The tree under which Encap packages are installed on the system, usually /usr/local/encap.
    Package Directory
    The directory in which a given Encap package is installed, usually /usr/local/encap/pkgname-version.
    Encap Tree
    General term for the Encap Source Directory and all packages installed under it.
    Encap Target Directory
    The central location referred to by users where symlinks are maintained, usually /usr/local.
    Encap Package Manager
    The program used to create the links from the Encap target directory into the Encap tree. See below for more information.
    Encapping
    Creating an Encap package, as in "I'm Encapping emacs."

    Back

  13. History of Encap
  14. Date Event
    1992-1994 Early development of Encap at various research labs at UIUC by Chuck Thompson, Steve March, Doug Bogia, and Jason Wessel.
    1995 Jason Wessel does initial development of encap.pl, first popular Encap package manager.
    1996-1997 Encap spreads to CCSO and is deployed on UIArchive, the Staff/Student Cluster, and EWS.
    Brian Swetland writes encapper.c, which operates on individual packages rather than batch mode like encap.pl.
    1998 Mark Roth develops epkg, which combines features from both encap.pl and encapper, as well as adding many extentions to the classic Encap package format.
    1999 The Encap Specification is written to formally define the Encap 2.0 package format.
    2002 The Encap 2.1 specification is written.

    Back


Part II: Using epkg, the Encap Package Manager

  1. How do I install epkg?
  2. The first step is to download epkg. It's available from:
    http://www.encap.org/epkg/
    There are two ways to install epkg:

    Back

  3. How do I install an Encap package?
  4. In the context of Encap, "installing" a package means creating the necessary symlinks from the Encap Target Directory to the files in the package.

    For example, to install an Encap of GNU sed-3.02, you would first download the Encap package and extract it into /usr/local/encap. This will create the following files:

    /usr/local/encap/sed-3.02/bin/sed
    /usr/local/encap/sed-3.02/man/man1/sed.1
    
    To install these files, run this command:
    epkg sed-3.02
    
    This will create the following symlinks:
    /usr/local/bin/sed -> ../encap/sed-3.02/bin/sed
    /usr/local/man/man1/sed.1 -> ../../encap/sed-3.02/man/man1/sed.1
    
    Back

  5. How does epkg handle multiple versions of a package?
  6. In order to ease the upgrade processes, epkg attempts to automatically handle multiple versions of the same package.

    For example, say that you already have the package sed-2.0 installed, but you want to upgrade to sed-3.02. After you extract the sed-3.02 files into /usr/local/encap, you should have these files:

    /usr/local/encap/sed-2.0/bin/sed
    /usr/local/encap/sed-2.0/man/man1/sed.1
    /usr/local/encap/sed-3.02/bin/sed
    /usr/local/encap/sed-3.02/man/man1/sed.1
    /usr/local/bin/sed -> ../encap/sed-2.0/bin/sed
    /usr/local/man/man1/sed.1 -> ../../encap/sed-2.0/man/man1/sed.1
    
    With epkg's versioning capability, you can simply say epkg sed. That will cause epkg to scan the Source Directory, find both versions of sed, and determine which one is the newest. Then it will remove the symbolic links for all of the old versions and install the symbolic links for the newest version automaticly.

    Note: Because there's no standard way to do version numbering, epkg uses fuzzy logic to determine which version is newer. It almost always guesses right, but when it doesn't you can use the -S flag to tell it to operate only on a specific package. In the example above, this would be epkg -Sr sed-2.0; epkg -S sed-3.02.

    Back

  7. Does epkg extract package archives?
  8. In order to save time, epkg has the ability to extract a package archive automatically during the installation process. Instead of extracting the archive manually into /usr/local/encap before invoking epkg, you can simply use something like this:
    epkg /scratch/sed-3.02.tar.gz
    
    This will tell epkg to extract the package archive into the Encap Source Directory before installing the package.

    Note: If epkg was built without libtar and/or zlib support, you will need to have your system's tar and/or gzip binaries in your PATH when you invoke epkg.

    Back

  9. Can epkg download package archives?
  10. If epkg was built with libfget and libcurl support, it will be able to automatically download packages from an FTP or HTTP server. For example, say that you want to update packages on your system from the Encap Archive. First, set this environment variable:
    ENCAP_UPDATE_PATH="http://www.encap.org/search/search.fcgi?results_only=yes&no_detail_links=on&search_string=all&search_platform=%p"
    
    Then, to automatically upgrade all packages to their latest versions, use this command:
    epkg -u
    
    If you want to upgrade only the package GNU sed, use this:
    epkg -u sed
    
    Back

  11. How does epkg handle filename conflicts?
  12. Occassionally, a package installation will fail because of a conflict with a file that already exists in the target directory. By default, epkg will only remove the pre-existing file if both of these conditions are met:

    Conversely, here are some situations in which epkg will not remove the pre-existing file:

    If you encounter one of these situations and you want to force epkg to remove the pre-existing file anyway, you can use the -f flag.

    Back

  13. How do I remove an Encap package?
  14. In the context of Encap, "removing" a package means to remove the symlinks from the Encap Target Directory to the files in the package. Please note that the files in the Encap tree are not removed when the package is removed, so they need to be removed manually when the administrator wishes to reclaim disk space.

    For example, to remove an Encap of GNU sed-3.02, execute this command:

    epkg -r sed-3.02
    
    This will remove the following symlinks:
    /usr/local/bin/sed -> ../encap/sed-3.02/bin/sed
    /usr/local/man/man1/sed.1 -> ../../encap/sed-3.02/man/man1/sed.1
    
    Back

  15. What about dangling symlinks?
  16. Sometimes, packages are accidentally deleted from the Encap Source Directory without using epkg to remove the corresponding symlinks from the Encap Target Directory. In order to clean up any dangling symlinks in the Encap Target Directory, you can use this:
    epkg -c
    
    Back

  17. How can I transition to using Encap slowly?
  18. epkg allows you to specify a list of package names which will be automatically overridden if they are in conflict with a package not on the override list. This feature is intended to ease the transition to Encap by allowing the system administrator to move the entire non-Encap /usr/local into an Encap package, and replacing it with Encap packages little by little.

    By default, there is only one package on the override list, old_local. To use this package as a way to transition to Encap, do something like this:

    cd /usr/local
    mkdir -p encap/old_local
    for n in `ls -a | egrep -v 'src|etc|encap|^\.{1,2}$|lost\+found'`; do \
    	mv ${n} encap/old_local; \
    done
    epkg old_local
    
    Back

Part III: Creating Encap Packages

  1. How do I create an Encap package for a given piece of software?
  2. In order to create an Encap package of a given piece of software, you must build the software so that it installs itself in its own Package Directory. For packages which use GNU autoconf, something like this usually works:
    ./configure --prefix=/usr/local/encap/pkgspec
    
    For software which doesn't use autoconf, you will need to edit the Makefile by hand.

    Once the software has been compiled and has installed its files into the Package Directory, you can create the Encap package with this command:

    mkencap pkgspec
    
    By default, mkencap will create a new encapinfo file in the Package Directory, and create a .tar.gz archive of the Package Directory which can be installed on other machines which use Encap. For more information, see the mkencap man page.

    Back

  3. Can I add a postinstall script to the package?
  4. An Encap 2.x package may include any of the following four package scripts in the Package Directory:

    Script Description
    preinstall Executed before installation.
    postinstall Executed after installation.
    preremove Executed before removal.
    postremove Executed after removal.

    If a preprocessing script returns a non-zero exit code, the package manager must immediately abort processing the package. A postprocessing script will only be executed if at least one link has been successfully created or removed, as appropriate, during the processing of the package.

    Package scripts must adhere to the following guidelines:

    1. A package's scripts may not modify system configuration files that cause it to be run when the system reboots (such as /etc/inetd.conf or any rc script) unless it has installed all data files that it needs to start properly. This is necessary to prevent the system from being unusable after a reboot.
    2. A package script which modifies or replaces pre-existing files must always save a copy with the suffix .pre.pkgspec, where pkgspec is the pkgspec of the package being installed.
    3. A package script which installs configuration files outside of the Encap tree must copy the files to that location, rather than create symlinks into the Encap tree. This prevents problems which can be arise when the Encap tree is unavailable at boot.
    4. A package script which installs configuration files outside of the Encap tree must not replace pre-existing customized versions, except where necessary to adhere to the previous guidelines. An example of a valid exception is when a package is upgraded to a version which changes the configuration syntax.
    5. A package script must react properly when it is invoked multiple times. For example, if a postinstall script intends to add an entry to /etc/services, it must not do so if the entry already exists.
    6. Package scripts may not prompt the user for input. A package script's stdout and stdin may be redirected somewhere other than the terminal. However, a script's stderr must always be sent to the terminal so that the user is informed of error conditions.

    Back

  5. Do all files in the package directory get linked in?
  6. By default, yes. You can override this behavior to specify certain files that do not get linked in. For Encap 2.x packages, this is done via the exclude directive in the encapinfo file. For Encap 1.x packages, this is done by means of an encap.exclude file. See the Encap Specification for more information.

    Back

  7. How do Encap packages interact with each other?
  8. The main advantage of encap is that the Encap target acts as an abstraction of the individual package directories. This is extremely critical when it comes time to upgrade a package, because you install the new version of the package in a different directory than the old one (since the version number is encoded in the name of the package directory). When this happens, you don't want to have to change every other program on the system that interacts with the package you're upgrading.

    To facilitate this, an Encap package may look for its own files directly in its package directory, but it should look for all other files in the Encap target (/usr/local). This is an extremely critical thing to keep in mind when building Encap packages.

    For example, say you have a system with perl 5.003 installed and you're installing a new package which contains some perl scripts. If you tell it the path to perl in perl's package directory, the perl scripts will have #!/usr/local/encap/perl-5.003/bin/perl at the top. If you later want to upgrade to perl 5.004, all of these scripts will have to be changed. But if they all say #!/usr/local/bin/perl, the upgrade should be completely transparent to them.

    Back

  9. What about local customizations of Encap packages?
  10. Many larger applications have a "site" directory for local customizations and scripts. If this "site" directory is in the package directory, you'll lose all of your local changes when you upgrade to a new version of the application, since the new version will be in a different package directory.

    To avoid this when Encapping these applications, it's important to configure them to look directly in the Encap target for their "site" directory instead of in their package directory. Good examples of this are perl's site_perl directory and emacs's site-lisp directory. Let's look at the latter example in detail.

    Say you have a system with emacs 19.34 installed, and all of your locally installed elisp code is in emacs's package directory (/usr/local/encap/emacs-19.34/share/emacs/site-lisp). When it comes time to upgrade to emacs 20.3, you can't remove the old version of emacs without losing all of your elisp files. To solve this problem, emacs should be told to look for its site-lisp files in the Encap target (/usr/local/share/emacs/site-lisp), and any locally-installed elisp files should be placed in their own Encap packages and linked into the Encap target. An example of this is the elisp file which comes with the ispell Encap package, which is installed as /usr/local/share/emacs/site-lisp/ispell.el -> /usr/local/encap/ispell-3.1.20/share/emacs/site-lisp/ispell.el.

    Back

  11. How can I create Encap packages of Perl modules?
  12. There is an excellent tool for automatically creating Encap packages of Perl modules. The tool is called cpanencap and is available from:
    http://www.encap.org/cpanencap/
    See the enclosed documentation for more information.

    Back


Part IV: Relocatable Packages and Other Tricks

  1. Why would I want to create a relocatable Encap package?
  2. While the Encap system is usually used for maintaining freeware packages in /usr/local, there are often good reasons for installing Encap packages in other locations:

    The Encap system doesn't have any built-in limitations with respect to directories. It allows you to specify any arbitrary directories as the Encap source and Encap target areas. However, individual packages often use hard-coded paths to find auxiliury files, and these paths are no longer correct when you install the software in a different location.

    It is not always possible to build a relocatable package. For example, packages that contain binaries that hard-code paths at compile time cannot be changed at install time. However, for packages that contain only scripts, it is often quite useful.

    Back

  3. How can I create a relocatable Encap package?
  4. The basic idea of this approach is to replace hard-coded paths with a special token that can be automatically replaced with the appropriate path. To explain this, let's look at a specific example.

    Let's say that the package foo-1.0 contains the following files:

    bin/foo.pl
    share/foo/foo.dat
    
    The foo.pl script looks like this:
    #!/usr/local/bin/perl
    
    $data = `cat /usr/local/share/foo/foo.dat`;
    print "$data\n";
    
    If the package is installed into the Encap target directory /usr/local, it will work fine. However, if it's installed in the Encap target directory /home/user, it won't be able to find the foo.dat file.

    To solve this problem, we can do the following:

    1. Replace /usr/local/share/foo/foo.dat with @ENCAP_TARGET@/share/foo/foo.dat in foo.pl and rename it to foo.pl.in.

    2. Add these commands to the package's preinstall script:
      echo "Creating bin/foo.pl...";
      sed -e "s|@ENCAP_TARGET@|${ENCAP_TARGET}|g" \
      	"${ENCAP_SOURCE}/${ENCAP_PKGNAME}/bin/foo.pl.in" \
      	> "${ENCAP_SOURCE}/${ENCAP_PKGNAME}/bin/foo.pl";
      
      Note that the ENCAP_SOURCE, ENCAP_TARGET, and ENCAP_PKGNAME variables are all set to the correct values by the package manager so that they can be used by the package scripts. Therefore, this preinstall code will dynamically create the foo.pl file from the foo.pl.in file using the correct path at the time that the package is installed.

    3. Add these commands to the package's postremove script:
      echo "Removing bin/foo.pl...";
      rm -f "{ENCAP_SOURCE}/${ENCAP_PKGNAME}/bin/foo.pl";
      
      This cleans up the foo.pl file when the package is removed. This isn't strictly necessary, but it is convenient for two reasons. First, it saves disk space. Second, it restores the package to a "pristine" state so that you can create a tar archive of it without needing to distribute a second copy of each file. (This means that you should remove the package with epkg -r before you create the tar archive.)

    4. Add this to the package's encapinfo file:
      exclude bin/foo.pl.in
      
      This tells the package manager not to create a symlink for the .in version of the file, since all you need is the dynamically-generated copy. Again, this isn't strictly necessary, but it does keep things clean.

    Back

  5. How can I process all of my .in files at once?
  6. If you have many .in files in a package, you can dynamically instantiate them all at once with a shell loop, like this:
    for INFILE in `find ${ENCAP_SOURCE}/${ENCAP_PKGNAME} -name '*.in'`; do
      OUTFILE=`echo ${INFILE} | sed 's|\.in$||'`;
      if [ ! -f ${OUTFILE} ]; then
        echo "Creating: ${OUTFILE}";
        sed -e "s|@ENCAP_TARGET@|${ENCAP_TARGET}|g" \
          < ${INFILE} > ${OUTFILE};
        if [ -x ${INFILE} ]; then
          chmod a+x ${OUTFILE};
        fi
      fi
    done
    
    Back

  7. What if my Encap package is also my source distribution?
  8. When maintaining Encap packages containing only scripts and data, such as Perl scripts, the Encap package usually doubles as the "source" distribution. As such, it's often convenient to distribute informational files (such as a ChangeLog) with the package. Because Encap does not process files that are directly in the package directory, you can put these files there. For example, package foo-1.0 might look like this:
    foo-1.0/
      COPYRIGHT
      ChangeLog
      TODO
      preinstall
      postremove
      bin/
        foo.pl.in
      share/
        foo/
          foo.dat
    
    Back

  9. How can I keep my Encap package in CVS?
  10. If you're using CVS to maintain a relocatable package, you will probably have bin/foo.pl.in checked in, but not bin/foo.pl, since it gets generated dynamically. In order to avoid spurious warnings, you can add foo.pl to the bin/.cvsignore file. (The .cvsignore file itself can also be checked into CVS.)

    Back

  11. How can I distribute default config files with my package?
  12. As per section 2.6.1 of the Encap 2.1 specification, packages should make allowances for maintaining locally-modified config files when a package gets upgraded. Packages should not allow the package manager to install files in the etc subdirectory of the Encap target area. Instead, the postinstall script should be used to copy the config files directly into the etc subdirectory of the Encap target area, and the software should know to look there directly (as opposed to looking in the package directory).

    For example, if the package foo-1.0 contained a config file called etc/foo.conf, we would need to do the following:

    1. Add this line to the encapinfo file:
      exclude etc
      
    2. Add these commands to the postinstall script:
      if [ ! -f "${ENCAP_TARGET}/etc/foo.conf" ]; then
        echo "Installing ${ENCAP_TARGET}/etc/foo.conf...";
        cp "${ENCAP_SOURCE}/${ENCAP_PKGNAME}/etc/foo.conf" \
          "${ENCAP_TARGET}/etc/foo.conf";
      fi
      
    3. Make sure that foo.pl.in looks for foo.conf as @ENCAP_TARGET@/etc/foo.conf.

    Back

  13. How can I ensure that my Perl modules will be found?
  14. If a relocatable package has a central perl module that you use in multiple scripts, you can use the same technique to make sure that your scripts can find it. For example, let's say that package foo-1.0 includes this perl module:
    lib/site_perl/Foo.pm
    
    You can modify the top of foo.pl.in to look like this:
    #!/usr/local/bin/perl -I@ENCAP_TARGET@/lib/site_perl
    
    use Foo;
    
    That will ensure that the right path is always added to @INC.

    Back

  15. Can I test for prerequisite packages if they're not installed in the same Encap tree?
  16. Unfortunately, there's no way to do this right now. One of Encap's current limitations is that prerequisites are only understood within the same Encap tree. So, for example, if the perl Encap package is installed in /usr/local and you're installing package foo-1.0 in /home/user, the installation will fail if you list perl as a prerequisite package in the encapinfo file.

    This limitation should be addressed in the future, but for now you can work around this in one of the following ways:

    Back


Part V: More Information

  1. Where can I find more information on Encap?
  2. The best source for information on the Encap system is the Encap Archive at http://www.encap.org/. There, you will find the latest version of this FAQ, information on the history of Encap, a large selection of precompiled Encap packages, and pointers to the Encap package manager, epkg.

    Back


encap@encap.org