update for rename of docs/user/DebugKernelCrashDumps.mdwn to docs/user/list/DebugKern...

[ikiwiki.git] / docs / handbook / handbook-pkgsrc-sourcetree-using.mdwn

## Using the pkgsrc® Source Tree 

The following sections provide basic instructions on using the pkgsrc source tree to install or remove programs from your system.

### Obtaining the pkgsrc Source Tree 



Before you can install pkgsrc® packages from source, you must first obtain the pkgsrc source tree--which is essentially a set of `Makefiles`, patches, and description files placed in `/usr/pkgsrc`.


#### CVS 

The primary method to obtain and keep your pkgsrc collection up to date is by using  **CVS**. This is a quick method for getting the pkgsrc collection using  **CVS** .



Run `cvs`:

    # cd /usr
    # cvs -d anoncvs@anoncvs.us.netbsd.org:/cvsroot co pkgsrc

 
Running the following command later will download and apply all the recent changes to your source tree.

    # cd /usr/pkgsrc
    # cvs up

  
#### Quick & Dirty Way 



As of the 1.10 release, you can use the `/usr/Makefile` to checkout & update the pkgsrc tree quickly.



as root:



    # cd /usr
    # make pkgsrc-checkout

to checkout, or


    # cd /usr
    # make pkgsrc-update

to update.



Please do edit the makefile to use an appropriately speedy CVS mirror for your location and to reduce load on the main pkgsrc cvs server.

### Installing Packages from Source

The first thing that should be explained when it comes to the source tree is what is actually meant by a ***skeleton***. In a nutshell, a source skeleton is a minimal set of files that tell your DragonFly system how to cleanly compile and install a program. Each source skeleton should include:

* A `Makefile`. The `Makefile` contains various statements that specify how the application should be compiled and where it should be installed on your system.


* A `distinfo` file. This file contains information about the files that must be downloaded to build the port and their checksums, to verify that files have not been corrupted during the download using [md5(1)](http://leaf.dragonflybsd.org/cgi/web-man?command#md5&section1).


* A `files` directory. This directory contains the application specific files that are needed for the programs appropriate run-time configuration.

  This directory may also contain other files used to build the port.


* A `patches` directory. This directory contains patches to make the program compile and install on your DragonFly system. Patches are basically small files that specify changes to particular files. They are in plain text format, and basically say ***Remove line 10*** or ***Change line 26 to this ...***. Patches are also known as ***diffs*** because they are generated by the [diff(1)](http://leaf.dragonflybsd.org/cgi/web-man?command#diff&section1) program.


* A `DESCR` file. This is a more detailed, often multiple-line, description of the program.


* A `PLIST` file. This is a list of all the files that will be installed by the port. It also tells the pkgsrc system what files to remove upon deinstallation.



Some pkgsrc source skeletons have other files, such as `MESSAGE`. The pkgsrc system uses these files to handle special situations. If you want more details on these files, and on pkgsrc in general, check out [The pkgsrc guide](http://www.netbsd.org/Documentation/pkgsrc/), available at the [NetBSD website](http://www.netbsd.org/).



Now that you have enough background information to know what the pkgsrc source tree is used for, you are ready to install your first compiled package. There are two ways this can be done, and each is explained below.



Before we get into that, however, you will need to choose an application to install. There are a few ways to do this, with the easiest method being the [pkgsrc listing on Joerg Sonnenberger's web site](ftp://packages.stura.uni-rostock.de/pkgsrc-current/DragonFly/RELEASE/i386/All/). You can browse through the packages listed there.



Another way to find a particular source tree is by using the pkgsrc collection's built-in search mechanism. To use the search feature, you will need to be in the `/usr/pkgsrc` directory. Once in that directory, run `bmake search key=program-name` where `program-name` is the name of the program you want to find. This searches packages names, comments, descriptions and dependencies and can be used to find packages which relate to a particular subject if you don't know the name of the program you are looking for. For example, if you were looking for `apache2`:



    

    # cd /usr/pkgsrc
    # bmake search key="apache2"
    Extracting complete dependency database.  This may take a while...
    ....................................................................................................
    100
    ....................................................................................................
    200
    <Snip />
    5800
    ....................................................................................................
    5900
    .................................................................................................Reading database file
    Flattening dependencies
    Flattening build dependencies
    Generating INDEX file
    Indexed 5999 packages
    <Snip />
    Pkg:    apache-2.2.6nb2
    Path:   www/apache22
    Info:   Apache HTTP (Web) server, version 2
    Maint:  tron@NetBSD.org
    Index:  www
    B-deps: perl>#5.0 apr-util>1.2.8 apr>=1.2.8 libtool-base>=1.5.22nb1 pkg-config>=0.19 expat>=1.95.7 gmake>=3.78 gettext-lib>=0.14.5 
    {gettext-tools>=0.14.5,gettext>=0.10.36<0.14.5} expat>=1.95.4 expat>=2.0.0nb1 apr-util>=1.2.8nb1
    R-deps: perl>#5.0 apr-util>1.2.8 apr>=1.2.8 expat>=1.95.7 expat>=1.95.4 expat>=2.0.0nb1 apr-util>=1.2.8nb1
    Arch:   any


The part of the output you want to pay particular attention to is the ***Path*** line, since that tells you where to find the source tree for the requested application. The other information provided is not needed in order to install the package, so it will not be covered here.

The search string is case-insensitive. Searching for ***APACHE*** will yield the same results as searching for ***apache***.



 **Note:** It should be noted that ***Extracting [the] complete dependency database*** does indeed take a while.



 **Note:** You must be logged in as `root` to install packages.



Now that you have found an application you would like to install, you are ready to do the actual installation. The source package includes instructions on how to build source code, but does not include the actual source code. You can get the source code from a CD-ROM or from the Internet. Source code is distributed in whatever manner the software author desires. Frequently this is a tarred and gzipped file, but it might be compressed with some other tool or even uncompressed. The program source code, whatever form it comes in, is called a ***distfile***. You can get the distfile from a CD-ROM or from the Internet.



 **Warning:** Before installing any application, you should be sure to have an up-to-date source tree and you should check http://www.pkgsrc.org/ for security issues related to your port.



A security vulnerabilities check can be automatically done by  **audit-packages**  before any new application installation. This tool can be found in the pkgsrc collection ([security/audit-packages](http://pkgsrc.se/security/audit-packages)). Consider running `auditpackages -d` before installing a new package, to fetch the current vulnerabilities database. A security audit and an update of the database will be performed during the daily security system check. For more informations read the audit-packages and [periodic(8)](http://leaf.dragonflybsd.org/cgi/web-man?command#periodic&section8) manual pages.



 **Note:** It should be noted that the current setup of DragonFly requires the use of `bmake` instead of `make`. This is because the current version of make on DragonFly does not support all the parameters that NetBSD's does.



 **Note:** You can save an extra step by just running `bmake install` instead of `bmake` and `bmake install` as two separate steps.



 **Note:** Some shells keep a cache of the commands that are available in the directories listed in the `PATH` environment variable, to speed up lookup operations for the executable file of these commands. If you are using one of these shells, you might have to use the `rehash` command after installing a package, before the newly installed commands can be used. This is true for both shells that are part of the base-system (such as `tcsh`) and shells that are available as packages (for instance, [shells/zsh](http://pkgsrc.se/shells/zsh)).



#### Installing Packages from the Internet 



As with the last section, this section makes an assumption that you have a working Internet connection. If you do not, you will need to put a copy of the distfile into `/usr/pkgsrc/distfiles` manually.



Installing a package from the Internet is done exactly the same way as it would be if you already had the distfile. The only difference between the two is that the distfile is downloaded from the Internet on demand.



Here are the steps involved:



    

    # cd /usr/pkgsrc/chat/ircII
    # bmake install clean
    => ircii-20040820.tar.bz2 doesn't seem to exist on this system.
    => Attempting to fetch ircii-20040820.tar.bz2 from ftp://ircii.warped.com/pub/ircII/.
    => [559843 bytes]
    [FTP transfer snipped]
    150 Opening BINARY mode data connection for 'ircii-20040820.tar.bz2' (559843 bytes).
    100% |***************************************|   550 KB  110.34 KB/s   00:00 ETA
    226 Transfer complete.
    [FTP transfer snipped]
    221 Thank you for using the FTP service on bungi.sjc.warped.net.
    => Checksum SHA1 OK for ircii-20040820.tar.bz2.
    => Checksum RMD160 OK for ircii-20040820.tar.bz2.
    work -> /usr/obj/pkgsrc/chat/ircII/work
    ##=> Extracting for ircII-20040820
    #########################################################################=
    The supported build options for this package are:

    socks4 socks5

    You can select which build options to use by setting PKG_DEFAULT_OPTIONS
    or the following variable.  Its current value is shown:

    
    PKG_OPTIONS.ircII (not defined)

    #########################################################################=
    #########################################################################=
    The following variables will affect the build process of this package,
    ircII-20040820.  Their current value is shown below:

    USE_INET6 = YES

    You may want to abort the process now with CTRL-C and change their value
    before continuing.  Be sure to run `/usr/pkg/bin/bmake clean' after
    the changes.
    #########################################################################=
    ##=> Patching for ircII-20040820
    ##=> Applying pkgsrc patches for ircII-20040820
    ##=> Overriding tools for ircII-20040820
    ##=> Creating toolchain wrappers for ircII-20040820
    ##=> Configuring for ircII-20040820
    ...
    [configure output snipped]
    ...
    ##=>  Building for ircII-20040820
    ...
    [compilation output snipped]
    ...
    ##=>  Installing for ircII-20040820
    ...
    [installation output snipped]
    ...
    ##=> [Automatic manual page handling]
    ##=> Registering installation for ircII-20040820
    ##=> Cleaning for ircII-20040820
    #





As you can see, the only difference are the lines that tell you where the system is fetching the package's distfile from.



The pkgsrc system uses [ftp(1)](http://leaf.dragonflybsd.org/cgi/web-man?command#ftp&section1) to download the files, which honors various environment variables, including `FTP_PASSIVE_MODE`, `FTP_PROXY`, and `FTP_PASSWORD`. You may need to set one or more of these if you are behind a firewall, or need to use an FTP/HTTP proxy. See [ftp(1)](http://leaf.dragonflybsd.org/cgi/web-man?command=ftp&section=1) for the complete list.



For users which cannot be connected all the time, the `bmake fetch` option is provided. Just run this command at the top level directory (`/usr/pkgsrc`) and the required files will be downloaded for you. This command will also work in the lower level categories, for example: `/usr/pkgsrc/net`. Note that if a package depends on libraries or other packages this will ***not*** fetch the distfiles of those packages as well.



 **Note:** You can build all the packages in a category or as a whole by running `bmake` in the top level directory, just like the aforementioned `bmake `fetch*** method. This is dangerous, however, as some applications cannot co-exist. In other cases, some packages can install two different files with the same filename.



In some rare cases, users may need to acquire the tarballs from a site other than the `MASTER_SITES` (the location where files are downloaded from). You can override the `MASTER_SORT`, `MASTER_SORT_REGEX` and `INET_COUNTRY` options either within the `/etc/mk.conf`.



 **Note:** Some packages allow (or even require) you to provide build options which can enable/disable parts of the application which are unneeded, certain security options, and other customizations. A few which come to mind are [`www/mozilla`](http://pkgsrc.se/www/mozilla), [`security/gpgme`](http://pkgsrc.se/security/gpgme), and [`mail/sylpheed-claws`](http://pkgsrc.se/mail/sylpheed-claws). To find out what build options the application you are installing requires type:



    

    # bmake show-options





 To change the build process, either change the values of PKG_DEFAULT_OPTIONS or PKG_OPTIONS.`***PackageName***` in `/etc/mk.conf` or on the commandline as so:



    

    # bmake PKG_OPTIONS.ircII="-ssl"





 An option is enabled if listed. It is disabled if it is prefixed by a minus sign.



#### Dealing with imake 



Some applications that use `imake` (a part of the X Window System) do not work well with `PREFIX`, and will insist on installing under `/usr/X11R6`. Similarly, some Perl ports ignore `PREFIX` and install in the Perl tree. Making these applications respect `PREFIX` is a difficult or impossible job.



### 4.5.3 Removing Installed Packages 



Now that you know how to install packages, you are probably wondering how to remove them, just in case you install one and later on decide that you installed the wrong program. We will remove our previous example (which was `ircII` for those of you not paying attention). As with installing packages, the first thing you must do is change to the package directory, `/usr/pkgsrc/chat/ircII`. After you change directories, you are ready to uninstall `ircII`. This is done with the `bmake deinstall` command:



    

    # cd /usr/pkgsrc/chat/ircII
    # make deinstall
    ##=>  Deinstalling for ircII-20040820


That was easy enough. You have removed `ircII` from your system. If you would like to reinstall it, you can do so by running `bmake reinstall` from the `/usr/pkgsrc/chat/ircII` directory.



The `bmake deinstall` and `bmake reinstall` sequence does not work once you have run `bmake clean`. If you want to deinstall a package after cleaning, use [pkg_delete(1)](http://leaf.dragonflybsd.org/cgi/web-man?command#pkg_delete&section1) as discussed in the [pkgsrc-using.html Pkgsrc section of the Handbook].



### Packages and Disk Space 



Using the pkgsrc collection can definitely eat up your disk space. For this reason you should always remember to clean up the work directories using the `bmake `clean*** option. This will remove the `work` directory after a package has been built, and installed. You can also remove the tar files from the `distfiles` directory, and remove the installed package when their use has delimited.



### 4.5.5 Upgrading Packages 



 **Note:** Once you have updated your pkgsrc collection, before attempting a package upgrade, you should check the `/usr/pkgsrc/UPDATING` file. This file describes various issues and additional steps users may encounter and need to perform when updating a port.



Keeping your packages up to date can be a tedious job. For instance, to upgrade a package you would go to the package directory, build the package, deinstall the old package , install the new package, and then clean up after the build. Imagine doing that for five packages, tedious right? This was a large problem for system administrators to deal with, and now we have utilities which do this for us. For instance the `pkg_chk` utility will do everything for you!



pkg_chk requires a few steps in order to work correctly. They are listed here.

    # pkg_chk -g # make initial list of installed packages
    # pkg_chk -r  # remove all packages that are not up to date and packages that depend on them
    # pkg_chk -a  # install all missing packages (use binary packages, this is the default
    # pkg_chk -as # install all missing packages (build from source)