Contributed by &a.jraynard;. The FreeBSD Ports collection allows you to compile and install a very wide range of applications with a minimum of effort. For all the hype about open standards, getting a program to work on different versions of Unix in the real world can be a tedious and tricky business, as anyone who has tried it will know. You may be lucky enough to find that the program you want will compile cleanly on your system, install itself in all the right places and run flawlessly “out of the box”, but this is unfortunately rather rare. With most programs, you will find yourself doing a fair bit of head-scratching, and there are quite a few programs that will result in premature greying, or even chronic alopecia... Some software distributions have attacked this problem by providing configuration scripts. Some of these are very clever, but they have an unfortunate tendency to triumphantly announce that your system is something you have never heard of and then ask you lots of questions that sound like a final exam in system-level Unix programming (Does your system's gethitlist function return a const pointer to a fromboz or a pointer to a const fromboz? Do you have Foonix style unacceptable exception handling? And if not, why not?). Fortunately, with the Ports collection, all the hard work involved has already been done, and you can just type make install and get a working program. The base FreeBSD system comes with a very wide range of tools and system utilities, but a lot of popular programs are not in the base system, for good reasons:- Programs that some people cannot live without and other people cannot stand, such as a certain Lisp-based editor. Programs which are too specialised to put in the base system (CAD, databases). Programs which fall into the “I must have a look at that when I get a spare minute” category, rather than system-critical ones (some languages, perhaps). Programs that are far too much fun to be supplied with a serious operating system like FreeBSD ;-) However many programs you put in the base system, people will always want more, and a line has to be drawn somewhere (otherwise FreeBSD distributions would become absolutely enormous). Obviously it would be unreasonable to expect everyone to port their favourite programs by hand (not to mention a tremendous amount of duplicated work), so the FreeBSD Project came up with an ingenious way of using standard tools that would automate the process. Incidentally, this is an excellent illustration of how “the Unix way” works in practice by combining a set of simple but very flexible tools into something very powerful. Programs are typically distributed on the Internet as a consisting of a Makefile and the source code for the program and usually some instructions (which are unfortunately not always as instructive as they could be), with perhaps a configuration script. The standard scenario is that you FTP down the tarball, extract it somewhere, glance through the instructions, make any changes that seem necessary, run the configure script to set things up and use the standard make program to compile and install the program from the source. FreeBSD ports still use the tarball mechanism, but use a to hold the "knowledge" of how to get the program working on FreeBSD, rather than expecting the user to be able to work it out. They also supply their own customised , so that almost every port can be built in the same way. If you look at a port skeleton (either on your FreeBSD system or the FTP site) and expect to find all sorts of pointy-headed rocket science lurking there, you may be disappointed by the one or two rather unexciting-looking files and directories you find there. (We will discuss in a minute how to go about ). “How on earth can this do anything?” I hear you cry. “There is no source code there!” Fear not, gentle reader, all will become clear (hopefully). Let's see what happens if we try and install a port. I have chosen ElectricFence, a useful tool for developers, as the skeleton is more straightforward than most. If you are trying this at home, you will need to be root. &prompt.root; cd /usr/ports/devel/ElectricFence &prompt.root; make install >> Checksum OK for ElectricFence-2.0.5.tar.gz. ===> Extracting for ElectricFence-2.0.5 ===> Patching for ElectricFence-2.0.5 ===> Applying FreeBSD patches for ElectricFence-2.0.5 ===> Configuring for ElectricFence-2.0.5 ===> Building for ElectricFence-2.0.5 [lots of compiler output...] ===> Installing for ElectricFence-2.0.5 ===> Warning: your umask is "0002". If this is not desired, set it to an appropriate value and install this port again by ``make reinstall''. install -c -o bin -g bin -m 444 /usr/ports/devel/ElectricFence/work/ElectricFence-2.0.5/libefence.a /usr/local/lib install -c -o bin -g bin -m 444 /usr/ports/devel/ElectricFence/work/ElectricFence-2.0.5/libefence.3 /usr/local/man/man3 ===> Compressing manual pages for ElectricFence-2.0.5 ===> Registering installation for ElectricFence-2.0.5 To avoid confusing the issue, I have completely removed the build output. If you tried this yourself, you may well have got something like this at the start:- &prompt.root; make install >> ElectricFence-2.0.5.tar.gz doesn't seem to exist on this system. >> Attempting to fetch from ftp://ftp.doc.ic.ac.uk/Mirrors/sunsite.unc.edu/pub/Linux/devel/lang/c/. The make program has noticed that you did not have a local copy of the source code and tried to FTP it down so it could get the job done. I already had the source handy in my example, so it did not need to fetch it. Let's go through this and see what the make program was doing. Locate the source code If it is not available locally, try to grab it from an FTP site. Run a test on the tarball to make sure it has not been tampered with, accidentally truncated, downloaded in ASCII mode, struck by neutrinos while in transit, etc. Extract the tarball into a temporary work directory. Apply any needed to get the source to compile and run under FreeBSD. Run any configuration script required by the build process and correctly answer any questions it asks. (Finally!) Compile the code. Install the program executable and other supporting files, man pages, etc. under the /usr/local hierarchy, where they will not get mixed up with system programs. This also makes sure that all the ports you install will go in the same place, instead of being flung all over your system. Register the installation in a database. This means that, if you do not like the program, you can cleanly all traces of it from your system. Scroll up to the make output and see if you can match these steps to it. And if you were not impressed before, you should be by now! There are two ways of getting hold of the FreeBSD port for a program. One requires a , the other involves using an If you answered yes to the question “Do you want to link the ports collection to your CDROM” during the FreeBSD installation, the initial setting up will already have been done for you. If not, make sure the FreeBSD CDROM is in the drive and mounted on, say, /cdrom. Then do &prompt.root; mkdir /usr/ports &prompt.root; cd /usr/ports &prompt.root; ln -s /cdrom/ports/distfiles distfiles to enable the ports make mechanism to find the tarballs (it expects to find them in /usr/ports/distfiles, which is why we sym-linked the CDROM's tarball directory to that directory). Now, suppose you want to install the gnats program from the databases directory. Here is how to do it:- &prompt.root; cd /usr/ports &prompt.root; mkdir databases &prompt.root; cp -R /cdrom/ports/databases/gnats databases &prompt.root; cd databases/gnats &prompt.root; make install Or if you are a serious database user and you want to compare all the ones available in the Ports collection, do &prompt.root; cd /usr/ports &prompt.root; cp -R /cdrom/ports/databases . &prompt.root; cd databases &prompt.root; make install (yes, that really is a dot on its own after the cp command and not a mistake. It is Unix-ese for “the current directory”) and the ports make mechanism will automatically compile and install all the ports in the databases directory for you! If you do not like this method, here is a completely different way of doing it:- Create a “link tree” to it using the lndir1 command that comes with the XFree86 distribution. Find a location with some free space, create a directory there and then cd to it. Then invoke the lndir1 command with the full pathname of the ports directory on the CDROM as the first argument and . (the current directory) as the second. This might be, for example, something like: &prompt.root; lndir /cdrom/ports . Then you can build ports directly off the CDROM by building them in the link tree you have created. Note that there are some ports for which we cannot provide the original source in the CDROM due to licensing limitations. In that case, you will need to look at the section on If you do not have a CDROM, or you want to make sure you get the very latest version of the port you want, you will need to download the for the port. Now this might sound like rather a fiddly job full of pitfalls, but it is actually very easy. The key to it is that the FreeBSD FTP server can create on-the-fly for you. Here is how it works, with the gnats program in the databases directory as an example (the bits in square brackets are comments. Do not type them in if you are trying this yourself!):- &prompt.root; cd /usr/ports &prompt.root; mkdir databases &prompt.root; cd databases &prompt.root; ftp ftp.freebsd.org [log in as `ftp' and give your email address when asked for a password. Remember to use binary (also known as image) mode!] > cd /pub/FreeBSD/ports/databases > get gnats.tar [tars up the gnats skeleton for us] > quit &prompt.root; tar xf gnats.tar [extract the gnats skeleton] &prompt.root; cd gnats &prompt.root; make install [build and install gnats] What happened here? We connected to the FTP server in the usual way and went to its databases sub-directory. When we gave it the command get gnats.tar, the FTP server up the gnats directory for us. We then extracted the gnats skeleton and went into the gnats directory to build the port. As we explained , the make process noticed we did not have a copy of the source locally, so it fetched one before extracting, patching and building it. Let's try something more ambitious now. Instead of getting a single port skeleton, let's get a whole sub-directory, for example all the database skeletons in the ports collection. It looks almost the same:- &prompt.root; cd /usr/ports &prompt.root; ftp ftp.freebsd.org [log in as `ftp' and give your email address when asked for a password. Remember to use binary (also known as image) mode!] > cd /pub/FreeBSD/ports > get databases.tar [tars up the databases directory for us] > quit &prompt.root; tar xf databases.tar [extract all the database skeletons] &prompt.root; cd databases &prompt.root; make install [build and install all the database ports] With half a dozen straightforward commands, we have now got a set of database programs on our FreeBSD machine! All we did that was different from getting a single port skeleton and building it was that we got a whole directory at once, and compiled everything in it at once. Pretty impressive, no? If you expect to be installing many ports, it is probably worth downloading all the ports directories. A team of compulsive hackers who have forgotten to eat in a frantic attempt to make a deadline? Something unpleasant lurking in the FreeBSD attic? No, a skeleton here is a minimal framework that supplies everything needed to make the ports magic work. The most important component of a skeleton is the Makefile. This contains various statements that specify how the port should be compiled and installed. Here is the Makefile for ElectricFence:- # New ports collection makefile for: Electric Fence # Version required: 2.0.5 # Date created: 13 November 1997 # Whom: jraynard # # $Id$ # DISTNAME= ElectricFence-2.0.5 CATEGORIES= devel MASTER_SITES= ${MASTER_SITE_SUNSITE} MASTER_SITE_SUBDIR= devel/lang/c MAINTAINER= jraynard@freebsd.org MAN3= libefence.3 do-install: ${INSTALL_DATA} ${WRKSRC}/libefence.a ${PREFIX}/lib ${INSTALL_MAN} ${WRKSRC}/libefence.3 ${PREFIX}/man/man3 .include <bsd.port.mk> The lines beginning with a "#" sign are comments for the benefit of human readers (as in most Unix script files). DISTNAME specifies the name of the , but without the extension. CATEGORIES states what kind of program this is. In this case, a utility for developers. MASTER_SITES is the URL(s) of the master FTP site, which is used to retrieve the if it is not available on the local system. This is a site which is regarded as reputable, and is normally the one from which the program is officially distributed (in so far as any software is "officially" distributed on the Internet). MAINTAINER is the email address of the person who is responsible for updating the skeleton if, for example a new version of the program comes out. Skipping over the next few lines for a minute, the line .include <bsd.port.mk> says that the other statements and commands needed for this port are in a standard file called bsd.port.mk. As these are the same for all ports, there is no point in duplicating them all over the place, so they are kept in a single standard file. This is probably not the place to go into a detailed examination of how Makefiles work; suffice it to say that the line starting with MAN3 ensures that the ElectricFence man page is compressed after installation, to help conserve your precious disk space. The original port did not provide an install target, so the three lines from do-install ensure that the files produced by this port are placed in the correct destination. The file containing the for the port is called md5, after the MD5 algorithm used for ports checksums. It lives in a directory with the slightly confusing name of files. This directory can also contain other miscellaneous files that are required by the port and do not belong anywhere else. This directory contains the needed to make everything work properly under FreeBSD. This program contains three quite useful files:- COMMENT — a one-line description of the program. DESCR — a more detailed description. PLIST — a list of all the files that will be created when the program is installed. Oh. You can do one of four (4) things : Fix it yourself. Technical details on how ports work can be found in Gripe. This is done by e-mail only! Send such e-mail to the &a.ports; and please include the name/version of the port, where you got both the port source & distfile(s) from, and what the text of the error was. Forget it. This is the easiest for most — very few of the programs in ports can be classified as essential! Grab the pre-compiled package from a ftp server. The “master” package collection is on FreeBSD's FTP server in the packages directory, though check your local mirror first, please! These are more likely to work (on the whole) than trying to compile from source and a lot faster besides! Use the pkg_add1 program to install a package file on your system. Great! Please see the for detailed instructions on how to do this. Q. I thought this was going to be a discussion about modems??! A. Ah. You must be thinking of the serial ports on the back of your computer. We are using “port” here to mean the result of “porting” a program from one version of Unix to another. (It is an unfortunate bad habit of computer people to use the same word to refer to several completely different things). Q. I thought you were supposed to use packages to install extra programs? A. Yes, that is usually the quickest and easiest way of doing it. Q. So why bother with ports then? A. Several reasons:- The licensing conditions on some software distributions require that they be distributed as source code, not binaries. Some people do not trust binary distributions. At least with source code you can (in theory) read through it and look for potential problems yourself. If you have some local patches, you will need the source to add them yourself. You might have opinions on how a program should be compiled that differ from the person who did the package — some people have strong views on what optimisation setting should be used, whether to build debug versions and then strip them or not, etc. etc. Some people like having code around, so they can read it if they get bored, hack around with it, borrow from it (licence terms permitting, of course!) and so on. If you ain't got the source, it ain't software! ;-) Q. What is a patch? A. A patch is a small (usually) file that specifies how to go from one version of a file to another. It contains text that says, in effect, things like “delete line 23”, “add these two lines after line 468” or “change line 197 to this”. Also known as a “diff”, since it is generated by a program of that name. Q. What is all this about tarballs? A. It is a file ending in .tar or .tar.gz (with variations like .tar.Z, or even .tgz if you are trying to squeeze the names into a DOS filesystem). Basically, it is a directory tree that has been archived into a single file (.tar) and optionally compressed (.gz). This technique was originally used for Tape ARchives (hence the name tar), but it is a widely used way of distributing program source code around the Internet. You can see what files are in them, or even extract them yourself, by using the standard Unix tar program, which comes with the base FreeBSD system, like this:- &prompt.user; tar tvzf foobar.tar.gz &prompt.user; tar xzvf foobar.tar.gz &prompt.user; tar tvf foobar.tar &prompt.user; tar xvf foobar.tar Q. And a checksum? A. It is a number generated by adding up all the data in the file you want to check. If any of the characters change, the checksum will no longer be equal to the total, so a simple comparison will allow you to spot the difference. (In practice, it is done in a more complicated way to spot problems like position-swapping, which will not show up with a simplistic addition). Q. I did what you said for and it worked great until I tried to install the kermit port:- &prompt.root; make install >> cku190.tar.gz doesn't seem to exist on this system. >> Attempting to fetch from ftp://kermit.columbia.edu/kermit/archives/. Why can it not be found? Have I got a dud CDROM? A. The licensing terms for kermit do not allow us to put the tarball for it on the CDROM, so you will have to fetch it by hand — sorry! The reason why you got all those error messages was because you were not connected to the Internet at the time. Once you have downloaded it from any of the sites above, you can re-start the process (try and choose the nearest site to you, though, to save your time and the Internet's bandwidth). Q. I did that, but when I tried to put it into /usr/ports/distfiles I got some error about not having permission. A. The ports mechanism looks for the tarball in /usr/ports/distfiles, but you will not be able to copy anything there because it is sym-linked to the CDROM, which is read-only. You can tell it to look somewhere else by doing &prompt.root; make DISTDIR=/where/you/put/it install Q. Does the ports scheme only work if you have everything in /usr/ports? My system administrator says I must put everything under /u/people/guests/wurzburger, but it does not seem to work. A. You can use the PORTSDIR and PREFIX variables to tell the ports mechanism to use different directories. For instance, &prompt.root; make PORTSDIR=/u/people/guests/wurzburger/ports install will compile the port in /u/people/guests/wurzburger/ports and install everything under /usr/local. &prompt.root; make PREFIX=/u/people/guests/wurzburger/local install will compile it in /usr/ports and install it in /u/people/guests/wurzburger/local. And of course &prompt.root; make PORTSDIR=.../ports PREFIX=.../local install will combine the two (it is too long to fit on the page if I write it in full, but I am sure you get the idea). If you do not fancy typing all that in every time you install a port (and to be honest, who would?), it is a good idea to put these variables into your environment. Q. I do not have a FreeBSD CDROM, but I would like to have all the tarballs handy on my system so I do not have to wait for a download every time I install a port. Is there an easy way to get them all at once? A. To get every single tarball for the ports collection, do &prompt.root; cd /usr/ports &prompt.root; make fetch For all the tarballs for a single ports directory, do &prompt.root; cd /usr/ports/directory &prompt.root; make fetch and for just one port — well, I think you have guessed already. Q. I know it is probably faster to fetch the tarballs from one of the FreeBSD mirror sites close by. Is there any way to tell the port to fetch them from servers other than ones listed in the MASTER_SITES? A. Yes. If you know, for example, ftp.FreeBSD.ORG is much closer than sites listed in MASTER_SITES, do as following example. &prompt.root; cd /usr/ports/directory &prompt.root; make MASTER_SITE_OVERRIDE=ftp://ftp.FreeBSD.ORG/pub/FreeBSD/distfiles/ fetch Q. I want to know what files make is going to need before it tries to pull them down. A. make fetch-list will display a list of the files needed for a port. Q. Is there any way to stop the port from compiling? I want to do some hacking on the source before I install it, but it is a bit tiresome having to watch it and hit control-C every time. A. Doing make extract will stop it after it has fetched and extracted the source code. Q. I am trying to make my own port and I want to be able to stop it compiling until I have had a chance to see if my patches worked properly. Is there something like make extract, but for patches? A. Yep, make patch is what you want. You will probably find the PATCH_DEBUG option useful as well. And by the way, thank you for your efforts! Q. I have heard that some compiler options can cause bugs. Is this true? How can I make sure that I compile ports with the right settings? A. Yes, with version 2.6.3 of gcc (the version shipped with FreeBSD 2.1.0 and 2.1.5), the -O2 option could result in buggy code unless you used the -fno-strength-reduce option as well. (Most of the ports don't use -O2). You should be able to specify the compiler options used by something like &prompt.root; make CFLAGS='-O2 -fno-strength-reduce' install or by editing /etc/make.conf, but unfortunately not all ports respect this. The surest way is to do make configure, then go into the source directory and inspect the Makefiles by hand, but this can get tedious if the source has lots of sub-directories, each with their own Makefiles. Q. There are so many ports it is hard to find the one I want. Is there a list anywhere of what ports are available? A. Look in the INDEX file in /usr/ports. Q. I went to install the foo port but the system suddenly stopped compiling it and starting compiling the bar port. What's going on? A. The foo port needs something that is supplied with bar — for instance, if foo uses graphics, bar might have a library with useful graphics processing routines. Or bar might be a tool that is needed to compile the foo port. Q. I installed the grizzle program from the ports and frankly it is a complete waste of disk space. I want to delete it but I do not know where it put all the files. Any clues? A. No problem, just do &prompt.root; pkg_delete grizzle-6.5 Q. Hang on a minute, you have to know the version number to use that command. You do not seriously expect me to remember that, do you?? A. Not at all, you can find it out by doing &prompt.root; pkg_info -a | grep grizzle Information for grizzle-6.5: grizzle-6.5 - the combined piano tutorial, LOGO interpreter and shoot 'em up arcade game. Q. Talking of disk space, the ports directory seems to be taking up an awful lot of room. Is it safe to go in there and delete things? A. Yes, if you have installed the program and are fairly certain you will not need the source again, there is no point in keeping it hanging around. The best way to do this is &prompt.root; cd /usr/ports &prompt.root; make clean which will go through all the ports subdirectories and delete everything except the skeletons for each port. Q. I tried that and it still left all those tarballs or whatever you called them in the distfiles directory. Can I delete those as well? A. Yes, if you are sure you have finished with them, those can go as well. Q. I like having lots and lots of programs to play with. Is there any way of installing all the ports in one go? A. Just do &prompt.root; cd /usr/ports &prompt.root; make install Q. OK, I tried that, but I thought it would take a very long time so I went to bed and left it to get on with it. When I looked at the computer this morning, it had only done three and a half ports. Did something go wrong? A. No, the problem is that some of the ports need to ask you questions that we cannot answer for you (eg “Do you want to print on A4 or US letter sized paper?”) and they need to have someone on hand to answer them. Q. I really do not want to spend all day staring at the monitor. Any better ideas? A. OK, do this before you go to bed/work/the local park:- &prompt.root cd /usr/ports &prompt.root; make -DBATCH install This will install every port that does not require user input. Then, when you come back, do &prompt.root; cd /usr/ports &prompt.root; make -DIS_INTERACTIVE install to finish the job. Q. At work, we are using frobble, which is in your ports collection, but we have altered it quite a bit to get it to do what we need. Is there any way of making our own packages, so we can distribute it more easily around our sites? A. No problem, assuming you know how to make patches for your changes:- &prompt.root; cd /usr/ports/somewhere/frobble &prompt.root; make extract &prompt.root; cd work/frobble-2.8 [Apply your patches] &prompt.root; cd ../.. &prompt.root; make package Q. This ports stuff is really clever. I am desperate to find out how you did it. What is the secret? A. Nothing secret about it at all, just look at the bsd.ports.mk and bsd.ports.subdir.mk files in your makefiles directory. Readers with an aversion to intricate shell-scripts are advised not to follow this link...)