Typical Startup Script Let’s look at a typical startup script, snmpd.sh, which is part of the net-snmp package that we’ll install a little later. All you need to know at this point is that it starts the SNMP server daemon at boot, and stops that same daemon on shutdown. Here’s the script: ………………………………………………………………………………………. v #! /bin/sh w if ! PREFIX=$(expr $0 : “(/.*)/etc/rc.d/$(basename $0)$”); then echo “$0: Cannot determine the PREFIX” >&2 exit 1 fi x case “$1″ in start) [ -x ${PREFIX}/sbin/snmpd ] && ${PREFIX}/sbin/snmpd && echo -n ‘ snmpd’ ;; stop) killall snmpd && echo -n ‘ snmpd’ ;; *) echo “Usage: `basename $0` {start|stop}” >&2 ;; esac exit 0 ………………………………………………………………………………………. The #!/bin/sh line (v) indicates that this is a shell script. The remainder of the file (which is similar to a Windows batch file) contains commands that are run by the script. The first section, set off by if (w) and fi, determines the path to the programs, and tells the rest of the script whether it was started in /usr/local, /usr/X11R6, or some other directory. The rest of the script (x) needs to know this, so it can find its commands. The ? case “$1″ in line (x) is where the script actually makes a decision. This part of the script reads the first argument that the script is called with. For example, if your script is run as snmpd.sh start, start is your first argument. If you run it as snmpd.sh stop, the stop is your first argument. The script has several smaller sections: Everything between the start) and the double semicolon (;;) are steps that are taken if the first argument is start. Everything between the stop) and the next double semicolon are actions that are taken if the first argument is stop. The last option, the *), is a wildcard for all other arguments that might be typed in. For example, if you run this script as snmpd.sh start, the script runs the following command: ………………………………………………………………………………………. [ -x ${PREFIX}/sbin/snmpd ] /&& ${PREFIX}/sbin/snmpd && echo -n ‘ snmpd’ ………………………………………………………………………………………. These are standard UNIX shell commands. This command first checks to see if the snmpd program exists. If it does, it runs it and prints out its name. Similarly, if you call the script with a stop argument, it unceremoniously kills all snmpd processes.
Note: If you are looking for good and high quality web space to host and run your application check Lunarwebhost Inexpensive Web Hosting services
Chapter 11: Advanced Software Management Overview This chapter covers several things you need to know about running software on FreeBSD. FreeBSD can run a wide variety of software packages, most of which are available as source code so they can be built as native FreeBSD software. And, thanks to some clever design, FreeBSD can also run software from many foreign operating systems. We’ll look at how to do this, focusing on the popular Linux compatibility package that allows FreeBSD to run unmodified Linux software. Also, for your programs to start at boot, and stop cleanly when the system shuts down, you must be able to edit and write proper startup and shutdown scripts. While some programs stop just fine when you kill the operating system they’re running on, others (like databases) demand a gentler shutdown process. While you can get by with a variety of ugly hacks, starting and stopping network services cleanly is an excellent habit to get into and enforce. We’ll examine how to properly implement and manage these systems in FreeBSD. And, while under normal circumstances you’ll never need to know how FreeBSD’s linking and shared library support works, we’ll discuss how shared libraries work and how to manage and configure them. Why? Because normal circumstances are, oddly, quite rare in the computer business. Finally, we’ll look at how systems with multiple processors work, and how they interact with software. While multiple processors can greatly increase system power, they won’t help you if your software isn’t properly configured. Startup and Shutdown Scripts While FreeBSD’s main system software is started by /etc/rc, add-on software is started by separate scripts. The ports and packages system installs these scripts for you. If you install your own software, however, you’ll need to create a script that handles this startup and shutdown process. Plus, to change an existing add-on package’s startup process, you must understand how the startup scripts function. Note This section assumes that you have some basic understanding of shell scripts. If you’ve never seen or used a shell script before, read the examples here very carefully. Shell scripting is not hard, and the best way to learn is to read examples. The /etc/rc shell scripts (see Chapter 9) handle the main system startup process. During boot up, the FreeBSD startup script checks several directories for additional shell scripts. The most popular directory for startup and shutdown scripts is /usr/local/etc/rc.d, though /usr/X11R6/etc/rc.d is another default location. These directories are specified in /etc/defaults/rc.conf, and can be overridden in /etc/rc.conf. (You can add additional script directories with the local_startup rc.conf variable.) The shell script locator just checks in those directories for any files ending in “.sh”. If it finds such a file, it assumes that the file is a shell script and executes it with an argument of start. During shutdown, FreeBSD runs these same scripts with an argument of stop. The scripts are expected to read those arguments and take appropriate action. 250
Note: If you are looking for good and high quality web space to host and run your application check Lunarwebhost Inexpensive Web Hosting services
out why some trivial change is causing things to go haywire. It can also keep another administrator from calling you up at 3 AM and asking why the heck the system doesn’t work with the default settings. When you decide to upgrade on a production system, map out your changes. You can use pkg_info -aRto see which packages require other packages. The general rule of thumb is to upgrade your dependencies first. Those packages that are required by other packages should be the first to be upgraded and tested. After all, if something’s wrong with lower-level software, everything that depends on it will fail. You can use pkg_delete -f to remove dependencies, and then install the newer versions from ports or packages. Then follow the chain upward, upgrading newer versions as required. Again, you can try to run a software package with a newer version of a dependency, but it might not work. The /usr/ports/sysutils/portupgrade tool is worth considering, because it can handle some of these tasks automatically. Still, you need to understand and be able to deal with conflicts and dependencies yourself. 249
Note: If you are looking for good and high quality web space to host and run your application check Lunarwebhost Inexpensive Web Hosting services
………………………………………………………………………………………. # make index && make readmes ………………………………………………………………………………………. Note Both of these commands (make index and make readmes) take quite some time to complete. By using the &&, you tie them together; when the first command completes successfully, the second command will run. This saves you the trouble of going back in half an hour and typing the second command. The ports collection upgrade doesn’t remove old work directories. If you have installed ports and haven’t run make clean, the work directories and older source code will still be around, together with the status-tracking hidden files. If you run make in these ports, the hidden files will show that the port is built and will refuse to run, and you’ll need to run make clean to build these ports. Ports Collection Upgrade Issues The problem with upgrading the ports collection is that any collection of software is intended to be used as a cohesive whole. If you have an older version of a commonly used tool that your new software requires, you might have to upgrade it as well as programs that depend upon it. The danger is that a simple upgrade can quickly become a cascading series. Of course, FreeBSD is not the only operating system that suffers from this problem. Every software package on every operating system platform has it. (In Windows you frequently see this manifest as DLL conflicts, unexplained program crashes, or any other weird and unpleasant behavior.) Excellent software design can minimize but not eliminate this problem. Unfortunately, excellent software design is rare. Checking Software Versions On a single-purpose machine, the daisy-chain upgrade isn’t that difficult; after all, a Web server doesn’t generally have hundreds of software packages installed. However, a workstation does, and even my laptop usually has over 200 entries in /var/db/pkg! (You know, I should really go through and uninstall what I don’t use anymore; do I really need that little daemon that follows my mouse?) So what do you do if your system has complex software dependencies? FreeBSD has a software-version-checking tool called pkg_version(1). Pkg_version compares the version of the software you have installed with the version number in /usr/ports/INDEX and, if your INDEX file is up to date, you’re all set. (You did follow my advice in the last section and update your index and readmes, didn’t you? Of course you did. You’re not the type of person that would go drop some hard-earned cash on a computer book and then ignore it, are you? Of course not.) A basic version check might look like this: ………………………………………………………………………………………. # pkg_version -v apache-1.3.20 = up-to-date with port autoconf-2.13_1 = up-to-date with port bzip2-1.0.1 = up-to-date with port cvsup-bin-16.1 ? orphaned: net/cvsup-bin emacs-20.7 = up-to-date with port gettext-0.10.35 = up-to-date with port gmake-3.79.1 = up-to-date with port 247
Note: If you are looking for good and high quality web space to host and run your application check Lunarwebhost Unix Web Hosting services
ispell-3.1.20c_2 < needs updating (port has 3.2.04_1) jade-1.2.1_1 = up-to-date with port libtool-1.3.4_2 = up-to-date with port links-0.96pre7 < needs updating (port has 0.96,1) m4-1.4 = up-to-date with port mutt-1.2.5 = up-to-date with port rrdtool-1.0.33 = up-to-date with port sftp-0.9.6_1 = up-to-date with port sudo-1.6.3.7 < needs updating (port has 1.6.3.7_1) ucd-snmp-4.2.1 = up-to-date with port unzip-5.42 = up-to-date with port uulib-0.5.13 = up-to-date with port xsysinfo-1.4a = up-to-date with port zip-2.3 = up-to-date with port .................................................................................................... Reading down the list of comments next to each piece of software, it's easy to see that most of the software on this system is the latest version. But take a look at the entry for ispell-1.2.10. The message shows that the port is out of date, and at some point you might want to update that program. You need to decide on your own if the package is important enough for you to spend the time needed to upgrade it. Now, since I personally installed every piece of software on this system, I'm familiar with it, and I know how important everything is. I know that ispell is a spell-checker and that its importance in the grand scheme of Web serving is minimal at best. I'm not going to worry about it. On the other hand, the sudo-1.6.3.7 package is a security tool used to control user privileges; correct operation of this program is absolutely vital. If a newer version is available I must investigate and probably upgrade. The entry for cvsup-bin-16.1 with the message of "orphaned: net/cvsup-bin" tells us that there is no entry for this piece of software in /usr/ports/INDEX, and hence no port for this package. I installed this port from a package; no port is available. Automatically Checking Software Versions You can add an automated software-version check to your weekly status email. Just add the following line to /etc/periodic.conf: .................................................................................................... weekly_status_pkg_enable="NO" .................................................................................................... Create /etc/periodic.conf if you don't have one. For full details on /etc/periodic.conf, see Chapter 9. Hints for Upgrading Most of the software-maintenance process is based upon knowing what your server is supposed to do. If you are the only administrator of a machine, things are very simple. Once you start having multiple administrators, however, you'll find that keeping track of this information becomes very difficult. I cannot stress highly enough the importance of keeping a server log for every system on your network! Even a text file, /etc/changes, where you jot down things like "mwlucas, 5-15-01: Installed sftp for client bufar@absolutebsd.com" can save you hours of pain later as you try to figure 248
Note: If you are looking for good and high quality web space to host and run your application check Lunarwebhost Unix Web Hosting services
1. The first thing to do when upgrading is to be sure that you have a package of the older version of the software available. If things go wrong on a production system, you’ll want to be able to fall back to the older version very quickly. If you’ve installed the software from CD-ROM, check that you still have that disk; if you installed via FTP, download the same package and keep it handy. 2. If at all possible, test the upgraded software on a nonproduction system. Production server upgrades can give even seasoned administrators white hair and worse tempers. Successfully upgrading once makes further upgrades much easier and faster. 3. Make sure you have a system backup. See Chapter 3 for details on how to do this with either a tape or a filesystem. 4. Get your upgraded software, preferably via a package you have built on your test machine. (That way, you know that the port actually builds and installs.) Otherwise, build the software from a port using make build. Don’t do the actual make install, just make build to confirm that you can actually compile the program cleanly. 5. Notify your users that you will be upgrading the service at such-and-such a time, and that the program or machine will be unavailable. 6. At the scheduled time, do a make deinstall or pkg_delete on the old package, then a make install on the new port. Be ready to fall back to the older version if this doesn’t work! The most frequent problem people have when upgrading is determining which software on their system needs upgrading. My general rule is that things that work should not be upgraded just because a newer version is available. This holds true especially for large, complicated software packages, such as some of the newer desktop window managers. Still, you may find that even though everything is working just fine, a newer version of a piece of software addresses a problem you have or provides needed functionality. You can make your life easier by upgrading your ports tree to allow you to easily install that newer piece of software. Upgrading the Ports Collection The FreeBSD upgrade process also handles upgrading your ports tree. You can use CVSup and the ports-supfile configuration to upgrade your ports to the latest version or, indeed, to any version you choose. To begin, you’ll need to install CVSup as described in Chapter 6, and edit the ports-supfile to use a particular CVSup mirror. When you’re done, run it with this command: ………………………………………………………………………………………. # cvsup ports-supfile ………………………………………………………………………………………. CVSup will crawl over your ports tree, comparing each file with the version on the CVSup mirror you’ve chosen, and make changes in your files as needed. When it finishes, you’ll have the latest version of the ports tree. Once you’ve finished upgrading your ports collection, you should upgrade your index and your readme files. To do so, go to /usr/ports and type this: 246
Note: If you are looking for good and high quality web space to host and run your application check Lunarwebhost Unix Web Hosting services
The command to create a package is make package. This will install the program on the local machine and create a package in that port’s directory. Simply copy this package to other systems and run pkg_add to install it. You can even set up an anonymous FTP server (see Chapter 12) and have a local master package repository. Remember the PACKAGESITE environment variable? Set that to a path on your anonymous FTP server and put your custom packages there. You can then use pkg_add -r on your other machines, and they will automatically grab the customized packages. Changing the Install Path If you have dozens, or even hundreds, of FreeBSD systems, all with mostly identical configurations, you might find the default port or package installation path of /usr/local problematic. In many large server farms, /usr/local is reserved for programs that are unique to the individual machine, and other software packages that are used by every system in the server farm are expected to be installed elsewhere. A common alternative to /usr/local is /usr/pkg, which you can set for your system with the PREFIX variable: ………………………………………………………………………………………. # make PREFIX=/usr/pkg install clean ………………………………………………………………………………………. When the port is installed, it will go into your chosen location. Setting Make Options Permanently If you get sick and tired of typing the same options repeatedly when building ports, you can list your options in make.conf to have them automatically used whenever you install a port. See the section on make.conf in Chapter 9 for details. Note While we’re at it, /etc/make.conf is scanned any time you run make. This means that any options you set there are applied to ports. While features like CPUTYPE might not make a difference for you, it’s possible that they will. In any event, it’s a possible source of confusion, and you should at least be aware that it exists. [4]SKIP is Sun’s Secure Connectionless Internet Protocol (the acronym stands for Simple Key-management for Internet Protocols). It is a wonderful virtual private network (VPN) protocol that has unfortunately fallen into disfavor in the face of IPSec. This is yet another example of the market bludgeoning cool technology into the grave. Upgrading Ports and Packages The software-upgrade process can be very simple or a complete nightmare, but with a bit of preparation you can avoid many common pitfalls. The following list of suggestions assumes that you’re upgrading an Internet server and that you have actual users depending on it. (If you’re upgrading your laptop, you can consider your user notified before you start.) 245
Note: If you are looking for good and high quality web space to host and run your application check Lunarwebhost Unix Web Hosting services
# ls -a total 21 969094 drwxr-xr-x 3 root wheel 512 Jul 22 21:24 . 778196 drwxr-xr-x 4 root wheel 512 Jul 27 20:42 .. 969343 -rw-r–r– 1 root wheel 17163 Jul 22 21:24 .PLIST.mktmp 969344 -rw-r–r– 1 root wheel 0 Jul 22 21:24 .PLIST.setuid 969345 -rw-r–r– 1 root wheel 19 Jul 22 21:24 .PLIST.startup 969341 -rw-r–r– 1 root wheel 0 Jul 22 21:23 .build_done 969115 -rw-r–r– 1 root wheel 0 Jul 22 21:22 .configure_done 969112 -rw-r–r– 1 root wheel 0 Jul 22 21:21 .extract_done 969342 -rw-r–r– 1 root wheel 0 Jul 22 21:24 .install_done 969113 -rw-r–r– 1 root wheel 0 Jul 22 21:21 .patch_done 318289 drwxr-xr-x 8 root wheel 512 Jul 22 21:23 apache_1.3.20 # ………………………………………………………………………………………. So what does this tell us? Well, all files whose names begin with a period (which is most of the files listed here, except for apache_1.3.20) are “hidden” files that don’t show up on a normal directory listing. The ports system and the make process uses these files to keep track of what stage the build process is in. Every port uses these files. See the hidden file .install_done? If that file exists, the port believes that it’s already installed, and it refuses to overwrite itself. So there’s our problem. Remove that file, and the make reinstall will succeed. Cleaning Up with Make Clean Ports can take up a lot of room. Some, such as XFree86, can soak up a couple hundred megs of disk once they’re extracted and built. Most of this disk usage is from the original source code, which you will no longer need. The ports system includes a method to remove excess source code. Once you have your program installed and configured the way you like it, you don’t really need the copy of the source code in the ports directory any more. You can remove it with make clean. (This blows away the work directory, so be sure that you’re happy with your program before you do it!) You can also clean a new port immediately on install by running make install clean when you install it. You might also clean the port’s original distfiles, which are stored in /usr/ports/distfiles. (Check this directory now and then, because it can fill up quickly if you build a lot of ports.) Removing unneeded distfiles frees considerable disk space. To clean the entire ports tree, run make clean directly under /usr/ports. This takes some time, though, and, while there are faster and more efficient ways to remove every work directory in the ports tree, this one is directly supported by the FreeBSD Project. Building Packages If you’re using ports, you can build your own packages to install on other FreeBSD machines, which can save you a lot of time and ensure that you have identical software on every machine. If you have several machines running Snort, for example, and you want them all to have the same features, you can build Snort once and then make a package out of it to install on all the other machines. 244
Note: If you are looking for good and high quality web space to host and run your application check Lunarwebhost Unix Web Hosting services
variable! Setting variables on the command line is much simpler than figuring out how to add these commands to the build process on your own. If you use several make commands to build the port (for example, make patch and then make install clean), you must include any options you want with every make command. Otherwise, the port might not build correctly. For example, if you wanted to use the flexresp option in Snort, but had your own custom patch to apply, you would need to run the following: ………………………………………………………………………………………. # make patch WITH_FLEXRESP=YES ………………………………………………………………………………………. Then you would apply your patch and run this: ………………………………………………………………………………………. # make install WITH_FLEXRESP=YES ………………………………………………………………………………………. Otherwise, you would patch your system with the customization option WITH_FLEXRESP, but you wouldn’t give it the correct instructions when compiling the software. Your program would be internally inconsistent, and quite possibly would fail. The hardest part of customizing the way your software builds is deciding which options you’d like. Unfortunately, there is no easy answer to this question, so it’s best to check the software manual or Web site to help you decide. More than once I’ve installed a piece of software, read the documentation, and turned right around to uninstall and reinstall with the options I needed. Uninstalling and Reinstalling One nice thing about installing ports is that, once installed, the port is treated just like a package. You can uninstall a port with pkg_delete, and learn about it with pkg_info. Since the port’s installation is recorded under /var/db/pkg, you can also go through the contents file and investigate every file the port includes. You also can uninstall a port from the port directory. For example, suppose FreeBSD includes several different versions of one port, like the Apache Web server. You might want to build several different versions of the port, evaluate each, and pick one to install for long-term use. Then, once you’ve evaluated one version of the program, and want to uninstall it, you can run make deinstall in the port directory to erase the program from the main system. Note Once you’ve run make install, the compiled program and source files still live under the work subdirectory in the port. You can run make reinstall to reinstall an uninstalled program. You can uninstall and reinstall a program as many times as you like. At some point, you may find that you want to reinstall a port you’ve removed with pkg_delete and that when you run make reinstall it fails, complaining that the port is already installed. Why? Well, do a “long list” of the problem port’s work directory: ………………………………………………………………………………………. # cd /usr/ports/www/apache13/work 243
Note: If you are looking for good and high quality web space to host and run your application check Lunarwebhost Unix Web Hosting services
You can always get information about a port’s additional features from the Makefile, but let’s look at an announcement and see how to use it first. For example, when you try to install /usr/ports/security/snort you’ll see a notice: ………………………………………………………………………………………. Set WITH_FLEXRESP, WITH_MYSQL, WITH_ODBC or WITH_POSTGRES to get additional support. ………………………………………………………………………………………. A bit of Web searching would show you that mysql, odbc, and postgresql are database packages, and this message tells you that you could build Snort with support for these databases. A similar search would show that flexresp is part of the libnet software package. If you get an announcement, and you want to use one of these options, press CONTROL-C to abort the port build. You can then set these options on the install command line like this: ………………………………………………………………………………………. # make WITH_ODBC=YES install ………………………………………………………………………………………. This command changes the way the port will be built and will build your version of Snort with support for ODBC database connections. With this feature built into Snort, you would then be able to log data across the network to any database that supports ODBC, such as Microsoft SQL Server or an Oracle database. Here’s one area where ports shine over packages. You couldn’t do this customization with a package, unless you had multiple versions of the same package. And sorting through snort-odbc-1.9.tgz, snort-mysql-1.9.tgz, snort-postgres-mysql-libnet-1.9.tgz, and so on would be utterly hideous, waste space on the CD, and be mostly unused. The Makefile itself will tell you the build options for a port. At the top of the Makefile, you’ll see a lot of stuff that describes the port, like this: ………………………………………………………………………………………. v PORTNAME= snort w PORTVERSION= 1.8 x CATEGORIES= security y MASTER_SITES= http://www.snort.org/Files/ http://www.physik.TU-Berlin.DE/~ibex/ ports/distfiles/ z DISTNAME= ${PORTNAME}-${PORTVERSION}-RELEASE { MAINTAINER= dirk@FreeBSD.org | GNU_CONFIGURE= yes } CONFIGURE_ARGS= –with-mysql=no –with-odbc=no –with-po stgresql=no ~ MAN8= snort.8 ………………………………………………………………………………………. Much of this is obvious to people who habitually build software from source. If you’re not at that point yet, don’t worry. You’ll get there with practice. Let’s consider this particular example. Not all of 241
Note: If you are looking for good and high quality web space to host and run your application check Lunarwebhost Unix Web Hosting services