Unix Web Hosting for Developers

nameservice set up, because the computer won’t know it has a nameserver running unless you tell it! Just about anything you do on a network will require a working nameservice client. Use keywords in /etc/resolv.conf to tell your system’s resolver where to look for information. Domain or Search Keywords When you’re working on machines on your own network, you don’t want to have to type the whole hostname. (If you have 30 Web servers, typing ssh www19.mycompany.com gets old.) To tell the resolver which domains to check by default, use either a domain or a search keyword in /etc/resolv.conf. Specifying the Local Domain The domain keyword tells the resolver which local domain name to check, by default, for hosts. For example, to specify http://absolutebsd.com/ as the local domain, enter this: ………………………………………………………………………………………. domain AbsoluteBSD.com ………………………………………………………………………………………. Once the local domain is specified, any command that would ordinarily require a domain name will be assumed to be pointing to http://absolutebsd.com/. Were I to ping www, the resolver would append the name http://www.absolutebsd.com/ to that and tell ping to try http://www.absolutebsd.com/. Specifying a List of Domains with Search Alternatively, I can use the search keyword to specify a list of domains to try. Perhaps my company has several domain names in use in different parts of the network I could enter the following: ………………………………………………………………………………….. search http://absolutebsd.com/ blackhelicopters.org stenchmaster.org ………………………………………………………………………………….. In this case, the resolver will check these three domain names in the order written, until it finds a match. For example, if I enter ping petulance, it will try to find petulance.AbsoluteBSD.com. If that fails, it will check for petulance.blackhelicopters.org, the next domain in order. Finally, it will check for petulance.stenchmaster.com. If no such host exists in any of these domains, the command will eventually fail. Note If you don’t list either a domain or a search keyword, the resolver will use the local machine’s domain name. 274

Note: If you are looking for good and high quality web space to host and run your application check Lunarwebhost PHP Web Hosting services

………………………………………………………………………………………. # dig AbsoluteBSD.com +norecurse ………………………………………………………………………………………. Try this one now, with a domain that you’re pretty certain your local nameserver has never looked at. My favorite DNS server test site is http://www.moo.com/, simply because it’s cool, obscure, cute, mostly harmless, and nobody ever looks at it. You’re almost guaranteed that nobody on your nameserver has looked at http://moo.com/ lately. If your nameserver doesn’t have this information cached, or if this information has expired from the cache, your first section will look much like that of our previous example. The authority section, however, will look quite different: ………………………………………………………………………………………. ;; AUTHORITY SECTION: com. 18h9m59s IN NS A.GTLD-SERVERS.NET. com. 18h9m59s IN NS G.GTLD-SERVERS.NET. com. 18h9m59s IN NS H.GTLD-SERVERS.NET. com. 18h9m59s IN NS C.GTLD-SERVERS.NET. com. 18h9m59s IN NS I.GTLD-SERVERS.NET. com. 18h9m59s IN NS B.GTLD-SERVERS.NET. com. 18h9m59s IN NS D.GTLD-SERVERS.NET. com. 18h9m59s IN NS L.GTLD-SERVERS.NET. com. 18h9m59s IN NS F.GTLD-SERVERS.NET. com. 18h9m59s IN NS J.GTLD-SERVERS.NET. com. 18h9m59s IN NS K.GTLD-SERVERS.NET. com. 18h9m59s IN NS E.GTLD-SERVERS.NET. com. 18h9m59s IN NS M.GTLD-SERVERS.NET. ………………………………………………………………………………………. The nameservers under GTLD-SERVERS.NET are the root servers. They contain the master lists of which nameservers control which domains. By giving you this output, your local nameserver is saying, “I don’t know, I’ll have to go ask someone else, but you told me to not recurse so I’m stopping here.” To query a root nameserver, combine the norecurse option and server name dig commands and try your query again: ………………………………………………………………………………………. # dig @a.gtld-servers.net +norecurse www.moo.com ………………………………………………………………………………………. Follow the chain of information for a site or two, and you’ll start to really understand how DNS works. Configuring a DNS Client: The Resolver Before you can have your system use a DNS server, you must tell the computer which nameserver to use, and how it should be used. Even a DNS server needs to have the client portion of 273

Note: If you are looking for good and high quality web space to host and run your application check Lunarwebhost Linux Web Hosting services

21.60.80.66.in-addr.arpa. 2h24m IN PTR www.megapathdsl.net…. # ………………………………………………………………………………………. Although we know that at least http://www.nostarch.com/ and http://nostarch.com/ live on this IP address, this reverse lookup shows us the most correct name for the host. Both of those machine names live on a machine that is actually called http://www.megapathdsl.net/. The PTR entry in the middle of the answer indicates that this is a pointer record. It is a specific sort of entry used to mark IP-address-to-hostname records. When you set up DNS, every host with an IP address will need both a PTR and an A record in your nameservice system. More Dig Options The dig program takes a wide variety of command-line options to control how it checks for information. Check dig(1) for a complete list of these options. We’ll only discuss the servername and norecurse options here, because they’re the ones most commonly used. Server Name The first option is the server name. By default, dig queries the first nameserver listed in /etc/resolv.conf. If you’re trying to debug a problem, however, you want to ask different nameservers. You can do this on the command line by using the @ option. For example, to ask http://dns1.yahoo.com/ what it knows about http://absolutebsd.com/ enter this: ………………………………………………………………………………………. # dig @dns1.yahoo.com AbsoluteBSD.com ………………………………………………………………………………………. The output from this command will look much like the sample output shown earlier, except that you’ll see references to Yahoo!’s nameserver. If you’re debugging a problem, you should compare this information carefully with that given by your local nameserver. If information from two different nameservers conflicts, you may well have found your problem. (DNS information should only change when you add hosts, rename hosts, or renumber hosts.) You can use some of dig’s other options to see exactly where the problem occurs. Controlling How Dig Queries Dig has two other sorts of options: those that control how dig itself runs, and those that control how dig makes queries. The options that control how dig runs are prefaced with a minus sign ( ); those that control how dig makes its queries are prefaced by a plus sign (+). While dig can do a lot of really nifty tricks, they’re beyond the scope of this chapter. Controlling how it makes the queries is quite useful, however. By default, a nameserver will recurse queries to return an answer. While this is helpful if you’re asking the nameserver about a domain, you don’t want the nameserver to dig up the answer for you when you’re debugging; instead, you want to check each nameserver in turn. You can do this with the norecurse option. 272

Note: If you are looking for good and high quality web space to host and run your application check Lunarwebhost Linux Web Hosting services

The type is the type of record we’re looking for. By default, dig looks for an A (address) record, which means that we have the name, but want the IP address. A PTR (pointer) record request means that we have an IP address and want the associated name. The Answer Section Next we have the answer section ({). The first thing it contains is the host we’re looking for (http://nostarch.com/). The next figure (2h13m2s IN CNAME) is the time to live for this information the amount of time your local nameserver may cache it. The 2h13m2s indicates that this data expires in 2 hours, 13 minutes, and 2 seconds. The IN, again, indicates that this is Internet data. Finally, CNAME means that what follows is a canonical name for the server, which tells us that what we’re looking for is actually an alias for something else. The dig output confirms that http://www.nostarch.com/ is actually an alias for http://nostarch.com/. The second line of the answer section is almost identical to the first, except it has an A instead of a CNAME. This tells us that what follows is an IP address. The Authority Section To get the IP address of http://www.nostarch.com/, we have to follow the chain of authoritative nameservers. The authority section (|) lists the servers responsible for the domain, their time to live, and the sort of data and servers that they are. In this case, the nameservers are ns1.megapath.net and ns2.megapath.net. Data for http://www.nostarch.com/ is to be cached for 7 hours, 48 minutes, and 45 seconds, and these are (again) Internet (IN) records. The NS means that these are nameservers. The Additional Section Finally, under the additional section (}), dig lists the IP addresses of all the hosts listed with the host we want. Our example lists the nameservers for http://www.nostarch.com/: ns1.megapath.net and ns2.megapath.net. The interesting thing here is that the time to live is 7 hours, 48 minutes, and 35 seconds. In this case, the time to live isn’t the value for how long the local nameserver should keep the information on the hosts, it’s how long the data on the nameservers for http://nostarch.com/ should be kept. Once this time passes, your nameserver will discard the information and go fetch the nameserver list from one of the Internet’s root nameservers. This isn’t necessarily good or bad, but if you’re trying to solve a problem, it’s good to know. Do a dig on a couple of domains and become familiar with how the output should look. Looking Up Hostnames with Dig Suppose you have an IP address and want to identify the associated hostname; for example, you might want to learn who owns a phone number. This is a common problem on the Internet you might see an IP address hitting your Web site every five seconds, and wonder who they are and what they’re trying to do. To look up the name, use reverse lookup with dig’s -x option. Since much of the result is completely identical to what you see in the forward lookup, we’ll only look at the section that’s different: the answer. ………………………………………………………………………………………. # dig -x 66.80.60.21 ;; ANSWER SECTION: 271

Note: If you are looking for good and high quality web space to host and run your application check Lunarwebhost Unix Web Hosting services

;; ->>HEADER<<- opcode: QUERY, status: y NOERROR, id: 4 ;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 2, ADDITIONAL: 2 z ;; QUERY SECTION: ;; www.nostarch.com, type = A, class = IN { ;; ANSWER SECTION: www.nostarch.com. 2h13m2s IN CNAME nostarch.com. nostarch.com. 2h13m2s IN A 66.80.60.21 | ;; AUTHORITY SECTION: nostarch.com. 7h48m45s IN NS NS1.MEGAPATH.NET. nostarch.com. 7h48m45s IN NS NS2.MEGAPATH.NET. } ;; ADDITIONAL SECTION: NS1.MEGAPATH.NET. 7h48m35s IN A 216.200.176.4 NS2.MEGAPATH.NET. 7h48m35s IN A 216.34.237.2 ;; Total query time: 11 msec ;; FROM: blackhelicopters.org to SERVER: default -- 127.0.0.1 ;; WHEN: Sun Apr 7 12:24:17 2002 ;; MSG SIZE sent: 34 rcvd: 144 .................................................................................................... Wow, talk about a lot of information! (When you're using dig, you're probably trying to debug something. It's better to have too much information than not enough.) So what have we learned? To start with, anything beginning with semicolons is a comment that either lists the options used by dig or divides the answers into sections. The first line (v) lists the version of dig you're using, and the command-line options you used. The second line (w) lists the options that dig is using. (Since we didn't specify any options on the command line, these are the default options; we'll discuss some useful options later.) The third line (x) tells us whether dig got an answer (it did). The next line (y) contains an important word, NOERROR, which tells us that dig found an answer that appears to be good. If you don't get a NOERROR, you have a problem. (Common errors are NXDOMAIN, meaning that the domain doesn't exist, or SERVFAIL, meaning that the domain is misconfigured on that server.) The next couple of lines contain codes that really aren't of use unless you're heavily into debugging DNS or doing some weird things. So let's jump to the four sections that follow: the query, answer, authority, and additional sections. The Query Section In the query section (z), we see how dig is treating the query. In our sample we see the following: .................................................................................................... ;; www.nostarch.com, type = A, class = IN .................................................................................................... Let's start with the easy bit, the class. While DNS can manage many different naming systems, the one we're concerned with is the Internet system, or IN. Internet domains should always have the class IN. 270

Note: If you are looking for good and high quality web space to host and run your application check Lunarwebhost Linux Web Hosting services

Basic DNS Tools FreeBSD includes several tools for inspecting DNS information. Since most DNS runs over the User Datagram Protocol (UDP), you cannot use telnet to manually query a server as we will do with email and Web services later. Your only access to live DNS information is through host(1) and dig(1). The Host Command To quickly check the IP address of a host, use the host(1) command. For example, to check my publisher’s Web page, I would do the following: ………………………………………………………………………………………. # host www.nostarch.com www.nostarch.com is a nickname for nostarch.com nostarch.com has address 66.80.60.21 nostarch.com mail is handled (pri=20) bysmtp.lax.megapath.net nostarch.com mail is handled (pri=10) by mail.nostarch.com # ………………………………………………………………………………………. This is somewhat interesting because it shows us that under DNS, one host can have multiple names and multiple Web pages on one IP address. This output tells us that the main No Starch Press Web page is actually a nickname for another hostname, http://nostarch.com/; one IP address can have any number of names. This is much like the phone system in a typical family household, in which several people share one telephone. The people are like hostnames, while the phone number is like an IP address. The host does have a single, canonical name, much as a phone is registered to a single person. Note Many server programs require much more than an IP address to function. For example, if you enter http://66.80.60.21/ in your Web browser, you’ll actually pull up the page for No Starch Press’s hosting provider, not the page for No Starch Press. We’ll look at how popular Web servers multiplex multiple Web sites onto a single IP address in Chapter 15, but it’s something you should keep in the back of your mind for future reference. Getting Detailed Information with Dig While the host command is quite helpful, it’s certainly not detailed. Also, you don’t know where this information came from whether it was taken straight from the cache or whether the nameserver dug it up from the domain’s nameserver. The standard program for finding detailed DNS information is dig(1). (Another tool, nslookup(1), was popular for many years but has since fallen out of favor.) Dig has a variety of options that allow you to debug a wide range of nameservice problems, though I’ll cover only the most basic ones here. In its most basic form, a dig command is simply dig and a hostname. For example, to dig up information on my publisher, I would enter this command: ………………………………………………………………………………………. dig www.nostarch.com v ; <<>> DiG 8.3 <<>>; www.nostarch.com w ;; res options: init recurs defnam dnsrch x ;; got answer: 269

Note: If you are looking for good and high quality web space to host and run your application check Lunarwebhost Unix Web Hosting services

Chapter 12: Finding Hosts With DNS The Domain Name Service (DNS) is one of those quiet, behind-the-scenes programs that doesn’t get half the attention it deserves. Although most users have never heard of it, DNS is what makes the Internet, as we know it, work. DNS, or nameservice as it’s also called, provides a map between hostnames and IP addresses. Without DNS, your Web browser or email programs wouldn’t use convenient names like http://www.cnn.com/; instead, you’d have to type in IP addresses. This would greatly reduce the Internet’s popularity. Any Internet service you implement will require DNS. We’ll discuss how DNS works, how to check DNS, how to configure your FreeBSD system to use DNS, and how to build your own DNS server. How DNS Works DNS simply maps IP addresses to hostnames, and hostnames to IP addresses. For example, a user doesn’t want to know that http://www.absolutebsd.com/ is actually 209.69.178.25; she just wants to type the URL into her Web browser and go. DNS does the translation. As the system administrator you must be able to install, inspect, and verify DNS information, and you must understand how your system will perform those same operations. DNS information can be available in any number of places: on the local system, on a local DNS server, on a remote nameserver. UNIX systems use a resolver to provide this information, a program that knows about all these information sources and interfaces with them. When a program wants to know the IP address of a host or the hostname for an IP address, it asks the resolver, which consults the appropriate information sources and returns the information to the program that needs it. We’ll look at how to configure the resolver later in this chapter. Most commonly, a resolver will direct a DNS query from a program to a nameserver, a computer running a program designed to gather DNS information from other computers on the Internet. Once a DNS request hits a nameserver, the nameserver checks its local cache to see if it has looked up that information recently. (Nameservers receive many identical DNS requests; for example, the nameserver at one Internet service provider I worked for received several hundred requests an hour for the IP address for http://www.cnn.com/. Multiply that by all the Yahoo!, eBay, and MSN requests out there, and that cache quickly becomes quite effective.) If the designated nameserver doesn’t have the information, it asks a root server, which keeps a list of the nameservers responsible for every domain on the Net. In a process called a recursive query, the root server tells the nameserver to go ask the appropriate nameservers, which may in turn refer the query to still other nameservers. Eventually, it is referred to the authoritative nameserver for that domain and the original nameserver gets its answer. NoteWhen you register a domain, you must list two nameservers. Hosts expect to be able to get information for that domain from those nameservers. If one nameserver fails, the other should pick up the load, and if all the nameservers for a domain fail, the domain vanishes from the Internet. If that happens, the next time someone browses to www.yourdomain.com, they will get a “domain not found” error. Mail will bounce. The world will believe that you don’t exist. Even big companies, such as Microsoft, do this on occasion. Your manager or customer will notice you, and not in a good way. Pay attention to your nameservice! 268

Note: If you are looking for good and high quality web space to host and run your application check Lunarwebhost Linux Web Hosting services

……………………………………………………………………………………… # make -j4 buildworld ……………………………………………………………………………………… This line tells make to run four processes, and hopefully it will complete more quickly. This doesn’t mean that your make will be completed in one-fourth the time, however; you still have other issues to contend with (see Chapter 14). Note Not all programs can handle being built with the -j flag. At times, even buildworld fails. (There is some discussion of disabling support for make -j in buildworld, as it causes many problems.) It’s worth trying, but if things go badly, you need to fall back to plain old serial make. Multiple processors are not the be-all and end-all of high-performance computing. Your application must be written to take advantage of them. If it isn’t, extra CPUs will not help. [3]Some CPUs (the Alpha) can do multiple things at once. These dual-issue and quad-issue processors are slowly becoming more common. This is one reason why the Alpha was such wonderful technology, and why it’s bad for us all that the Alpha is no more. [4]This example assumes that everyone drives the speed limit, taking turns and not cutting each other off, and in general not acting like real drivers in any American city. Advanced Software Management 267

Note: If you are looking for good and high quality web space to host and run your application check Lunarwebhost Cheap Web Hosting services

NoteThe BGL could have been ripped out entirely and replaced with mutexes everywhere in one massive frenzy of hacking (as commercial OS vendors do), completing the process in only a couple of months, so why not do so? Because doing so would have meant that FreeBSD-current would have been utterly unusable for several months, and 5.0-release would have been poorly debugged. Too, the volunteer developers working on other parts of the system would have had nothing to do. (Telling volunteer developers that they can’t do anything is an excellent way to lose them.) This should give you enough understanding of how SMP works that you can administer it reasonably well. Now, let’s look at the details of handling an SMP system. Using SMP When using SMP, remember that multiple processors don’t necessarily make things go faster. One processor can handle a certain number of operations per second; a second processor just means that the computer can handle twice that many operations per second, but those operations are not necessarily faster. Think of the CPU count as lanes on a road.[4] If you have one lane, you can move one car at a time past any one spot. If you have four lanes, you can move four cars past that spot. Although the four-lane road won’t necessarily allow those cars to reach their destination more quickly, there’ll be a lot more of them arriving at any one time. If you think this doesn’t make a difference, contemplate what would happen if someone replaced your local freeway with a one-lane road. CPU bandwidth is important. Most user processes don’t have to worry about when to use SMP; a process just requests some CPU time and the kernel allocates it. The program doesn’t worry about where this CPU time is coming from. The problem with SMP occurs when you want to have one process use multiple CPUs. The short answer is, you can’t do that unless the program is threaded. Threaded programs are written specifically to run on multiple processors. (Check the program documentation to see if the program is threaded.) Programs such as Apache, which run multiple processes to serve requests, are not threaded but might as well be. Taken as a whole, Apache takes excellent advantage of multiple CPUs. SMP and Upgrades The most common “problem” people encounter with SMP is when performing the default torture test, an upgrade from source. It appears that no matter what, the system never seems to use more than one CPU at a time. The “top” program will show that the system is 50 percent idle, no matter what. Trust your eyes. If the system appears to be half idle, you’re only using one of your CPUs. The make program that handles building software issues a command, waits for a response, then issues another command. Each of these subtasks might be assigned to a different CPU, but the actual make command won’t try to do anything until that original process comes back successful. It only does one thing at a time. You can get around this problem with make’s -j flag, which tells make to run multiple processes simultaneously. The -j flag takes its own argument, the number of make processes to run: 266

Note: If you are looking for good and high quality web space to host and run your application check Lunarwebhost PHP Web Hosting services

This situation changed (for the better) after version 2.2. FreeBSD 3.0 SMP The first implementation of FreeBSD SMP was pretty straightforward: Processes were scattered between the CPUs (achieving a rough load balance), and there was a “lock” on the kernel. The CPU had to hold this lock to run the kernel, and before a CPU would try to run the kernel, it checked to see if the lock was available. If the lock was available, it took the lock and ran the kernel. If the lock was unavailable, the CPU knew that the kernel was being run elsewhere and went on to handle something else. This lock was called the Big Giant Lock (BGL). Under this system, the kernel could know that data would not change from under it. Essentially, it guaranteed that the kernel would only run on one CPU, just as it always had. This strategy worked well enough for two CPUs: You could run a mediumlevel database and Web server on a twin-CPU machine, and feel confident that the CPU wouldn’t be your bottleneck. If one CPU was busy serving up Web pages, the other would be free to answer database calls. But if you wanted to run an eight-CPU machine, you were in trouble; the system would spend a lot of time just waiting for the Big Giant Lock to become available! The kernel still knew that it was only doing one thing at a time, and if a kernel instruction changed some internal value, it would still be that way when it returned. There are many problems with this system, but fundamentally it’s simplistic, and neither efficient nor scalable. In fact, the standard textbooks on SMP rarely mention this method of handling the kernel because it’s so clunky. Still, it beats some other operating systems’ methods of handling SMP. For example, a twinprocessor Windows 2000 system’s default setup dedicates one processor to the user interface and uses the other processor for everything else. While the interface is snappy and the mouse doesn’t drag when you load the system, I would hope that most people don’t purchase SMP hardware to address graphical interface problems. With the growth of system hardware, multiple-CPU systems will become very common in just a few years. For FreeBSD to continue to be a quality operating system, this problem must be addressed. FreeBSD 5 SMP One of the benefits of the BSDi/Walnut Creek merger was the release of the BSD/OS 5.0 code base to the FreeBSD development community. BSD/OS contains a great deal of proprietary information, so the source code cannot be released to the general public. Still, FreeBSD developers were able to read portions of the code. The most interesting part of this code was that multiple CPUs could be in the kernel at once something that will be heavily implemented in version 5, and which will mark one of the big differences between FreeBSD version 4 and version 5. To prevent information corruption, the new FreeBSD SMP system combines the Big Giant Lock with a smaller lock called a mutex. When a piece of the kernel wants to work on a chunk of data, it slaps a mutex over it. When another part of the kernel tries to access this mutex-locked data, it says, Oh, I can’t touch that, and either waits for the resource to become available or tries to allocate some other resource. The goal is to eliminate the Big Giant Lock, and to have all kernel operations only mutex-lock the small bits of data that they need. As the kernel’s smaller systems are rewritten to take advantage of mutexes, their need to hold the BGL will be eliminated. According to Greg Lehey, a major FreeBSD developer and member of the SMP project, this method is expected to scale to beyond 32 processors. 265

Note: If you are looking for good and high quality web space to host and run your application check Lunarwebhost Cheap Web Hosting services