Now that you have a drive, you can actually tell Vinum to recover the disk. Vinum is fairly smart; it doesn’t start drives it cannot talk to. All you have to do is tell Vinum to start the replaced plex, and it will recover it for you: ……………………………………………………………………………………… #vinum start mirrortest.p1 Reviving mirrortest.p1.s0 in the background # vinum[685]: reviving mirrortest.p1.s0 # ……………………………………………………………………………………… You can watch its progress with vinum list: ……………………………………………………………………………………… # vinum list 2 drives: D alpha State: up /dev/da0s1e A: 384/3551 MB (10%) D beta State: up /dev/da1s1e A: 2/3170 MB (0%) 1 volumes: V mirrortest State: up Plexes: 2 Size: 3167 MB 2 plexes: P mirrortest.p0 C State: up Subdisks: 1 Size: 3167 MB P mirrortest.p1 C State: faulty Subdisks: 1 Size: 3167 MB 2 subdisks: S mirrortest.p0.s0 State: up D: alpha Size: 3167 MB S mirrortest.p1.s0 State: R 11% D: beta Size: 3167 MB # ……………………………………………………………………………………… Note the last line: mirrortest.p1.s0. Our replacement disk is 11 percent recovered. If your mirror is under heavy load, this will take a while. Eventually, however, it will recover, and you’ll be back in business with fully mirrored disks. Pretty cool, huh? [2]You might notice that disklabel claims that there are eight partitions, but we only see three. The slice can hold up to eight partitions, but five aren’t actually in use. [3]This is the computing equivalent of writing your rent check on the night before payday. [4]As I was writing this chapter, I noticed a high-pitched noise coming from my computer. Upon investigation, one of my two SCSI hard drives was warm enough to make me jerk my hand back quickly. Since the computer room is kept at about 65 degrees Fahrenheit, it seemed to be time to replace the drive. Sometimes, Fate works with technical folks. Other times, it just picks a decent time to abuse us. 405
Note: If you are looking for good and high quality web space to host and run your application check Lunarwebhost PHP Web Hosting services
Note At the rate disk drive size grows these days, it’s quite likely that a replacement disk will be much larger than the original, and probably even faster. Feel free to allocate the remainder of the disk. When I have extra space left over, I frequently move a little-used chunk of data say, /usr/src or /usr/ports/distfiles onto a large new partition. It’s accessed infrequently enough that it won’t inflict a great performance hit upon my Vinum system, and yet frees disk space elsewhere. Move data that doesn’t need the resilience and speed that Vinum provides, and that is easily recoverable. Plex Recovery Finally, you’re ready to recover your volume. First off, you need to tell Vinum that it has a new disk. Remember, Vinum’s configuration is cumulative. If you just recreate the whole volume, you’ll add the old information to itself. This will confuse Vinum and make you lose your data. For example, our vinum.conf shows drives named alpha and beta. Vinum already knows about those drives. If I were to use that same vinum.conf and run vinum create a second time, we would be telling Vinum that it had two alpha drives and two beta drives, both on the same physical device! This would be vastly confusing to us, let alone a poor, dumb machine. Similarly, you don’t want to obliterate the entire configuration, and make it forget about the drives it already has. All you want to do is change a bit of the existing setup. In this example, we want to recover drive beta. Since we don’t care about what’s on the shiny new replacement drive beta, we can overwrite that part of Vinum’s configuration. To do so, create a Vinum configuration file, /etc/vinum.conf.recover. List only the one drive in there, without any additional information on plexes or subdisks. ……………………………………………………………………………………… drive beta device /dev/da1s1e ……………………………………………………………………………………… That’s the drive name and the partition it’s on. Now, we add this information to the existing Vinum configuration by running vinum create: ……………………………………………………………………………………… # vinum create /etc/vinum.conf.recover 2 drives: D alpha State: up /dev/da0s1e A: 384/3551 MB (10%) D beta State: up /dev/da1s1e A: 2/3170 MB (0%) 1 volumes: V mirrortest State: up Plexes: 2 Size: 3167 MB 2 plexes: P mirrortest.p0 C State: up Subdisks: 1 Size: 3167 MB P mirrortest.p1 C State: faulty Subdisks: 1 Size: 3167 MB 2 subdisks: S mirrortest.p0.s0 State: up D: alpha Size: 3167 MB S mirrortest.p1.s0 State: stale D: beta Size: 3167 MB # ……………………………………………………………………………………… Notice that the state on our beta drive is now “up,” instead of “referenced.” The plex is still “faulty,” mind you, but we’re making progress. 404
Note: If you are looking for good and high quality web space to host and run your application check Lunarwebhost Linux Web Hosting services
Next, you need to replace the failed hard drive, which can be difficult. For one thing, you’ve got to identify the failed drive. A vinum list will tell you which of your drives has failed, identifying the drive by device number. You can then compare the device name to those listed in /varx/run/dmesg.boot and get the SCSI ID, LUN, and bus number. Once you identify the failed drive, then shut down the computer, find the matching hard drive, replace it with a drive with the same settings, and boot up again. Once you reboot, then check to be certain that the new drive came up with the same drive device name as the old one. (This will make later configuration simpler.) Now, ask Vinum how it’s doing: ……………………………………………………………………………………… # vinum list 1 drives: D alpha State: up /dev/da0s1e A: 384/3551 MB (10%) D beta State: referenced A: 0/0 MB 1 volumes: V mirrortest State: up Plexes: 2 Size: 3167 MB 2 plexes: P mirrortest.p0 C State: up Subdisks: 1 Size: 3167 MB P mirrortest.p1 C State: faulty Subdisks: 1 Size: 3167 MB 2 subdisks: S mirrortest.p0.s0 State: up D: alpha Size: 3167 MB S mirrortest.p1.s0 State: stale D: beta Size: 3167 MB # ……………………………………………………………………………………… The beta drive that was on /dev/da1s1 shows up as “referenced,” which means that it’s included in the configuration but isn’t working correctly. In a cascading chain of errors, the plex on this particular drive is “faulty” because the drive isn’t working. The underlying subdisk is “stale,” as the data on it does not match the mirror. None of this is good, but it’s all to be expected. As a virgin replacement, the disk partition we’ve called “beta” has nothing whatsoever on it. If you see any of these errors on an existing Vinum volume, however, it means that the drive in question is having trouble and should be replaced. Preparing the Replacement Drive Before we can use the new drive, we need to partition and label it. You did that earlier in this chapter with sysinstall(8), and this will be almost exactly the same. To begin, start sysinstall with the fdisk menu option. Next, delete any disk partitions shown, and dedicate the whole drive to FreeBSD. Then go into the Label editor. From our previous setup in /etc/vinum.conf, we know that our Vinum partition was 3243037KB. We have a slightly more convenient measurement in the earlier vinum ls, where we can see that one subdisk is 3167MB. We must leave at least that much space for our Vinum partition on our new drive. (I recommend that you allocate a couple extra meg, just in case, if possible.) Now, write your changes to the disk, as if you were adding a disk to your system, and then run disklabel to change the slice type of your Vinum partition, as if you were first setting up Vinum. 403
Note: If you are looking for good and high quality web space to host and run your application check Lunarwebhost PHP Web Hosting services
Other Vinum Commands The vinum(8) program includes a variety of simple commands to allow you to view its condition. Status Check The simplest thing of all is to ask Vinum how it’s doing. The vinum ls command shows each plex you have, its size, the name of its drive, and its state: ……………………………………………………………………………………… # vinum ls S mirrortest.p0.s0 State: I 43% D: alpha Size: 3167 MB S mirrortest.p1.s0 State: I 41% D: beta Size: 3167 MB # ……………………………………………………………………………………… Everything should all be in an “I” state. If it isn’t, check vinum(8) and the Vinum Web site (http://www.vinumvm.org/) and see what your error means. If you want more information, you can do vinum list -r. This will print out detailed status information on every volume, plex, and subdisk in your system. Configuration Check To double-check your Vinum system configuration, use vinum printconfig. This shows us the drive names, followed by the volume name, and then the plexes and subdisks. ……………………………………………………………………………………… #vinum printconfig # Vinum configuration of magpire.blackhelicopters.org, saved at Fri Aug 17 19:24:43 2001 drive alpha device /dev/da0s1e drive beta device /dev/da1s1e volume mirrortest plex name mirrortest.p0 org concat vol mirrortest plex name mirrortest.p1 org concat vol mirrortest sd name mirrortest.p0.s0 drive alpha plex mirrortest.p0 len 6486074s driveoffset 265s plexoffset 0s sd name mirrortest.p1.s0 drive beta plex mirrortest.p1 len 6486074s driveoffset 265s plexoffset 0s # ……………………………………………………………………………………… Replacing a Failed Mirrored Plex The purpose of a mirror is redundancy in case of a drive failure. So, if a drive fails, how do you replace it?[4] First, if at all possible, be sure you have a good backup. You’ll be working directly on the Vinum device, and any mistake can destroy your data. If you don’t have a current backup, take a backup before proceeding. Your remaining plex(es) should still be up and serving data, after all! 402
Note: If you are looking for good and high quality web space to host and run your application check Lunarwebhost Linux Web Hosting services
Vinum. Lying to your computer is not good. The computer will usually catch on quite quickly and panic. (This leads directly to Chapter 20.) Before we use the volume, however, we will newfs it and overwrite everything, so Vinum will never have an opportunity to catch on.[3] This time we have two plexes, each of the concatenated type. Each plex is concatenated, but has only one subdisk, each of which lives on a separate disk. Our first plex has one subdisk on drive alpha, while the other has one subdisk on drive beta. Again, they are the same size. We know that a plex is one copy of the data. Since we have two plexes, each on a separate disk, we know that the data is mirrored across the disks. Now that you have a configuration, use the vinum command to implement it on the actual hard drives. ……………………………………………………………………………………… #vinum create vinum.conf 2 drives: D alpha State: up /dev/da0s1e A: 384/3551 MB (10%) D beta State: up /dev/da1s1e A: 619/3786 MB (16%) 1 volumes: V mirrortest State: up Plexes: 2 Size: 3167 MB 2 plexes: P mirrortest.p0 C State: up Subdisks: 1 Size: 3167 MB P mirrortest.p1 C State: up Subdisks: 1 Size: 3167 MB 2 subdisks: S mirrortest.p0.s0 State: up D: alpha Size: 3167 MB S mirrortest.p1.s0 State: up D: beta Size: 3167 MB # ……………………………………………………………………………………… You have a mirrored volume, as you can see by the two plexes listed. Newfs, mount, and you’re ready to go! Starting Vinum at Boot To start the Vinum system at boot, edit /etc/rc.conf and /etc/fstab. The rc.conf option is quite simple: ……………………………………………………………………………………… start_vinum=”YES” ……………………………………………………………………………………… We looked at /etc/fstab in Chapter 16. Follow the examples already in that file, and add a line something like this to the filesystem table: ……………………………………………………………………………………… /dev/vinum/mailvol /var/mail ufs rw 2 2 ……………………………………………………………………………………… 401
Note: If you are looking for good and high quality web space to host and run your application check Lunarwebhost PHP Web Hosting services
certain interactions. If you use an even number you might very well find that you have all of your inodes on one drive. You might even have all of your data on only one drive! This would utterly eradicate any advantages of striping. You want to be absolutely certain that Vinum will use all of your drives, and this number works. The subdisk labels in our output look slightly different than in the earlier example, however. Since we’re striping, we need to have subdisks of identical size. If one disk is larger than the other, that space will be useless because both disks will use the amount of space available on the smaller drive, wasting whatever space remains on the larger drive. If you intend to use Vinum striping, plan your drive partitions accordingly. Once you have your new vinum.conf set up, run vinum create. The output of vinum create will look almost exactly like our previous example, but it will say “striped” instead of “concat.” Again, examine it carefully to be sure it says what you want it to! Then run newfs and mount the new volume, and see what you have. ……………………………………………………………………………………… # mount /dev/vinum/test /mnt # df Filesystem 1K-blocks Used Avail Capacity Mounted on /dev/ad4s1a 248111 65146 163117 29% / devfs 1 1 0 100% /dev /dev/ad4s1f 2032839 133492 1736720 7% /test1 /dev/ad4s1g 2032839 1266476 603736 68% /test2 /dev/ad4s1h 29497862 4193767 22944267 15% /usr /dev/ad4s1e 3048830 123222 2681702 4% /var procfs 4 4 0 100% /proc /dev/vinum/test 6286319 1 5783413 0% /mnt # ……………………………………………………………………………………… This new /mnt partition is smaller than the concatenated volume, since we lost the extra space on one drive, but accesses will be faster since we’re sharing the load across multiple disks. The faster access speed will be especially obvious with multiple processes writing to the volume simultaneously. Mirrored Volumes Finally, let’s mirror our drives. To begin, first unmount the partition, eliminate your old Vinum configuration with vinum destroy, and create a new /etc/vinum.conf. ……………………………………………………………………………………… drive alpha device /dev/da0s1e drive beta device /dev/da1s1e volume mirrortest setupstate plex org concat sd length 3243037k drive alpha plex org concat sd length 3243037k drive beta ……………………………………………………………………………………… This configuration is a bit different from our previous ones. Our drives are the same, but everything else looks a little different. For one, our volume has a different name, mirrortest. The setupstate keyword tells vinum(8) to assume that the existing contents of the disks are identical we’re lying to 400
Note: If you are looking for good and high quality web space to host and run your application check Lunarwebhost Linux Web Hosting services
……………………………………………………………………………………… # vinum vinum > ……………………………………………………………………………………… Here you can enter vinum(8) commands and interact more directly with the system. You can start over with the resetconfig command: ……………………………………………………………………………………… vinum > resetconfig WARNING! This command will completely wipe out your vinum configuration. All data will be lost. If you really want to do this, enter the text NO FUTURE Enter text > ……………………………………………………………………………………… Yes, that’s NO FUTURE in all caps. It’s Vinum’s subtle way of reminding you that you’re about to absolutely destroy any data on that partition, as well as your configuration. If you’re not certain, just hit ENTER to go back to the main vinum prompt. If you’re sure, enter NO FUTURE and hit ENTER. ……………………………………………………………………………………… Enter text > NO FUTURE Vinum configuration obliterated vinum > ……………………………………………………………………………………… Press CONTROL-D to leave vinum. You now have no Vinum configuration and are ready to build a new one. Striped Volumes The vinum.conf file that follows shows a striped volume. (Remember, a striped volume spreads its data between two disks.) ……………………………………………………………………………………… drive alpha device /dev/da0s1e drive beta device /dev/da1s1e volume test plex org striped 279k sd length 3243037k drive alpha sd length 3243037k drive beta ……………………………………………………………………………………… This looks quite similar to the concatenated setup, with some minor changes. We still have our alpha and beta drives, and the volume name is still test. The plex org keyword has changed to striped and a number. The number at the end of the plex setup line indicates the stripe size; the amount of data that will be written to one drive before switching to the other drive. The 279KB (279k) shown here is a reasonable default. Note Why use such a weird size for the stripe? Well, Vinum and the filesystem have 399
Note: If you are looking for good and high quality web space to host and run your application check Lunarwebhost PHP Web Hosting services
You’ll see newfs output scroll by, listing every superblock on the drive. Once you have a filesystem, mount the drive and see what happens. ……………………………………………………………………………………… #mount /dev/vinum/test /mnt #df Filesystem 1K-blocks Used Avail Capacity Mounted on /dev/ad4s1a 248111 55829 172434 24% / devfs 1 1 0 100% /dev /dev/ad4s1f 2032839 133492 1736720 7% /test1 /dev/ad4s1g 2032839 1266476 603736 68% /test2 /dev/ad4s1h 29497862 3974853 23163181 15% /usr /dev/ad4s1e 3048830 6757 2798167 0% /var procfs 4 4 0 100% /proc /dev/vinum/test 7282594 1 6699986 0% /mnt # ……………………………………………………………………………………… On the last line of the preceding df output, we see that our Vinum volume is ready and available for use, and that it’s roughly the combined size of the two smaller disks we used to create the volume! (If you want more human-readable output, try df -h to see disk space in a more friendly manner.) ……………………………………………………………………………………… /dev/vinum/test 7282594 1 6699986 0% /mnt ……………………………………………………………………………………… As you can see, concatenating disks is useful for combining several small disks into one large one, giving new life to older or smaller disks. While this doesn’t give us any reliability, it does help make older and smaller disks useful again. Other uses of Vinum are more interesting, such as striping. Let’s see how that works by turning these two disks into a striped system. Removing Vinum Configuration Vinum uses an accumulative configuration process. If you don’t destroy your old configuration, any new configuration is simply added to it. Accumulative configuration works well when you’re adding disks to a RAID system or restoring a damaged volume. It works poorly when you just want to start over. This is reasonable after all, most people in a production system alter a configuration more frequently than they erase it and start over! To start from scratch, you must erase the existing configuration from the on-disk database. You cannot just edit the configuration file, because it has no real relationship to Vinum’s internal operations. To get rid of our existing Vinum configuration and the mounted partitions, we must first unmount the existing Vinum partition. (It is always a poor idea to destroy anything while the operating system is using it.) Once the Vinum partition is safely unmounted, we can destroy the existing configuration. In the following example, we’ll use vinum’s interactive mode. Type vinum at the command line to drop into a vinum shell. 398
Note: If you are looking for good and high quality web space to host and run your application check Lunarwebhost Linux Web Hosting services
Note You can go into great detail about exactly how you want your volume set up, what sort of disk block and stripe size you want to use, and so on, but Vinum doesn’t require that level of detail. It defaults to sensible values. Creating the Volume Now we create the volume with vinum create and the filename to tell Vinum to read your configuration and create everything: ……………………………………………………………………………………… # vinum create /etc/vinum.conf v 2 drives: D alpha State: up /dev/da0s1e A: 384/3551 MB (10%) D beta State: up /dev/da1s1e A: 409/3786 MB (10%) w 1 volumes: V test State: up Plexes: 1 Size: 6543 MB x 1 plexes: P test.p0 C State: up Subdisks: 2 Size: 6543 MB y 2 subdisks: S test.p0.s0 State: up D: alpha Size: 3167 MB S test.p0.s1 State: up D: beta Size: 3376 MB # ……………………………………………………………………………………… Read your vinum(8) output carefully to see if the result matches your understanding of your configuration file. Our most important indicator of success is that every entry has the state of up, which indicates that the operation was successful. Let’s have a look at this output. First we see our two drives, alpha and beta, both of which are up (v). We also see the device names and the space usage. Then we have a list of volumes (w). Our one volume appears, named test, with a total size of 6543MB. Similarly, we have one plex, with two subdisks (x). The plex is named after the volume name, with a trailing “.p0″. Finally, we see our subdisks and the drives that they were created on (y). Each subdisk is named after its volume, the plex it is assigned to, and an arbitrary subdisk number, such as test.p0.s1. Our Vinum partition is now available. It will have a device name under /dev/vinum, named after the volume name. For example, our “test” volume is available as /dev/vinum/test. Initializing Vinum Partitions Before we can use our Vinum volume, it needs a filesystem. Since sysinstall doesn’t recognize Vinum (yet!), you need to do this with newfs(8). If you’re an experienced systems administrator, you can use any newfs flags you like to choose block sizes, and so on. The defaults work fine for most people: ……………………………………………………………………………………… # newfs -v /dev/vinum/test ……………………………………………………………………………………… 397
Note: If you are looking for good and high quality web space to host and run your application check Lunarwebhost Unix Web Hosting services
interactive mode. Note The program name being the same as the feature name can confuse people. When talking about the whole Vinum system, the V is capitalized. The program always appears in the manual section (vinum(8)). Vinum.conf Vinum stores its actual configuration in a database on the disk, and you cannot read it without using vinum(8). You can create a vinum(8) configuration file, however, which is useful when initially configuring Vinum. You can check your initial setup work more easily when you have a configuration file, and you can change and rebuild your configuration without too much trouble when you track everything in a configuration file. Once Vinum is configured, however, the configuration file is irrelevant; the actual configuration is stored in the on-disk database. You can make changes directly to the Vinum system without touching the configuration file. Do not look at the configuration file for current configuration information; remember to consult vinum(8) for the current setup. Note In this chapter, I’ll be using /etc/vinum.conf as a configuration file. If you come across this file on a system, just remember that it might not mean anything at all. Concatenated Plex We’ll start by setting up a single concatenated volume to give us the biggest possible disk size. We’ll use a single plex, or only one copy of the data. There is no redundancy in this model, but it will create a larger disk. Here’s an /etc/vinum.conf file for a concatenated plex: ……………………………………………………………………………………… drive alpha device /dev/da0s1e drive beta device /dev/da1s1e volume test plex org concat sd length 3243037k drive drive1 sd length 3457779k drive drive2 ……………………………………………………………………………………… Have a look at the first two lines in this listing. The first thing we do is identify the drive partitions with the drive keyword. The two drives in this example are /dev/da0s1e and /dev/da1s1e. (Remember, in Vinum terms, partitions dedicated to Vinum are drives.) Each drive needs a unique name, and I’ve named them alpha and beta. We then need to name our volume, or virtual partition, with the volume keyword. In this case, on the third line of the listing, I’ve named it test. Next, on the fourth line, we say how we’re organizing our plex with the plex org keyword. We use concat to specify concatenation. Finally, on the fifth and sixth lines, we tell Vinum the size of our subdisks on this drive. Remember, our original check with df showed us that /dev/da0s1e had 3243037KB available and /dev/da1s1e had 3457779KB available. We want to use all available space on these partitions for Vinum. 396
Note: If you are looking for good and high quality web space to host and run your application check Lunarwebhost PHP Web Hosting services