Upgrading Linux

Mark Roth

Most Linux distributions have upgrade paths with their installation. They are, however, usually only useful for a sub-release upgrade, such as from 7.1 to 7.2 or 7.3. However, I have yet to run into one that will successfully and cleanly upgrade from one release to the next full release -- they all suggest a clean install, including Red Hat and SUSE. When migrating from one distribution to another, regardless of whether you choose install or upgrade, the process simply fails because it is unable to successfully install numerous packages.

However, with some thought and a decent understanding of *Nix [1], it's possible to upgrade or migrate without too much pain and to fall back, if necessary, to your previous version without failure of multiple applications [2]. From my experience, the most important part of preparing for the upgrade is to think long and hard about how to partition your disks. I say that because, with proper design, you can have several versions on your system, all of which can be run with only minor single-user mode changes to your system.

Partitions

I recommend a small /boot partition; I'm currently assigning it 125 MB, and it should be a primary partition. Because this contains boot loader information, the initial ramdisk, and a few other files, none of which is huge (the 2.6 kernel is on the order of 2 MB), the size I've suggested will easily contain several versions of Linux and not be straining at the seams for years.

Next to consider is the root (/) partition, and this really should also be a primary partition. The reason I recommend that /boot and / be plain primary partitions is that, in case of partition or disk problems, it is far easier to recover with vanilla partitions, which have fewer layers of software and partition and directory tables to worry about. Also in the case of such problems, if you have / as a primary partition, coming up in single-user mode will give you a fully functional system, complete with all the standard tools.

Root doesn't really need to be too large, perhaps 2G-4G. This will allow more than adequate space for the several sets of backed up directories, which I'll discuss later. One factor that may affect this is whether you are building the operating system and utilities from source, since the /lib/modules directory is where much of the information is contained. In that case, you may want a larger root partition. Also, I have included /tmp under /, without making it a separate partition, and your use of /tmp may also affect the sizing.

Beyond this, my current practice is to create a large partition for an LSM volume. For those who are used to other variants of Unix, LSM is a virtual volume manager, the Linux version of LVM or Veritas Volume Manager. Here I will place the /usr, /var, /home, and /opt directories. I place the /home directory here, because this is where much unique information is, and this will allow us, if necessary, to reformat the other partitions without bothering this one [3]. Of course if your database is hung off /, as Oracle prefers, that should obviously be another partition.

Note that if this system is on an intranet, and users' home directories are to be mounted from another server, there's no point in creating a virtual partition for that. Simply create the directory off / for use as a mount point, and save the extra space as unused LVM space. It can be added later to another LVM partition that might grow to need it.

The reason for creating the other three directories as separate partitions is this: for an upgrade, and assuming you have the disk space to spare, you can create another LSM volume, or new partitions in the existing one, and then mount those on their respective mount points. Thus, by pointing LILO or GRUB to the correct partitions, most of whichever version you want to run is there in its full glory.

The last step is a bit more nerve-racking. Before rebooting the system into an installation, you'll need to take it down to single-user mode and unmount swap, then you must do two things. First, copy or rename /etc, /lib, /sbin, /bin, and /dev to something obvious, such as /etc_rh9. You could actually leave /lib alone; however, during the installation, many symbolic links will change, so if you have to fall back, links such as glibc.so, which will now be pointing to the new glibc, will have to be reset to point to the old one. Second, reboot with the installation CD/DVD in your drive with that as the first boot device. Don't worry about failures in the shutdown procedure, because those all involve attempts by the system to save state to disk.

Installation

From here, you can follow the installation as a new install, as long as you choose to custom partition the disk and prevent the partitioning tool from formatting the partitions and directories that you've saved with the renaming. There are several reasons for this. First, you want to preserve the state of the system, so that if you do need to fall back to the previous version, you can simply go to single-user mode, rename the current directories to *_new, rename the old ones to their original names, and then reboot after assuring that the boot menu option mounts the correct partitions for /usr/, /var, and /opt.

A second reason for the rename is that during the install procedure, as the rpms are installed, some may fail if they find existing files (especially in /dev and /etc). Or, in the case of a migration, some files in the original distro may have been single configuration files, while in the new distro, they will exist as directories containing individual configuration files.

Finally, performing an install rather than attempting an upgrade, will add the new maps, files, and directories to /boot, rather than replacing the existing ones. This leaves them available for fallback.

Maintenance

With the above partitions in place, future upgrades should be far less prone to problems, whoever maintains the systems will have a simple script to follow, and the preparatory work for upgrade or fallback will be a 20-minute or so procedure, rather than hours of analysis and preparation. Further, there is no danger of losing data in the /home directories, which will add vastly to the comfort level of your users.

At some point, however, you should remove older versions of files, perhaps those from before the release from which you have just upgraded. This will prevent confusion and possible problems as well as free up disk space. Remember, if you want to perform rpm uninstalls, that if you've gone from Radish Linux 17.4 to Baluchatherium Linux 15, but you previously had Radish 17.1 and 17.2, you'll need to fall back to Radish 17.4 to uninstall the 17.1 and 17.2 releases. This is because, if you followed my procedure, the rpm database for Baluchetherium will be brand new and will contain no information related to Radish.

If you have an emergency need to fall back, the procedure is the reverse of what we did to perform the upgrade. To begin, take the system down to single-user mode and make sure swap is off. Next, rename the current /etc, /lib, /sbin, and /bin to /etc_new, and so forth. Then, rename the old ones from /etc_RHEL3 back to /etc and reboot. Remember that you don't need to worry about these directories, because they've been hidden by their use as mountpoints for the new release. Since you performed an install, rather than an upgrade, you should be presented by the bootloader with a choice of boots. Choose the old one that you want to bring up, and its configuration files will mount the correct partitions, LSM or otherwise. Then you should be where you were before the upgrade.

One thing that I have not discussed is the automation of varying releases on boot. This is something that should probably be considered either in a production environment or where you may be alternately booting between a bleeding-edge distro and a stable one. Although I have not found this necessary, it may be possible to handle this in the low-numbered start scripts under rc0.d or its equivalent. It certainly cannot be done any later in the boot process. Note that this is yet another reason why you will want / as a primary partition; you will have early access to the various versions of /etc that you're maintaining as well as their vital files, such as fstab.

Note that the procedure I've outlined here will not solve all your problems on upgrades, because the applications and utilities that were not part of the new distribution very possibly may need to be upgraded or rebuilt. This is especially true if there has been a major change to glibc, the standard C library used by almost everything, or to other commonly used libraries. In-house libraries will almost certainly require a rebuild, but this should be fairly straightforward, unless you are leapfrogging several kernel releases.

Another detail to consider is that the distro itself frequently does not contain the latest releases of other open source software; in that case, you have another item on your task list. For example, in the case of SUSE 10 to which I migrated, I immediately had to upgrade from the Firefox 1.0.6 that came on the CDs to 1.5, if for no other reason than to resolve its insistence on blocking the pop-up compose window for my ISP's Webmail.

Notes

1. Or, as Æleen Frisch, in her indispensable Essential Systems Administration (O'Reilly) refers to it, "The Unix Way".

2. Such failures are almost always the result of libraries being upgraded, with the new libraries having incompatibilities with the old functions or utilities that depend on the upgraded libraries. I remember, for example, several years ago, a complete failure when trying to upgrade Abiword 1.x to Abiword 2, where the aspell and ispell utilities broke everything.

3. I will pass over here the decision on LSM-assigned striping, or RAID.

Mark Roth has been a computer professional in Unix, DOS & Windows, and mainframes for more than two dozen years.