Wandering Thoughts archives

The brute-force way to install missing Solaris 9 packages

Every so often I wind up having to fix up a Solaris 9 machine that doesn't have Sun's wget or the like installed. Presumably there is an official way of dealing with this, but so far I have just used my unofficial brute force approach.

Solaris packages in their native form are just subdirectories (with piles of stuff), with the directory name being the package name; on the Solaris 9 CDs, they're found in the subdirectory Solaris_9/Product. One of the formats pkgadd -d will deal with is just this, a directory full of package subdirectories.

So all you need to do is get the subdirectories for the packages you want into some scratch directory on the target system and point pkgadd -d at it. What I usually do is mount the Solaris CD-ROMs on some handy Linux system, bundle up the directories I need with tar, scp the tarball to the target, unpack it into a new subdirectory in /tmp, and run pkgadd -d /tmp/pkgs.

(If I think I may need the result on several systems, I save the tarball, so I don't have to go fishing around on the CD-ROMs again. For example, I have a carefully salted away one with SUNWfns.)

The one thing to watch out for is unexpected dependencies; pkgadd will prompt you about whether you want to go on even though they're missing, and you want to say 'no' (and then go find them and bundle them up too).

(You can see the full package dependency list for any package by examining its install/depend file, which is usefully in plain text.)

PS: wget needs both SUNWwgetu and SUNWwgetr. In our set of CD-ROMs, they are on the 'Solaris 9 Software 2 of 2' CD-ROM. SUNWfns is on the first Software CD-ROM, along with SUNWfnsx.

'In place' filesystem defragmentation with Disksuite

While the Berkeley FFS and derivatives like Solaris UFS are much, much better at dealing with fragmentation than the original V7 filesystem (which continued on into System V before SysVR4), they can actually get fragmented over time to a degree that matters.

Traditional Unix, Solaris included, really doesn't have any tools for defragmenting filesystems; instead, you get to to do it the brute force way, by copying everything into a clear filesystem. With Solaris Disksuite and mirrored disks, it is possible to do this 'in place', so that you don't have to copy twice or remount the filesystem on any NFS clients.

The procedure is slightly less nerve-wracking if you have a three way mirror, but should work OK even for a two-way mirror. Here's how it goes:

1. bring the machine into single-user mode, but do not unmount the filesystem you want to defragment.
2. insure that all the submirrors are in sync.
3. metadetach all but one submirror.
4. rm everything in the filesystem except the lost+found directory.
5. ufsdump 0f - /dev/md/rdsk/<detached submirror> | ufsrestore rf -
6. check and make sure the data is there. No, really, do it twice.
7. metattach the detached submirrors.
8. bring the machine back up multi-user. (Optionally, first wait for the resync to finish.)

You're done. NFS mounts are even still intact, although anyone with an open file or a current directory in a subdirectory of the filesystem will see a few small problems.

(In our case it was a mail spool, so there weren't any subdirectories to worry about.)

There should be a similar procedure with Linux software RAID, although it'll probably be slightly more troublesome since the Linux equivalent of metadetach is somewhat more abrupt. (I'd probably unmount the filesystem before ripping the mirror off.)

PS: I am not sure if I am grumpy that Sun didn't make 'metattach' be called 'metaattach' instead, for complete consistency among the Disksuite command names.

A Solaris 8 Disksuite single user mode surprise

If you boot a Disksuite-using Solaris 8 machine into single-user mode to do maintenance and do a metastat, you'll discover that all of your mirrored metadevices are marked as needing to be metasync'd, even if they actually are fully consistent.

What seems to be going on is that Disksuite doesn't update things from the on-disk metadata state database when the kernel brings up the metadevices themselves in early boot. Instead, it defers this until you explicitly run 'metasync -r', which is normally done in /etc/init.d/lvm.sync, which is only run as part of going into runlevel 2.

(At least I assume that the kernel is bringing up the Disksuite devices itself in early boot, since these machines have their root filesystem on Disksuite mirrors. I am not quite up on the black box of early Solaris boot.)

The fix is pretty simple; once you're up in single-user mode, just remember to run '/etc/init.d/lvm.sync start' before you start futzing around much with the disks.

(Our experience is that it goes like lightning unless something is genuinely troublesome, which is about what you'd expect. But check with metastat afterwards, just to be sure. You probably don't need to do this if you're bringing a system down from normal operation into single-user mode, just if you're booting straight into single-user mode, but I haven't tested this to be sure.)

This makes a certain sort of sense from the right viewpoint, since it means that the system is doing as little as possible when coming up into single user mode. I have no idea how the kernel picks what to write to when it has to write to a metadevice, though. And it does mean you have to remember an extra step for most routine boots in single-user mode.

(The good news is that the excitement this caused us when we stumbled over this will probably insure that I don't forget this any time soon.)