Wandering Thoughts archives

I hate hardware (Dell 2950 edition)

The Dell 2950 is a decent 2U server that has recently started being popular around here; like most recent servers, it has dispensed with PS/2 connectors and only has USB for the keyboard and mouse. It has four USB connectors, two on the front and two on the back.

If you plug your keyboard into the back USB connectors, the Ubuntu 6.06.1 x86_64 server kernel hangs with 'BUG: soft lockup detected on CPU#0!'. (Sometimes it recovers from this hang, sometimes not.)

If you plug your keyboard into the front USB connectors, everything works great.

I love hardware. I really do.

(Don't ask how long it took to find out just why our 2950s were locking up when we tried to install Ubuntu, or how close we came to pitching Ubuntu out a window, as other distributions had kernels that worked fine.)

Sidebar: where this error message comes from

This message comes from kernel/softlockup.c and gets generated if the CPU's kernel watchdog thread hasn't run in ten seconds. (Kernel watchdog threads are the 'watchdog/<N>' processes in ps output.)

This means that some bit of code has locked out scheduling for those ten seconds. Usually this means both that the code has a bug and that it has run into some sort of hardware problem it wasn't expecting, since people writing kernel code rarely knowingly allow ten second stalls.

An inspection of what stack backtrace I can recover from the system is not particularly revealing as to what driver or bit of hardware might be at fault. It runs more or less:

atkbd_connect → atkbd_activate → i8042_interrupt → ps2_command → ps2_sendbyte → i8042_kbd_write → _spin_unlock_irqrestore → idr_get_new <interrupt>

(Transcribed by hand and accuracy not guaranteed, especially as I have omitted the offsets, which means that the code might actually be in a nearby static function or something. The Ubuntu 6.06.1 server install kernel is some version of 2.6.15, and there have been fixes to lib/idr.c since then.)

RPM tricks for dealing with multiarch machines

Why do we need some new RPM tricks for dealing with multiarch machines? Simple:

; rpm -q glibc
glibc-2.5-10.fc6
glibc-2.5-10.fc6

That's not an error, that's just my machine having both the 32-bit and the 64-bit glibc RPMs installed. On a multiarch machine it's routine to have RPMs with identical names, and rpm itself hasn't been fully adapted for this situation.

The big RPM trick for dealing with a multiarch machine is to know that you can pick out a specific architecture's RPM by putting the name of the architecture on the end of the name of the RPM:

; rpm -q glibc.x86_64
glibc-2.5-10.fc6

(You can use this format with yum as well as with rpm.)

The other trick is using RPM query formats to actually show you which architecture a specific RPM is for. Query formats are sufficiently complicated that a full explanation is well beyond the scope of this entry, but the basics are reasonably simple:

; rpm --qf '%{NAME}.%{ARCH}\n' -q glibc
glibc.x86_64
glibc.i686

The normal 'rpm -q' output format is '%{NAME}-%{VERSION}-%{RELEASE}\n', or '%{N}-%{V}-%{R}\n' in the compact form (technically this is slightly incorrect because it omits the epoch, but that almost never matters). To add the architecture you use an explicit query format of '%{N}-%{V}-%{R}.%{ARCH}\n', and for many purposes just the name and the architecture is enough, so you can just use '%{NAME}.%{ARCH}\n'.

(And the '\n' is important. Many a time I've accidentally left it out and gotten all of the output smashed together into one line.)

There is probably something you can set up in your .rpmmacros file to make rpm use this query format by default, but I don't know it. You can just make a cover script instead, so you don't have to type the query format all the time:

; cat rpmq
#!/bin/sh
exec rpm --qf '%{N}-%{V}-%{R}.%{ARCH}\n' "$@"

(My fingers are busy telling me that this should also supply the '-q' argument too, so I can just do 'rpmq kernel' and it'll work.)

An irritation with yum localupdate

One of the few things that keeps me running while I am only on dialup is 'yum localupdate'. With a combination of 'yum check-update' and 'yum localupdate' I can get a list of what RPMs I need to update, pull down copies of those RPMs at work, dump them onto a USB key, take them home, and then update them locally.

In theory.

In practice, 'yum localupdate' seems to sometimes ignore RPMs that I've supplied on the command line if they're merely things that need to be installed for dependencies. Then, of course, it insists on downloading them again, over my slow PPP link, which causes me to use rude language, hit ^C and ^\ a lot, and (this time) finally end up using kill -9 on yum to get it to stop (insert rant here).

Part of the problem is that yum localupdate doesn't really give any clear picture of what it's going to do, since it is hijacking status displays not intended to draw a distinction between stuff that will be downloaded and stuff that doesn't need to be; this means that things like the 'amount to be downloaded' line have no useful information.

(Perhaps this is a sign that I should just use 'rpm -U', since I suppose it's not as if yum is doing anything for me in this situation; I already should have the full set of dependencies.)

A modern environment's need for broadband

I have an embarrassing confession: my home Internet connection is still a dialup 28.8Kbps PPP link. (Back when I first set it up, this was only slightly behind the times, as 33.3Kbps modems had just come out.)

For a long time this was fine, because all I really did with it was text-based stuff, mostly sshing to my machines at the university; while I did do some web surfing, pretty much all of the sites I visited regularly were mostly text. As time went by, along with webcomics and streaming Internet radio and big PDFs and so on, things got more awkward. But I could still make do, and I'm both stubborn and lazy so I stuck with my PPP link.

(I run both ends of my PPP link myself, so it does exactly what I want it to. This matters a lot. Also I can defer bandwidth consuming things until I'm at my office machine.)

Then recently I moved to a modern Linux version with a wide package selection that got frequent updates (ie, Fedora Core 6), and it rapidly became obvious that dialup just doesn't cut it any more. Modern environments really need broadband; there are just so many things that assume you have abundant bandwidth.

(Fedora is particularly bad, because the index files yum downloads are over a megabyte per repository. This makes 'yum check-updates' one of those things that I have to plan ahead of time.)

I can limp along for now, using various baroque coping strategies, but it's clear that broadband is in my future (and no later than Fedora Core 7; I don't want to think about trying to upgrade without it). And you just can't experiment with the huge package selection in Fedora Extras over a dialup link, which is no fun.

(Due to local circumstances, broadband for me means DSL, which means I have to pick which local DSL provider I want to deal with, since the local pseudo-monopoly provider is not suitable as they have recently introduced a (relatively low) monthly transfer cap for new customers. Which is a massive headache, since there are so many choices and options and whatnot.)