Ubuntu, illustrating how to utterly fail at kernel security updates
Just like last time, this is a rant.
Here's a simple procedure that you too can follow:
- develop fixes for some CVE issues in the various kernel versions that you maintain.
- announce and release new kernel security updates for most of your currently supported distributions, but not for 10.04 LTS. Trained sysadmins running 10.04 will ignore them, and even if they don't there's no 10.04 updates for them to apply.
- announce and release new kernel security updates for specialized
versions of 10.04, like the one that runs on Amazon's EC2.
Trained sysadmins will ignore
them and even if they don't, there's still no general 10.04 update.
- realize that oops, you forgot to do a general 10.04 kernel update.
- quietly put a kernel update into your security repository. Do not announce it on your security mailing list, because that might be embarrassing. Besides, who reads those things anyways? People using Ubuntu should just install every available update, and right away; as we've already established, putting useful details in kernel security announcements is optional anyways.
(Yes, I checked the ubuntu-security-announce archives just to make sure that our mail system hadn't swallowed an announcement.)
Of course, as a standard thing it appears that the Ubuntu changelog for the kernel package doesn't necessarily include a list of the CVEs that are fixed in any particular revision of the package. It would make sense that the specialized versions have the same bugs fixed as the main one, but given past issues it's hard to tell unless I spend far more time on this than I have any interest in doing.
Right now, I am very, very angry with Ubuntu's slipshod practices. We run Ubuntu on servers, in an environment where we can't just reboot machines because Ubuntu feels that we should; we really do need to be able to assess the importance of security updates, especially when many of the bugs don't even apply to our specific environment (they require protocols we don't use, hardware we don't have, or the ability to plug devices with maliciously corrupted filesystems into servers in a machine room).
PS: my story may not be exactly what happened (see eg), but from the outside it sure looks like a plausible story.
PPS: there was also a stealth 8.04 kernel security update, but it's less clear what's going on with that one. On one hand, the package changelog between 2.6.24-29.90 and 2.6.24-29.91 lists a whole bunch of CVEs. On the other hand, many of them are old. Did Ubuntu miss a significant set of security updates for the 8.04 kernel, and only notice things now? Ubuntu 10.04 had fixes for at least some of these CVES back in March.
(See also the tracking bug for the 8.04 update.)
Update: Ubuntu has now released two notices for this kernel update, for the 10.04 version and for the 8.04 version. It remains the case that Ubuntu published security updates in the repository well before an announcement was made (apparently surprising some people), and that they let a general kernel package update lag well behind at least specialized version updates.
Something to remember: HTML forms are anonymous
By and large, web programming frameworks have settled on a common
model of handling HTML forms. You have named (or typed) forms
with named and typed fields and you use the framework to render
them into HTML and extract them from
POST responses. Django programmers, for example, have a
familiar, reflexive idiom:
class MyForm(forms.Form): name = forms.CharField(...) ... def handle_my_url(request): if request.method == "POST": form = MyForm(request.POST) if form.is_valid(): ...
This simple, clear approach is misleading. It's misleading because it
makes the whole process look sort of like storing objects, which means
that of course you're only going to get a valid
MyForm back from the
HTTP POST if you actually put one there in the first place (or the user
made up the POST data themselves to fool you, which you can ignore).
The thing is, HTML forms are anonymous. In their natural state,
the only way you can tell different types of forms apart is by the URL
they are sent to and the names of the form fields that they have. There
is nowhere natural in a HTML form where you can say 'this is a MyForm
form'; in general, you have to infer that from the fact that it has
all the fields that MyForm has and is
POST'd to a URL that expects a
(Your web framework may be adding a hidden label field that it uses to be sure, but you have to check the generated HTML in order to know for sure.)
So suppose that you have two different forms with the same form fields; this means that the only way to tell these forms apart is by the URL that each of them uses. If they use the same URL (for example because there are alternate versions of the page, with forms that have a different meaning), you can't tell them apart at all. You can render a page with a 'MyForm1', have the user POST it back, and happily retrieve a 'MyForm2' from the POST response. Although these two forms looked like they were distinct and different objects in your code, in HTML they are actually the same thing.
(It's as if your programming language ignored the type of things when doing 'is-a' and equivalence checks and only checked that two instances had all of the same fields. There are languages that work this way; I believe the term of art for it is 'structural equality'.)
All of this is abstract sounding, so let me give a concrete example where I almost shot my foot off this way. Our account request system allows privileged users to do two very special operations to requests: if a request is marked as having been either accepted or rejected, you can reset it to 'pending', and if a request is pending you can immediately delete it. In both cases you need to confirm that you really do want to do this by ticking off a checkbox, and both operations are done from the same 'detailed information about this request' URL; which option the page gives you depends on the request's state.
So we can create two forms:
class ReallyRevive(forms.Form): yes_really = forms.BooleanField(...) class ReallyDelete(forms.Form): yes_really = forms.BooleanField(...)
Then we write code that tries to get and validate a ReallyRevive form
POST response if the request is not pending, and do the same
with a ReallyDelete form instead if the request is pending. And we have
just created a dangerous race.
Suppose that two privileged users are both trying to revive the same request at the same time. Both see the page rendered with a ReallyRevive form, both tick the checkbox, and both submit the form, one somewhat after the other. In the first form submission, the code retrieves a valid ReallyRevive form and sets the request back to the pending state. In the second form submission, the code successfully retrieves a valid ReallyDelete form from the POST response despite the fact that it is actually a ReallyRevive response, and immediately deletes the just-revived request. Oops.
(You can see this as a REST violation if you want to. My view is that these things happen in practice so I should be aware of the bear traps waiting in the underbrush.)
The solution is to give your forms different field names; here we would
really_revive checkbox in one form and a
checkbox in the other.