SELinux bites man: a story
A co-worker recently came to me with an interesting and mysterious
problem. He was setting up a Red Hat Enterprise 4 based machine with
MySQL, using a default setup except he'd changed the location of MySQL's
data directory from
/var to a different partition. Now MySQL wasn't
The symptoms were really funny: the init.d script installed by the
system didn't work, except if he ran it by hand with '
sh -x', it did.
So he renamed it out of the way and grabbed a copy from another RHEL4
machine with a working MySQL, which worked. But it turned out that
the two scripts were identical. So why did the new one from the other
machine work but the old one from the install not work?
(This was where he called me in.)
Fortunately I had been recently reading a series of articles on
SELinux; something about the whole situation tickled
the back of my mind, and a little light labeled 'file contexts' lit up.
lsattr showed that the two scripts had different contexts; the
'as installed' one had a special context, and the one copied from the
other system had a generic one.
And this was the problem: the SELinux MySQL context lacked the magic SELinux permissions to access the new data directory location, because (of course) the new location hadn't been SELinux labeled as a MySQL area. However, the normal root context could access everything fine.
(In Red Hat's SELinux setup, many daemons are deliberately run with extra SELinux-imposed restrictions so that if someone finds and exploits a vulnerability in the daemon it does less damage.)
So when the original MySQL init.d script was run directly, it switched
into the MySQL SELinux context and failed to access its data directory.
However, special contexts on a shell script only get switched into when
you execute the shell script directly, so when the original script was
run via '
sh -x' it was instead running in the normal root context,
could access its data directory, and worked fine. The copied script
always ran in the normal root context since it was not specially
labeled, so it worked fine all the time.
(We tested this guess by running the original init.d script with just
sh mysql start' instead of '
sh -x mysql start', and it worked
fine then too.)
I believe my co-worker's workaround was to turn off SELinux, on the grounds that he didn't want to try wrestling with that particular pig right then.
Pointers to some SELinux explanations
SELinux is one of those things that have been cropping up on my radar ever since I had to start telling the Red Hat installer not to turn it on. (I kept not enabling it because changing an entire security architecture of a system is not to be done lightly, even when it's a new system I'm setting up.)
SELinux is an imposing system with equally imposing documentation. Fortunately, recently Dan Walsh of Red Hat has been posting some very useful (to me) 'SELinux for beginners' documentation:
- a basic introduction
- How does SELinux enforce policy?
- Applications that work with SELinux
- File contexts and mv/cp/install
- How logging in and so on work
/etc/selinux/configfile and how to change what SELinux level your system uses
- config files in general
- Managing file contexts
- the restorcond daemon
- SELinux manpages and AVC messages
- Role Based Access Controls (RBAC)
- Using RBAC in a MLS policy
- more on Module handling
- SELinux reveals bugs in other code
- Loadable Modules - File Context
There's also a Fedora Core 5 SELinux FAQ, with links to other FAQs. However, from reading through it I think Dan Walsh's stuff is easier to follow.
(This entry is a bit belated, because Dan Walsh didn't so much wrap up his series of entries as stop writing them, which I can't exactly blame him for, and I was sitting on it until the series was 'complete'.)