Ubuntu 16.04's irritatingly broken MySQL updates
So, Ubuntu 16.04 can't apply MySQL server updates if you have the server installed but have disabled it running. Good show, you lot.
We install the MySQL server package on a few machines but deliberately don't start the daemon. In older versions of Ubuntu, this worked reasonably well; you could do it, you could keep the daemon from starting on boot, and you could apply updates (although doing so generally started the daemon up, so you had to remember to then go stop it). In 16.04, if you've disabled the daemon your attempts to apply updates will error out:
mysql_upgrade: Got error: 2002: Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) while connecting to the MySQL server Upgrade process encountered error and will not continue. mysql_upgrade failed with exit status 11
The direct cause of this problem is that the mysql-server-5.7
postinstall script needs to run
mysql_upgrade, which requires
the server to be running. Perhaps at this point you sigh, run
service mysql start', and try the upgrade again. It'll still
fail, because the postinstall script is more complicate and more
wrong than that.
The postinstall script needs to stop the MySQL daemon, do some
things, and then start the daemon again and run
(and then restart the daemon yet again). It does all of this starting
and restarting by running
specifically refuses to start disabled daemons. In the grand Unix
tradition, this behavior is burried in an innocuous phrasing in
invoke-rc.d is a generic interface to execute System V style init script /etc/init.d/name actions, obeying runlevel constraints as well as any local policies set by the system administrator.
Via a complex chain of actions, what 'obeying runlevel constraints'
translates to here is that if you do '
systemctl disable <whatever>',
invoke-rc.d will decided that <whatever> is specifically blocked
from running and not start it.
(Invoke-rc.d in general is the wrong tool in Ubuntu 16.04, because
it's actually fairly tied to the System V init framework. The system
goes through ugly hacks in order to make it work 'right' on things
that are actually native systemd
.service units, as the MySQL
This selectivity is the wrong approach, or at least it's in the
wrong place. What the postinst script should really be doing is
unconditionally shutting down the server, unconditionally starting
it to run
mysql_upgrade, unconditionally shutting it down again,
and only then using
invoke-rc.d to conditionally start it again.
This would achieve the twin goals of upgrading MySQL while not
leaving the daemon running if it's disabled. This would actually
be an improvement over the 14.04 situation, instead of a massive
(Of course I expect that the real answer is simply that no one thought about this possibility, and that if we were to file a bug report we'd be told that disabling the daemon is not a supported configuration.)
The workaround is simple. Before you try to apply a MySQL server pack
update, do '
systemctl enable mysql'. After it's done, do '
disable mysql; systemctl stop mysql' to return to the original state.
Sidebar: 'Runlevel constraints' and
Invoke-rc.d checks to see whether something is enabled or disabled
by looking for S* and K* symlinks in
16.04, the 'runlevel' is arbitrary and is reported as '5', so we're
/etc/rc5.d. When you do '
systemctl disable' or
systemctl enable' on an Ubuntu 16.04 system and the service also
/etc/init.d file, systemctl helpfully maintains rcN.d S*
and K* symlinks for you. So running '
systemctl disable mysql'
also creates a
/etc/rc5.d/K02mysql, which invoke-rc.d will then
see as saying that mysql is specifically constrained to not start
in runlevel 5, and so should not be started.
(If there was no
/etc/rc5.d symlink at all, invoke-rc.d would
also conclude that it shouldn't start the MySQL daemon.)