Packaging systems should support overlays

January 15, 2010

One of the very common things that sysadmins do when dealing with more than one system is that we develop a common set of files that we slap on every system (or in large environments, on subsets of our systems). Sometimes this is done by hand, sometimes this is done with hand-built tools, and sometimes this is done with things like Cfengine or Puppet.

If you accept that packaging systems should be comprehensive, then packaging systems should also support this common need. Specifically, packaging systems should have the idea of 'overlays', packages that replace files from other packages; this would allow sysadmins to build one or more overlay packages that contained the various collections of their necessary customizations and localizations.

(Ideally overlays would be slightly more general, allowing you to at least remove files as well.)

There are several advantages of doing this through the packaging system. The first is that all of the package manager querying and verification tools will now work on your files as well; you can see that a particular file comes from this version of your localizations and has not been changed from what it should be, you can see what all of your localizations on this system are, and so on.

(As far as I know, Puppet and Cfengine do not support this sort of state querying.)

The next is that upgrading system packages can now be much more aware of your local customizations; the package manager does not have to guess what to do when upgrading things you've changed (or just ignore your changes). It becomes trivial to maintain your local overlays across base package upgrades and to block these upgrades if your local overlays aren't ready for them. Equally, you can use the power of the package manager when changing your local overlays, doing them as overlay package upgrades. Thus you can require base package upgrades, add and change dependencies, detect conflicts, and so on.

(The implication of this is that the package manager can also show you the differences between the overlay-supplied version of a file and the base version, since the package manager needs to keep the base version around to restore it in case you remove your overlay or the next version of the overlay package does not overlay this particular file any more.)

Comments on this page:

From at 2010-01-18 08:25:40:

The first difficulty inherent in this approach is that if you want your package manager to be truly authoritative, a package overlay needs to explicitly understand and document what versions of something it is depending on. It's absolutely possible that during, for example, an upgrade from RHEL 5.3 to 5.4, a version of something is upgraded in a way that doesn't necessarily preserve binary compatibility, some trivial bit of configuration syntax, or some other important detail that can keep your overlay from working.

The second is that not all systems, even ones deployed by the same organization, are going to be heterogeneous. A server in the DMZ, for example, may very well rely on different servers for NTP and LDAP, may use LDAP for authentication instead of Kerberos, and other important details that are difficult to express in package overlays but incredibly simple to express in a comprehensive configuration templating system. Say you have seven high-availability clusters running OpenAIS/Pacemaker as the CRM; each cluster resource manager needs to be specifically configured, and needs to have a listing of services, groups, dependencies, and other nodes in the cluster. Do you create seven separate overlay packages, or one template, and use appropriate variables in your configuration management system? Which is really more maintainable?

I know you're tired of hearing me harp on about Puppet and Cfengine over and over, but unless the configuration management functionality was actually rolled into a monolithic package manager, they're very often the right way to go. :)


Written on 15 January 2010.
« Packaging systems should be comprehensive
Different visions of what packaging systems are for »

Page tools: View Source, View Normal, Add Comment.
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Fri Jan 15 00:17:03 2010
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.