Trying to explain my harshness on configuration management tools
It started with a Tweet that became a conversation:
@thatcks: I'm biased but if a CM system needs more than an apt-get to install I'm not going to use it to set up our Ubuntu machines. Sorry Ansible.
@jgoldschrafe: That's kind of a silly approach - most modern CM tools are evolving so fast that it's folly to stick to outdated versions
@thatcks: For us it's an issue of both (long-term) risk and effort, and expected payoff from using a CM system for system setup.
(By needing just an apt-get to install, I mean that the CM system must be in standard Ubuntu repositories. Adding a PPA is one step too many.)
To start with I'm restricting this to (initial) system setup, the stuff where you go from bare metal to a working system. As I've mentioned we already have a mostly scripted system for this, build instructions for all machines, and a central repository of everything we need to rebuild a machine. Our build instructions are almost all quite short and basically amount to 'install some packages and put some configuration files into place'. The best case for using a CM system for initial system setup is that it reduces all of these extra build steps down to 'install CM system; run CM system once' (a script can make that one step). But given that the steps are pretty simple to start with, the payoff to doing this is relatively low and automation adds its own overhead that further reduce it.
(There are various ways in which the payoff is not zero and using a CM system for initial setup opens the door to using it for ongoing changes, which has additional potential advantages. The idea was attractive enough that I considered experimenting with it with Ansible until I found out the problem.)
Anything beyond a simple apt-get for the install adds friction to using the CM system. This friction directly reduces the payoff of using the CM system in the first place; at the worst, setting up the CM system takes about as many steps as the by-hand build instructions would. At this point my co-workers would sensibly reject it out of hand. In addition, using any sort of unofficial package or direct local build (I've seen people suggest cloning git repos, for example) leaves us exposed to a collection of risks. With real Ubuntu packages there is in theory some sort of tracking of upstream security issues, some degree of quality control, and some degree of stability over time. None of these three are guaranteed for PPAs or other distribution mechanisms, which means that ultimately they become our responsibility (at a minimum we get to check that our package supplier is doing them and hasn't gotten busy with other things).
(Here I want to mention that we're unlikely to consider it a feature that a core component of our build instructions keeps evolving rapidly on us. We like stability for that sort of stuff.)
For me in our environment, the result is that an unpackaged CM system is not worth the hassle for system setup. The likely payoff is too low and the risks, however moderate, are not appealing. We already have a system that works; we don't really need one that works maybe a bit better but is a lot more complicated.