What OSes we use here (as of May 2020)
I was recently asked on Twitter if we had changed our 2012 views on what OSes have succeeded or failed here, especially the choice of Ubuntu versus RHEL/CentOS. The short version is that we haven't, and if anything we've reduced the number of OSes we've used. However, I think it's perhaps more useful to run down the ecological niches where we use various Unixes (and it's certainly something I haven't done explicitly before).
Our primary OS is Ubuntu LTS. It's our default and we use it on almost everything. If we were starting from complete scratch, we might well use it for even more services than we already are. Our login servers, our compute servers, our web servers, our email servers (both IMAP and the actual mailer machines), and so on are all Ubuntu LTS. Even our current generation of NFS fileservers are Ubuntu LTS.
CentOS, specifically CentOS 7, is used in two places. First, it's the best and most recent OS the vendor supports for our commercial anti-spam system; while they sometimes support relatively current versions of Ubuntu, they don't update their support fast enough to account for the churn in supported LTS versions. Second, we use it on our long lived centralized syslog servers and console server. We like these machines to run for a very long time without having to be reinstalled, and CentOS fits that bill.
OpenBSD is used in an assortment of roles, some of which are historical. Our largest use is for firewalls, where we value its stability and we really like OpenBSD's PF firewall rule system (and we also have a lot of experience with it and a lot of current PF firewall rules). This extends to our VPN servers (both OpenVPN and L2TP), because they have firewalls as part of the VPN service and in general we've had good experiences with OpenBSD based VPNs. We also use OpenBSD for our internal DNS resolvers, our official public DNS primary, and as the DHCP server for most of our internal 'sandbox' private networks.
(Our DNS stealth master and the DHCP servers for our 'laptop network' and our wireless network are all Ubuntu LTS machines, though.)
We're most strongly attached to OpenBSD for firewalls and for VPN servers. It's possible that we'd move other services from OpenBSD to Ubuntu in the future, but at the same time our motivation for doing so is low. For all that I've griped about this issue in the past, moving from OpenBSD version to OpenBSD version is generally very easy for us; almost all of the time, everything just drops into place in the new version. Moving to Ubuntu LTS would likely be more work.
Our existing CentOS logging machines work well enough and we like their stability, so when the day comes that CentOS 7 drops out of support we're pretty likely to replace them with more CentOS machines running current versions. It's possible that at some point we will stop using our current commercial anti-spam software; if that happens, we would switch to relying on rspamd and ClamAV, which we already have running in parallel on an Ubuntu machine.