2018-05-23
Almost no one wants to run their own infrastructure
Every so often, people get really enthused about the idea of a less concentrated, more distributed Internet, one where most of our email isn't inside only a few places, our online chatting happens over federated systems instead of Twitter, there are flower gardens of personal websites and web servers, there are lots of different Git servers instead of mostly Github, and so on. There are many obstacles in the way of this, including that the current large providers don't want to let people go, but over time I have come to think that a large underappreciated one is simply that people don't want to run their own infrastructure. Not even if it's free to do so.
I'm a professional system administrator. I know how to run my own mail and IMAP server, and I know that I probably should and will have to some day. Do I actually run my own server? Of course not. It's a hassle. I have things on Github, and in theory I could publish them outside Github too, on a machine where I'm already running a web server. Have I done so? No, it's not worth the effort when the payoff I'd get is basically only feeling good.
Now, I've phrased this as if running your own infrastructure is easy and the only thing keeping people from doing so is the minor effort and expense involved. We shouldn't underestimate the effects of even minor extra effort and expense, but the reality is that doing a good job of your own infrastructure is emphatically not a minor effort. There is security, TLS certificates, (offsite) backups, choosing the software, managing configuration, long term maintenance and updates, and I'm assuming that someone else has already built the system and you just have to set up an instance of it.
(And merely setting up an instance of something is often fraught with annoyance and problems, especially for a non-specialist.)
If you use someone else's infrastructure and they're decently good at it, they're worrying about all of that and more things on top (like monitoring, dealing with load surges and DDOSes, and fixing things in the dead of night). Plus, they're on the right side of the issues universities have with running their own email; many such centralized places are paying entire teams of hard-working good people to improve their services (or at least the ones that they consider strategic). I like open source, but it's fairly rare that it can compete head to head with something that a significant company considers a strategic product.
Can these problems be somewhat solved? Sure, but until we get much better 'computing as a utility' (if we ever do), a really usable solution is a single-vendor solution, which just brings us back to the whole centralization issue again. Maybe life is a bit better if we're all hosting our federated chat systems and IMAP servers and Git repo websites in the AWS cloud using canned one-click images, but it's not quite the great dream of a truly decentralized and democratic Internet.
(Plus, it still involves somewhat more hassle than using Github and Twitter and Google Mail, and I think that hassle really does matter. Convinced people are willing to fight a certain amount of friction, but to work, the dream of a decentralized Internet needs to reach even the people who don't really care.)
All of this leads me to the conclusion that any decentralized Internet future that imagines lots of people running their own infrastructure is dead on arrival. It's much more likely that any decentralized future will involve a fair amount of concentration, with many people choosing to use someone else's infrastructure and a few groups running most of it. This matters because running such a big instance for a bunch of people generally requires real money and thus some way of providing it. If there is no real funding model, the whole system is vulnerable to a number of issues.
(See, for example, Mastodon, which is fairly centralized in practice with quite a number of large instances, per the instance statistics.)