Almost no one wants to run their own infrastructure

May 23, 2018

Every so often, people get really enthused about the idea of a less concentrated, more distributed Internet, one where most of our email isn't inside only a few places, our online chatting happens over federated systems instead of Twitter, there are flower gardens of personal websites and web servers, there are lots of different Git servers instead of mostly Github, and so on. There are many obstacles in the way of this, including that the current large providers don't want to let people go, but over time I have come to think that a large underappreciated one is simply that people don't want to run their own infrastructure. Not even if it's free to do so.

I'm a professional system administrator. I know how to run my own mail and IMAP server, and I know that I probably should and will have to some day. Do I actually run my own server? Of course not. It's a hassle. I have things on Github, and in theory I could publish them outside Github too, on a machine where I'm already running a web server. Have I done so? No, it's not worth the effort when the payoff I'd get is basically only feeling good.

Now, I've phrased this as if running your own infrastructure is easy and the only thing keeping people from doing so is the minor effort and expense involved. We shouldn't underestimate the effects of even minor extra effort and expense, but the reality is that doing a good job of your own infrastructure is emphatically not a minor effort. There is security, TLS certificates, (offsite) backups, choosing the software, managing configuration, long term maintenance and updates, and I'm assuming that someone else has already built the system and you just have to set up an instance of it.

(And merely setting up an instance of something is often fraught with annoyance and problems, especially for a non-specialist.)

If you use someone else's infrastructure and they're decently good at it, they're worrying about all of that and more things on top (like monitoring, dealing with load surges and DDOSes, and fixing things in the dead of night). Plus, they're on the right side of the issues universities have with running their own email; many such centralized places are paying entire teams of hard-working good people to improve their services (or at least the ones that they consider strategic). I like open source, but it's fairly rare that it can compete head to head with something that a significant company considers a strategic product.

Can these problems be somewhat solved? Sure, but until we get much better 'computing as a utility' (if we ever do), a really usable solution is a single-vendor solution, which just brings us back to the whole centralization issue again. Maybe life is a bit better if we're all hosting our federated chat systems and IMAP servers and Git repo websites in the AWS cloud using canned one-click images, but it's not quite the great dream of a truly decentralized and democratic Internet.

(Plus, it still involves somewhat more hassle than using Github and Twitter and Google Mail, and I think that hassle really does matter. Convinced people are willing to fight a certain amount of friction, but to work, the dream of a decentralized Internet needs to reach even the people who don't really care.)

All of this leads me to the conclusion that any decentralized Internet future that imagines lots of people running their own infrastructure is dead on arrival. It's much more likely that any decentralized future will involve a fair amount of concentration, with many people choosing to use someone else's infrastructure and a few groups running most of it. This matters because running such a big instance for a bunch of people generally requires real money and thus some way of providing it. If there is no real funding model, the whole system is vulnerable to a number of issues.

(See, for example, Mastodon, which is fairly centralized in practice with quite a number of large instances, per the instance statistics.)

Comments on this page:

By Davies, Adam at 2018-05-25 06:52:35:

A large number of people want to only consume but not build stuff. Installation of open source tools with large complexity like gitlab or even email causes serious issues. Especially if it is not shiny as the SAAS ones. In our uni, SPAM is such a big issue - the mailadmin cant improve the situation. I gladly wish we moved to Gmail but cannot due to data protection (and huge human egos). Many scientists spend time cleaning up spam/phishing emails instead of useful work! Many scientists - postdocs - have limited contracts - so we prefer github or gitlab as we can use their own repos wherever they work in the next years. Compare that to copying your home folder or cgit or so on by hand. This is important especially with email - all your contacts for life that can help in job and fun. All my Uni Cambridge (PhD) emails were inaccessible a day after I finished. (Thankfully I managed to IMAP a copy to Gmail). The Uni computing services offer no help anyone. Imagine a non-tech user, (like the ones who do not read your blog). These days I therefore suggest everyone to redirect at least one copy to gmail/icloud/hotmail. It is sad but in unstable world of job market, the only stable things are big providers.

Written on 23 May 2018.
« Bad versions of packages in the context of minimal version selection
Registering for things on the Internet is dangerous these days »

Page tools: View Source, View Normal, Add Comment.
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Wed May 23 00:52:55 2018
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.