The practical problems with simple web apps that work as HTTP servers

September 26, 2014

These days there are a number of languages and environments with relatively simple to learn frameworks for doing web activity (I am deliberately phrasing that broadly). Both node and Go have such things, for example, and often make a big deal of it.

(I know that 'let's do some web stuff' is a popular Go tutorial topic to show easy it is.)

All of this makes it sound like these should be good alternatives to the CGI problem (especially with their collections of modules and packages and so on). Unfortunately this is not the case in default usage and one significant part of why not is exactly that these systems are pretty much set up to be direct HTTP servers out of the box.

Being a direct HTTP server is marvelously simple approach for a web app if and only if you're the only thing running on the web server. If you have a single purpose web server that exists merely for your one web application, it's great that you can expose the web app directly (and in simple internal setups you don't particularly need the services of Apache or nginx or lighttpd or the like). But this single purpose web server is very rarely the case for simple CGI-level things. Far more common is a setup where you have a whole collection of both static pages and various simple web applications aggregated together under one web server.

(In general I feel that anything big enough for its own server is too big to be sensible as a 'simple CGI'. Good simple CGI problems are small almost by definition.)

If you try hard you can still make this a single server in Go or node but you're going to wind up with kind of a mess where you have several different projects glued together, all sharing the same environment with each other (and then there are the static files to serve). If the projects are spread across several people, things will get even more fun. Everything in one bucket is simply not a good engineering answer here. So you need to separate things out, and any way you do that makes more and more work.

If you separate things out as separate web servers, you need multiple IPs (even if you embody things on the same host) and multiple names to go with them, which are going to be visible to your users. If you separate things out with a frontend web server and reverse proxying, all of your simple web apps have to be written to deal with this (and with the various issues involved in using HTTP as a transport). Both complicate your life, eroding some of the theoretical simplicity you're supposed to get.

(However Go does have a FastCGI package (as well as a CGI package, but then you're back to CGI), apparently with an API that's a drop in replacement for the native Go HTTP server. It appears that node has at least a FastCGI module that's said to be a relatively drop in replacement for its http module. FastCGI does leave you with the general problems of needing daemons, though.)

PS: I'm handwaving the potentially significant difference in programming models between CGI's 'no persistent state between requests' and the shared context web app model of 'all the persistent state you want (or forget to scrub)'. I will note that the former is much simpler and more forgiving than the latter, even in garbage collected environments such as Go and Node.

Sidebar: the general issues with daemons

Although it is not specific to systems that want to be direct HTTP servers, the other problem with any sort of separate process model for simple web apps is exactly that it involves separate processes for each app. Separate processes mean that you've added more daemons to be configured, started, monitored and eventually restarted. Also, those daemons will be sitting there consuming resources on your host even if their app is completely unused at the moment.

You can make this easy if you try hard. But today it involves crafting a significant amount of automation because pretty much no out of the box Unix system is designed for this sort of operation. Building this automation is a not insignificant setup cost for your 'simple' web apps (well, for your first few).

(If you try really hard and have the right programming model you can get apps to be started on demand and stopped when the demand goes away, but this actively requires extra work and complexity in your systems and so on.)

Written on 26 September 2014.
« Why CGI-BIN scripts are an attractive thing for many people
DWiki, Python 3, Python, and me »

Page tools: View Source, Add Comment.
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Fri Sep 26 03:09:43 2014
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.