Wandering Thoughts archives

An example of the progress of the modern web

I've recently wound up being in the (Toronto) real estate market, which has given me an interesting opportunity to see both how much the modern web has changed the world (okay, the way things are done), and how much further it still could go.

(Usually I don't have much exposure to ordinary commercial websites unless I'm looking for something; it's only then that I get to experience the full joys and annoyances of dealing with whatever companies have decided is theoretically useful today.)

On the changed the world front: I've been doing much of my searching through realtor.ca, which is run by the Canadian realtors association and has a powerful set of tools for both filtering things for sale and seeing where they are. I really don't want to think about what this process would have been like even, say, five years ago, much less before the web had caught on at all. Realtor.ca is very clearly a creation of the modern web, the whole environment where it's possible to do complex, highly interactive 'website as application' sites with fluidly interactive elements, information popups, and so on. Even a five years ago version of this would have been much simpler, much less interactive, and much less useful, partly because I think it wouldn't have had mapping; my perception is that the whole interactive mapping boom on websites is ultimately due to Google Maps, which had only just launched five years ago.

(I suspect that this is both positive and negative for realtors. On the one hand, this must result in much less time wasted in their office with prospective buyers and a big map, sorting through piles of listings; on the other hand, knowing listings and locating things presumably used to be part of their specialized expertise and value.)

At the same time, there's a lot of ways that the whole experience could still be improved, some of them illustrated by Zillow's interface (sadly Zillow is not available for Canada). For example, imagine what you could do with analytics on real estate data; given a bunch of selling prices for a bunch of properties with various features, you should be able to work out things like what on average a certain amount of money gets you in various neighborhoods, or how much more (or less) a given set of features will cost you in different areas of the city (essentially giving you neighborhood price differentials). Once you have the information, there's lots of interesting mapping visualizations you could provide on top.

(You could also go on to suggest an answer to everyone's hot question of whether or not a particular listing is probably overpriced (or even underpriced), or what a particular house is probably worth. Also, Toronto at least used to be infamous for properties being deliberately underpriced in order to induce bidding wars between prospective buyers. I don't know if this is still going on here, but you could easily spot this by comparing listing prices with sales prices, and give people clear indications of what areas this was happening in, or alternately what areas people were overpricing things in.)

Normally I would say that there are clearly some opportunities for startups here, but unfortunately all of this dreaming points to a problem with doing more sophisticated and engaging web interfaces for this sort of stuff in today's world: access to data. In order to build a website that offers a better real estate experience, you really need the realtor.ca data and that data is not freely available; it's owned and controlled by the realtors and their trade associations. I can't imagine that they're is interested in either another competing website or in adding features to their own website that seem likely to make realtors even less necessary than before.

(From that perspective, I'm honestly surprised that realtor.ca offers as much as it does, and I'm amazed at Zillow's apparent ability to get the equivalent data in the US.)

PS: it appears that actual sales information is public information in Canada, although it's not online as far as I can see. However, listing information, probably including all of the property features that realtor.ca tracks, is owned by realtors et al.

My current Apache SSL cipher settings

Like a number of other security protocols, TLS (aka SSL) comes in several generations and offers a large selection of potential ciphers. Many of these options are actually a bad idea from a security perspective; the ciphers are weak or use things that have been broken, and older versions of SSL have known weaknesses. By default, Apache's mod_ssl enables them all and even sometimes defaults to (very) bad choices, all in the vague name of accommodating some export version of a browser from 1997 that might perhaps show up some day and be unable to do any version of SSL since then.

(More realistically, the mod_ssl defaults were probably set ten years or more ago and have not really been changed since then for all sorts of reasons. Well, actually these are more like the OpenSSL defaults, since mod_ssl defers most of these decisions to OpenSSL.)

I've recently decided to start changing that in our Apache SSL configurations, partly prompted by this and this (PDF). My current settings are to not support SSL v2 at all, disallow 'export' ciphers, which were deliberately made really weak so that they could be easily decrypted, and disallow everything that OpenSSL considers a low-strength cipher (which actually includes export ciphers).

Specifically, I'm setting:

SSLProtocol all -SSLv2
SSLCipherSuite ALL:!ADH:!EXP:!LOW:!SSLv2:RC4+RSA:+HIGH:+MEDIUM

This is more or less researched from the Apache documentation. It's tempting to explicitly disallow MD5, although I don't know if this is just superstition.

Now, two cautions apply here. First, our SSL sites are by and large for internal use, so I can afford to be somewhat casual about how many ciphers I support; if this turns out to be a too small set, I will hear about it. Second, our web servers are nowhere near CPU constrained so I don't care very much about which good cipher is the most CPU-efficient; a more active site might well.

(Considering that SSLv3 dates from 1996, disabling SSLv2 should be really quite safe by now.)

It would be nice (I say wistfully) if software like mod_ssl would come preconfigured with a sensible and limited set of ciphers and perhaps notes on really ancient software that's known to require you to use less than ideal cipher choices.

(Partly I am writing all of this down now so that I have a reference for it later.)

Modern versions of Apache and Redirect

Our main web server is currently running Ubuntu 8.04. For reasons beyond the scope of this entry, we want to upgrade it to Ubuntu 10.04, so today I did a test install, which wound up exploding with a bunch of log messages that looked like:

[...] .htaccess: Redirect: invalid first argument (of three)

Internet searches will give you inconclusive and somewhat confused answers; it doesn't help that Apache doesn't report a line number, and that Redirect can validly take three arguments.

What you have probably done is written a Redirect that looks like this:

Redirect /a/path http://some.where/ [L]

This is a superstition. Redirect doesn't take a flags argument at the end; only RewriteRule does. It used to be harmless to put flags on the end anyways, so people did it for various reasons (mistaken belief that it was required, laziness, forgetfulness, etc). Modern versions of Apache are more picky about Redirect directives and so choke on this with the error message you see here.

(This error is especially fun in a .htaccess, because if there are any problems with a .htaccess Apache locks you out of the directory it's in. As it happened, some of our crucial .htaccess files had this problem.)

Sidebar: the Apache versions involved

To save you looking this up: Ubuntu 10.04 has Apache 2.2.16 and Ubuntu 8.04 has Apache 2.2.8. The pickiness was apparently introduced in this change in July of 2009, and it appears that this then appeared in 2.2.12 (per the CHANGES_2.2 file, which lists this PR as one of the 2.2.12 changes). Before the change, Apache would just ignore the third argument of a three-argument Redirect if it couldn't interpret the first argument as a redirection status. After the change, it considers it an error.

Looking at the change, it's hard to criticize the Apache people for it; the previous behavior was a great way to have mistakes compound themselves.

Redirecting from HTTP to HTTPS is a bad security idea

Every so often, people setting up SSL websites have the bright idea that to be friendly, they should have a non-SSL version of the site that redirects people to the SSL website. They are entirely correct that this is the friendliest approach, but you shouldn't do it if you care about security.

(There are plenty of reasons to use SSL even if you don't care about security. Pervasive SSL frustrates all of the people who want to stick their fingers in traffic between actual users and your website, starting with the user's ISP (with various sorts of stupid proxy tricks) and moving on to less allegedly friendly sorts.)

If you care about security, it is deadly for anything that should be SSL-encrypted to be accidentally transmitted in the clear. One of the ways that you avoid both accidents and users developing bad habits is to make them not work; that way, users are conditioned not to do that and if your website ever makes a URL-generation mistake you will notice right away because it doesn't work at all.

To put it another way: if the encryption matters, non-SSL URLs can't be interchangeable with SSL URLs because one isn't encrypted and the other is. If they are interchangeable, you are taking the thing that you said was important and making it unimportant.

The worst sort of HTTP to HTTPS redirection is when you do URL-based redirection, so a page on the HTTP site is redirected to the same page on the HTTPS site; this enables the most user bad habits and hides the most mistakes. It is especially bad if this works for things like your login form, where you are transmitting the most sensitive data. Next worst is redirection from any URL on the HTTP site to the root of your HTTPS site. If you have to have an HTTP site at all, my opinion is that it should just be a plain page without any links on it that tells people to use your HTTPS site.

(If you have a HTTP site at all, consider setting up log monitoring for what URLs turn up and having alarms go off if any sensitive URLs show up.)

Sidebar: why you don't want any links on your HTTP site

It's very friendly for attackers if you train users that they can go to your HTTP site (by just typing the name into their URL bar) and then click on a link on the page to go to your secure website. We already know that users are terrible at actually looking at where they are going, so if an attacker can interpose their own version of your unprotected web page they can send your users to any secure web server they want to and most users will be none the wiser.

Sidebar: my view on ISP meddling in web traffic

Yes, I will admit that some ISPs sometimes have excellent reasons to want to transparently impose things like caching proxies between their users and outside websites. But it's a little bit like the email world; your intentions may be good, but unfortunately a lot of other people have already poisoned that particular well. There is too much of a history of various sorts of bad and evil ISP meddling for me to take that chance any more, and so I am willing to block the attempts of the rare good caching ISP in order to frustrate all the other ISPs out there.

This is only going to become more acute if various governments get their way about requiring ISPs to keep more and more records of people's Internet traffic. If that happens, I will also hope very hard that TLS SNI actually becomes usable.