Wandering Thoughts archives

Turning over a rock on some weird HTTP requests to our web server

I recently made the mistake of looking at our Apache access.log, and in fact watching it live with 'tail -f'. Me being me, I can't just let what I saw sit quietly, so now I'm here to tell you about the big weirdness I saw. Put simply, it was a whole rapid burst of requests that looked like:

IP - - [28/Feb/2016:17:18:38 -0500] "GET /mmievslc.txt HTTP/1.1" 404 [...]
IP - - [28/Feb/2016:17:18:39 -0500] "GET /mmievslc.txt HTTP/1.1" 404 [...]
IP - - [28/Feb/2016:17:18:39 -0500] "GET /mmievslc.txt HTTP/1.1" 404 [...]

When I started digging, I saw multiple IPs making requests like this for multiple different 8-character .txt URLs in the root of our web server (none of which have ever existed). On random spot checks, they almost all happen in bursts (although there can be pauses), and there are a lot of them.

How many? Yesterday, we saw 34,500 such requests (about 10% of the total HTTP requests), from 116 different IPs and for 122 different names. The top three IPs all made over 1000 requests each; the median made 233 requests. Every such request had the same user-agent:

"Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"

(Only 50 other requests from 13 different IPs used this user-agent.)

On a random spot check of IP addresses doing this, I can't find any that aren't in China. Some but not many of the IP addresses are listed in things like the SBL; others claim to be entirely clean in my blocklist checks.

I spot-checked the IPs doing this yesterday against the IPs doing this today and about two thirds of them are different; checking yesterday against the day before yielded the same result. So there seems to be a different set of sources doing this over time.

We have multiple virtual hosts on this web server, and only two of them are affected; the main departmental web server name and another one (which saw far less volume of these requests). There's nothing obvious that's different between unaffected hosts and affected ones.

And what makes this really mysterious is I have no idea what these requests are supposed to accomplish. Are they an attack of some sort? Are they an accidental side effect of other software? Are they being done deliberately in order to create some sort of useful side effect? Are they traffic cloaking or obfuscation of some sort? Who knows. I may have turned over this rock, but I have no idea how to understand what's scuttling around underneath it.

Mozilla, Symantec, SHA-1 certificates, and the balance of power

In theory, all CAs are supposed to have stopped issuing SHA-1 certificates on January 1st. In Payment Processors Still Using Weak Crypto (via), Mozilla has now announced that they will allow Symantec to issue a limited number of SHA-1 certificates. The reactions I've seen are reasonably harsh. While I don't entirely disagree, I have an additional cynical perspective that's based on the balance of power between CAs and browsers.

Let us be blunt here: Symantec wants to issue these certificates. They are undoubtedly getting to charge a large amount of money for them (especially under the circumstances) and we have plenty of evidence that many CAs do not care one whit about phasing out SHA-1. Browser demands to the contrary are an irritating distraction from the important business of taking money for random numbers.

Browser people are caught in a difficult three-way situation. On the one hand, they hold significant power given the only purpose of most TLS certificates is to display a lock in the browser. On the other hand, browsers are generally commodities themselves. If a browser stops working for you, which includes 'letting you browse HTTPS sites that you want to browse', most people are going to switch to one that does work. The result is that browsers are engaged in a giant game of CA chicken with each other, much like XHTML chicken and DRM chicken. If all browsers remove a popular CA for violating policies, all is fine. But if one or more browsers blink and do not do so, the remaining strict browsers lose; some decent amount of their users will find important sites not working and move one of the browsers that still include the CA. So if you are a CA, you actually hold a fair amount of power over browser vendors provided that you can deal with them in isolation. Finally, on the gripping hand I think that many browser people genuinely want to do what's right, which includes not screwing people in various ways, especially over risks that are (unfortunately) theoretical at the moment.

If Mozilla were to take a hard line here but no other browser were to do so, it feels likely that Symantec would issue those SHA-1 certificates anyways. If Mozilla were to make Firefox stop trusting Symantec certificates, a lot of people would switch away from Firefox (and it doesn't have a huge user base any more). If Mozilla didn't, its threats to do so in the future to misbehaving CAs would be much less credible. So it comes down to whether other browsers will pull Symantec's root certificates over this. Will they? I suspect not, although we'll find out soon enough.

(For the record: I don't think that the Mozilla people involved made the decision they did because of fear of this happening. I'm sure they're sincere in their desire to do the right thing, and I'm sure the harm to various people of Symantec not issuing these certificates weighed on their minds. But I can't see this situation and not think of the balance of power behind the scenes and would probably happen if Mozilla's decision had gone differently.)

Two models of dealing with cookies in Firefox with addons

Recently on Twitter, Dan McDonald was looking for a Firefox cookie handling addon. I had some opinions on this but Twitter being Twitter and me being me I wasn't entirely articulate about them at the time. So here is my attempt to do it better.

There are at least two fundamental models for dealing with cookies (in Firefox and probably elsewhere). The first model is to not allow cookies into your browser session at all. You default-deny all cookies (even first party ones) and whitelist only selected sites; some of them you may accept permanent cookies from, others you may force nominally permanent cookies to become session cookies or short duration ones instead. This is the model I use in my set of extensions, enforced partly via a filtering proxy and partly via a succession of extensions over the years (first CookieSafe, currently CS Lite Mod, and apparently I'm going to need to switch to CookieShield at some point). I believe this is the dominant model of other addons, such as Cookie Monster.

The problem with this strict and narrow approach to cookie management is that there are a steadily increasing number of websites that absolutely require you to accept their cookies in order to use them (one prominent offender is Google's Blogspot). I personally don't mind this for various and sundry reasons, but I suspect that a lot of people do eventually get tired of living this way. So the second model of dealing with cookies is to accept most of them into your browser in a casual and relaxed way, but then delete them again the moment that you don't need them. All of those places that demand the ability to place cookies into your browser before they'll let you see anything get pacified (and you get to see their content), but you get rid of those extremely temporary cookies the moment you're done looking at the site. This is the model of Self-Destructing Cookies.

So the first thing you need to do when looking at Firefox cookie handling and cookie addons is to decide which model sounds more attractive to you. Do you want the low friction model where sites get to temporarily drop cookies while you look at their content, but then those cookies get unceremoniously ejected? Or do you want the strict model where you don't take the risk of any 'bad' cookies managing to get into your browser? To be clear about this, I don't think there's any general right answer; people will have different preferences and tolerances.

(And there are likely other models and intermediate steps between these two. These are just the ones I have direct exposure to.)

(Although I'm currently in the 'strict' camp and have been for a long time, I may someday switch, perhaps just to see what it's like. I admit that the idea of having my browser accept all those cookies makes me nervous, even if they're theoretically only very temporary. But that's an irrational feeling.)

You aren't entitled to good errors from someone else's web app

This particular small rant starts with some tweets:

@liamosaur: Developers who respond to bad URLs with 302 redirects to a 200 page with error info instead of a proper 404 page should be shot into the sun

@_wirepair: as someone who does research for web app scanners, a million times this.

@thatcks: It sounds like web apps are exercising good security against your scanners & denying them information.

If you are scanning someone else's web application, you have absolutely no grounds to complain when it does things that you don't like. Sure, it would be convenient for you if the web app gave you all the clear, semantically transparent HTTP errors you could wish for that make your life easy, but whatever error messages it emits are almost by definition not for you. The developers of those web apps owe you exactly nothing; if anything, they owe you less than nothing. You get whatever answers they feel like giving you, because you are not their audience. If they go so far as to give you deliberately misleading and malicious HTTP replies, well, that's what you get for poking where you weren't invited.

(Google and Bing and so on may or may not be part of their audience, and if so they may give Google good errors and you not. Or they may confine their good errors to the URLs that Google is supposed to crawl.)

Good HTTP error responses (at least to the level of 404's instead of 302s to 200 pages) may serve the goals of the web app developers and their audience. Or they may not. For a user-facing web app that is not intended to be crawled by automation, 302s to selected 200 pages may be more user friendly (or simply easier) than straight up 404s. As a distant outside observer, you don't know and you have no grounds for claiming otherwise.

(There are all sorts of pragmatic and entirely rational reasons that developers might do things that you disagree with.)

It's probably the case that web app developers are better served over the long term by doing relatively proper HTTP error handling, with real 404s and so on (although I might not worry too much about the exact error codes). However this is merely a default recommendation that's intended to make the life of developers easier. It is not any sort of requirement and developers who deviate from it are not necessarily doing it wrong. They may well be making the correct decision for their environment (including ones to deliberately make your life harder).

(See also Who or what your website is for and more on HTTP errors, which comes at the general issue from another angle.)

PS: If you are scanning your own organization's web apps, with authorization, it may be worth a conversation with the developers about making the life of security people a little easier. But that's a different issue entirely; then 'our security people' are within the scope of who the web app is for.