One of the joys of running web software that is rather paranoid is getting to see all sorts of weird things that float around the web, generally run by spammers and other people who are up to no good. Today's oddity could be called 'the case of the zero-length POST' and is just what it sounds like: POST requests that have a Content-Length of 0 bytes.

(Or at least they have a Content-Length of '0' after Apache gets through passing the request to DWiki. It's possible that Apache is silently sanitizing some bizarre C-L header value to 0.)

I don't know the full headers for these POST requests but because of the code flow inside DWiki, I know that they claim to be form submissions (I check Content-Type before looking at Content-Length). It's possible that the requests have some other header that is supposed to preempt Content-Length. It's also possible that this software is submitting empty POST form requests to see what happens or because this evades security precautions in some applications.

Based purely on the claimed User-Agent values I can say that this software is up to no good, since some of the time it claims to be Googlebot (or at least some of the requests claim to be from Googlebot, since I suppose I shouldn't assume that there's just one piece of software that's doing this). All of the requests I've pulled out of the logs seem to be HTTP/1.1 requests and generally are for regular URLs. The software involved also seems to almost always lower-case the URLs it's using, which very much doesn't work very well here.

(Looking at User-Agent suggests there may be two different programs involved, one of which claims to be Googlebot and one of which doesn't sent a User-Agent at all. Only the Googlebot-faker seems to lowercase its URLs; the other program mostly POSTs to Wandering Thought's main page but occasionally POSTs to other, correctly-cased URLs. The second program seems to be the more active one.)

I don't have any answers to this particular mystery and in fact now that I've looked into it it's more mysterious than before. Sometimes that's how it goes on the web these days.

Sidebar: volume and source details

These requests aren't happening in high volume but are generally happening several times a day from various different IPs. In the last ten days there have been at least 150 instances from 50 different IPs; the most prolific five IPs made 22, 13, 11, 8, and 6 requests each respectively. I haven't tried to run down the origin of all of the IPs, but China shows up a lot in the top-N list. One IP is currently in the SBL, in SBL181621 (a /24 listing due to blackhat SEO spammer hosting).