The mystery of
POSTs with a zero
One of the joys of running web software that is rather paranoid is
getting to see all sorts of weird things that float around the web,
generally run by spammers and other people who are up to no good.
Today's oddity could be called 'the case of the zero-length
and is just what it sounds like:
POST requests that have a
Content-Length of 0 bytes.
(Or at least they have a
Content-Length of '0' after Apache gets
through passing the request to DWiki. It's possible that Apache is
silently sanitizing some bizarre C-L header value to 0.)
I don't know the full headers for these
POST requests but because
of the code flow inside DWiki, I know that they claim to be form
submissions (I check
Content-Type before looking at
It's possible that the requests have some other header that is supposed
Content-Length. It's also possible that this software is
POST form requests to see what happens or because
this evades security precautions in some applications.
Based purely on the claimed
User-Agent values I can say that this
software is up to no good, since some of the time it claims to be
Googlebot (or at least some of the requests claim to be from Googlebot,
since I suppose I shouldn't assume that there's just one piece of
software that's doing this). All of the requests I've pulled out of
the logs seem to be HTTP/1.1 requests and generally are for regular
URLs. The software involved also seems to almost always lower-case the
URLs it's using, which very much doesn't work very well here.
User-Agent suggests there may be two different programs
involved, one of which claims to be Googlebot and one of which
doesn't sent a
User-Agent at all. Only the Googlebot-faker seems
to lowercase its URLs; the other program mostly
Wandering Thought's main page but occasionally
other, correctly-cased URLs. The second program seems to be the
more active one.)
I don't have any answers to this particular mystery and in fact now that I've looked into it it's more mysterious than before. Sometimes that's how it goes on the web these days.
Sidebar: volume and source details
These requests aren't happening in high volume but are generally happening several times a day from various different IPs. In the last ten days there have been at least 150 instances from 50 different IPs; the most prolific five IPs made 22, 13, 11, 8, and 6 requests each respectively. I haven't tried to run down the origin of all of the IPs, but China shows up a lot in the top-N list. One IP is currently in the SBL, in SBL181621 (a /24 listing due to blackhat SEO spammer hosting).