How to have your web spider irritate me intensely (part 2)
January 29, 2007
In the spirit of previous cleverness, here's a simple new technique:
This wasn't Referer spamming, since the websites in the Referer headers were completely random URLs, apparently drawn from legitimate sites around the Internet (often repeated). Nor were the websites ones that actually linked to us, or had any relationship to the URLs that were being crawled.
Even in low volume this is a sure-fire ticket to our kernel level IP filters, since it insures that we're mostly unable to get anything useful from our Referer logs without a lot of additional work and is therefor deeply irritating.
Today's offender is the IP address 184.108.40.206, which is an unnamed
iol.it IP address; it is using a User-Agent value of 'Mozilla/5.0
(arianna.libero.it,email@example.com)'. It does seem to have
* * *
Atom feeds are available; see the bottom of most pages.