Chris's Wiki :: blog/web/GooglebotStillCrawlingFeeds Commentshttps://utcc.utoronto.ca/~cks/space/blog/web/GooglebotStillCrawlingFeeds?atomcommentsDWiki2015-07-04T16:58:08ZRecent comments in Chris's Wiki :: blog/web/GooglebotStillCrawlingFeeds.By Twirrim on /blog/web/GooglebotStillCrawlingFeedstag:CSpace:blog/web/GooglebotStillCrawlingFeeds:760d377460be8e606a9775c291f1e242f93eb87bTwirrim<div class="wikitext"><p>Taking a look through my own logs, looks like the web browser "Let's pretend we're something we're not" infection has spread:</p>
<p>"Feedly/1.0 (+<a href="http://www.feedly.com/fetcher.html">http://www.feedly.com/fetcher.html</a>; like FeedFetcher-Google)"</p>
<p>I did somewhat hope we were past that. I'm not sure how many sites even pay that much attention to the user agent string any more.</p>
<p>It does look like I'm seeing "Tiny Tiny RSS" from one IP address bounce off my rss feed a remarkable amount, roughly every 10-15 minutes, but note I'm behind Cloudflare, and they might actually be hiding requests from me.</p>
</div>2015-07-04T16:58:08ZBy Twirrim on /blog/web/GooglebotStillCrawlingFeedstag:CSpace:blog/web/GooglebotStillCrawlingFeeds:199b28928ca046c84574fe79515058af4ee90a7cTwirrim<div class="wikitext"><p>Depending on how much you care about Google you could automatically take the origin IP address and drop it into an ipset. Then hook that into a REJECT or DROP rule. The less you care about google, the higher up the subnet you go starting with /24 and working up to /8 :)</p>
</div>2015-07-04T16:20:27Z