How brain-amputated developers created the social media plague
SoCal Delicious
JANUARY 15, 2010
ThingFetcher sometimes requests a (shortened) URI 30 times per second, from different IPs. A method to verify the bot All IP addresses used by a bot should resolve to server names having a unique pattern. For example Googlebot comes only from servers named "crawl" + "-" + replace($IP, ".", "-") + ".googlebot.com"
41 
Let's personalize your content