SoCal Delicious
JANUARY 15, 2010
How to Identify the Bot You Want to Block Before you can block a bot, you will need to know at least one of two things: the IP address where the bot is coming from or the"User Agent string" that the bot is using. Once youve located the entries that belong to the bot, look for the IP address and the user agent string. Be careful though.
SoCal Delicious
SEPTEMBER 22, 2008
Where it might look something like: 1000, profile.photo.update, 1, 9:30AM 3/14/2008. Typically I would start out with a series of pretty plain HTML pages using tables that just print out SQL queries. September 18th, 2008 at 8:00 am. Virtual Goods Summit 2008 + last week’s Twitter links » Add New Comment.
SoCal Delicious
JANUARY 15, 2010
ThingFetcher sometimes requests a (shortened) URI 30 times per second, from different IPs. That means, the Web server has to deal with the same load as with a GET request , it just deletes the content portion (the compelety formatted HTML page) when responding, after counting its size to send the Content-Length response header.
Expert insights. Personalized for you.
38 
Let's personalize your content