SoCal Delicious

JANUARY 15, 2010

How to Identify the Bot You Want to Block Before you can block a bot, you will need to know at least one of two things: the IP address where the bot is coming from or the"User Agent string" that the bot is using. Once youve located the entries that belong to the bot, look for the IP address and the user agent string. Be careful though.

IP 38

IP PHP Web RSS 38

SoCal Delicious

SEPTEMBER 22, 2008

Heres 5 steps to start exploring: View the "Best Of" list with 50+ essays on viral marketing, gaming, and ads » Get introduced: About this blog, why entrepreneurs and marketers recommend it » Receive updates by email or RSS feed or Twitter. Where it might look something like: 1000, profile.photo.update, 1, 9:30AM 3/14/2008.

Startup 29

Startup IP Email Report 29

SoCal Delicious

JANUARY 15, 2010

While handling bots, you should detect search engine crawlers, too: /* lookup your crawler IP database to populate $isCrawler; then, if the IP wasnt identified as search engine crawler: */ if ($isCrawler !== from the IP address range 65.52.0.0 Other rogue bots identify themselves by IP addy, user agent, and/or referrer.

SEO 37

SEO IP PHP Blogging 37

SoCal Delicious

JANUARY 15, 2010

ThingFetcher sometimes requests a (shortened) URI 30 times per second, from different IPs. A method to verify the bot All IP addresses used by a bot should resolve to server names having a unique pattern. For example Googlebot comes only from servers named "crawl" + "-" + replace($IP, ".", "-") + ".googlebot.com"

Social Media 41

Social Media Develop Development Media 41