SoCal Delicious

JANUARY 15, 2010

Those bots retrieve a copyof your web page so that they can include relevant terms from that page in their search index. Not all bots are benign however.Some bots go through your website looking for web forms and email addresses to send you spam. Who is this article for? You need to be able to check your sites raw web logs.

IP 38

IP PHP Web RSS 38

SoCal Delicious

JANUARY 15, 2010

Sebastian’s Pamphlets If you’ve read my articles somewhere on the Internet, expect something different here. Most of these bots don’t obey robots.txt , that means you can’t even block them applying Web standards ( learn how to block rogue bots ). It can get worse when a URI gets retweeted often.

Social Media 41

Social Media Develop Development Media 41

SoCal Delicious

JANUARY 15, 2010

Sebastian’s Pamphlets If you’ve read my articles somewhere on the Internet, expect something different here. requestHandler.php [L] Please note that the code above kinda disables the Web server’s error handling. Here are a few ideas how to block rogue (crappy, not behaving, …) Web robots.

SEO 37

SEO IP PHP Blogging 37

SoCal Delicious

SEPTEMBER 22, 2008

Heres 5 steps to start exploring: View the "Best Of" list with 50+ essays on viral marketing, gaming, and ads » Get introduced: About this blog, why entrepreneurs and marketers recommend it » Receive updates by email or RSS feed or Twitter. Every web product ultimately has a bunch of user flows contained within it.

Startup 29

Startup IP Email Report 29