SoCal Delicious

JANUARY 15, 2010

Those bots retrieve a copyof your web page so that they can include relevant terms from that page in their search index. Not all bots are benign however.Some bots go through your website looking for web forms and email addresses to send you spam. You need to be able to check your sites raw web logs. htaccess overrides" enabled.If

IP 38

IP PHP Web RSS 38

SoCal Delicious

JANUARY 15, 2010

For example when you operate a WordPress blog, you could code nearly everthing below in your 404 page (consider alternatives ). requestHandler.php [L] Please note that the code above kinda disables the Web server’s error handling. Here are a few ideas how to block rogue (crappy, not behaving, …) Web robots.

SEO 37

SEO IP PHP Blogging 37

SoCal Delicious

JANUARY 15, 2010

Those crappy bots not only cost you money because they keep your server busy and increase your bandwidth bill, they actively decrease your advertising revenue because your visitors hit the back button when your page isn’t responsive due to the heavy bot traffic. Or leave the Web, respectively pay the publishers.

Social Media 41

Social Media Develop Development Media 41