How to Block Unwanted Bots from Your Website with.htaccess
SoCal Delicious
JANUARY 15, 2010
Those bots retrieve a copyof your web page so that they can include relevant terms from that page in their search index. Not all bots are benign however.Some bots go through your website looking for web forms and email addresses to send you spam. You need to be able to check your sites raw web logs.
38 
Let's personalize your content