Bots: How to block bots that don’t Respect your robots.txt file
In this article, we will show you how to block bots that don’t respect your robots.txt file.
For this article, you will need the PHP script mentioned in our other article here.
Once you have the block.php file in place, edit or create a robots.txt file in the public_html directory.
Enter the following…
Disallow: block.php Disallow: /block.php
Now we need to set the trap. Enter the following code at the top of your website (just after the body tag)…
<a title="" href="http://yourdomain.com/block.php"><img alt="" src="pixel.gif" width="1" height="1" /></a>
Make sure you replace yourdomain.com with your actual domain.