Jump to content

Chinese Robots Crawling my Site


JohnCourt

Recommended Posts

Is there anything I can do about this onslaught of robots crawling my site? Can I block an entire country? I looked up the ip address and this is the information I retrieved.

  baiduspider-180-76-15-15.crawl.baidu.com

 

I thought there was a way to ban IP Address's but I cannot find the feature in ACP to do so. We are running IPB v4.1.8.1

Thanks for any help!

Link to comment
Share on other sites

5 hours ago, Courtland said:

Or perhaps I could ban the domain baidu.com, is it possible?

You can stop domains in .htaccess. Something like this:

# Kill bad sites
RewriteCond %{HTTP_REFERER} baidu\.com [NC]
RewriteRule .* - [F]

Those guys don't honor the Robots.txt protocol 

 

Link to comment
Share on other sites

Archived

This topic is now archived and is closed to further replies.

  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...