Jump to content

Featured Replies

Posted

On one of my boards, I have a particular bot that's in my online list with nearly 200 instances. I've banned the IP block (114.119.*.*) It shows as "petalbot.(with the specific IP address).aspiegel".com.

Is there any better way of blocking that specific bot?

 

Do you think it's worth letting it crawl?

No, block it via robots.txt as it shows on that link.   Or ban the ip range on the server if you have access. 

  • Author
 

No, block it via robots.txt as it shows on that link.   Or ban the ip range on the server if you have access. 

I hadn't thought about doing it server side, thanks. 

Recently Browsing 0

  • No registered users viewing this page.