Invision Community 4: SEO, prepare for v5 and dormant account notifications By Matt Monday at 02:04 PM
VaBeach_Guy Posted June 2, 2020 Posted June 2, 2020 On one of my boards, I have a particular bot that's in my online list with nearly 200 instances. I've banned the IP block (114.119.*.*) It shows as "petalbot.(with the specific IP address).aspiegel".com. Is there any better way of blocking that specific bot?
VaBeach_Guy Posted June 2, 2020 Author Posted June 2, 2020 32 minutes ago, Rhett said: https://aspiegel.com/petalbot Do you think it's worth letting it crawl?
Rhett Posted June 2, 2020 Posted June 2, 2020 5 minutes ago, VaBeach_Guy said: Do you think it's worth letting it crawl? No, block it via robots.txt as it shows on that link. Or ban the ip range on the server if you have access. VaBeach_Guy 1
VaBeach_Guy Posted June 2, 2020 Author Posted June 2, 2020 11 minutes ago, Rhett said: No, block it via robots.txt as it shows on that link. Or ban the ip range on the server if you have access. I hadn't thought about doing it server side, thanks.
Recommended Posts