Please help me to stop bots

[Kurt De Pauw]

Hello,

The last weeks we have some troubles with bots who are continuous hammering the website.

+1000 connections and the site isn't responsive anymore.

How can I block these ?

Is there something that could block IP addresses if they connect more then once ?

We are using PLESK

 

I'm checking the user list, I'm checking from who the IP address is and then block it i, .httacces

I also have a huge list (modrewrite) in robots.txt .... but they just ignore it.

 

please help

[Makoto]

Can you provide us with a few example IP addresses that are causing this traffic surge on your site?

This way we can identify if this is a legitimate Web spider or a rogue bot/scraper.

If you'd like me to take a look at it for you personally, feel free to send me a PM and I'll see what I can do to help.

[Kurt De Pauw]

Here you go

@Makoto

Deny from 185.138.241
Deny from 46.229.168
Deny from 5.62.62
Deny from 5.62.61
Deny from 5.62.60
Deny from 5.62.63
Deny from 5.254.82
Deny from 185.212.169
Deny from 217.170.204
Deny from 185.244.214
Deny from 5.62.39
Deny from 185.145.66
Deny from 85.203.22
Deny from 5.62.41
Deny from 185.128.27
Deny from 185.104.185
Deny from 185.244.212
Deny from 185.244.213
Deny from 185.101.32
Deny from 31.204.154.124
Deny from 5.35
Deny from 91.219.239
Deny from 91.219.236
Deny from 91.219.237
Deny from 91.219.238
Deny from 185.236.201
Deny from 185.128.25
Deny from 46.227.66
Deny from 185.253.99
Deny from 185.145.38
Deny from 94.242.231
Deny from 82.102.16
Deny from 185.246.208
Deny from 31.13.191
Deny from 194.99.104
Deny from 185.216.33
Deny from 185.213.20
Deny from 195.181.172
Deny from 195.181.173
Deny from 5.9.17
Deny from 212.7.217
Deny from 185.246.210
Deny from 91.214.44
Deny from 31.204.155
Deny from 91.214.44

 

[bfarber]

Usually you would use a firewall of some sort (like WAF) if you are getting flooded with too many connections. This will require cooperation and assistance from your web host in most cases. There are other services that sort of sit in front of your site like a CDN which can perform similar functions as well (e.g. block an IP if there are a set number of connections from it in a short period of time).

[ASTRAPI]

If you are using Nginx it is very easy to block the user agent and using a firewall like csf the ip but ofcourse it is much better if you are using something Cloudflare and block them there...

[Kurt De Pauw]

would Googl's CDN have this feature ?

[Rhett]

7 hours ago, Kurt De Pauw said:

would Googl's CDN have this feature ?

a CDN won't control or clear up this issue, some do have filtering available to some degree, the solution to your problem is a 5 minute fix, just talk to your host about a proper firewall setup to auto mitigate these issues for you.