High loads

[p4guru]

might want to verify bing related ips http://www.bing.com/toolbox/verify-bingbot to make sure it's really bing

[Gabriel Torres]

p4guru, thanks for the tip.

Maybe the Bing thing was just a coincidence... I am still investigating this...

By using Grumpy's handy shell script, something pops up immediately (I changed the script so it analyses two days worth of logs instead of the latest 100,000 hits):

6026 72.14.199.124

8212 184.172.56.203

9646 173.193.181.197

62975 87.238.41.135

We have this IP 87.238.41.135 with... 62,975 hits?!?!?! It is related to something called integrasco.com. The log entry seems "legit", but no real user would hit us 62.975 times in 48 hours...

87.238.41.135 [22/Jul/2014:18:59:28 -0300] "GET /forum/48-leitores-e-gravadores-de-cd-dvd-e-blu-ray/page-11 HTTP/1.1" 200 24131 "-" "Mozilla/5.0 (X11; U; Linux x86_64; en-US) AppleWebKit/533.4 (KHTML, like Gecko) Chrome/5.0.375.99 Safari/533.4"

I am not sure if I should blacklist this guy or not, as it may be something related to the ads we run on our forum.

The other top two IPs, 173.193.181.197 and 184.172.56.203, are hosted at softlayer and also seems "legit":

173.193.181.197 [23/Jul/2014:09:39:23 -0300] "GET /topic/593864-wwwnetcomputadorescombr-A?-confiA?vel/ HTTP/1.1" 301 36 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.153 Safari/537.36"

184.172.56.203 [23/Jul/2014:09:58:17 -0300] "GET /topic/968531-resolvido-problema-na-fontebateria/ HTTP/1.1" 200 21582 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.153 Safari/537.36"

What is interesting is that these three IPs seem to be from the same crawler, as the user-agent string is very similar. So, it is either a crawler or a DDoS attack.

The forth IP is from Google and then all other top IPs are from either Google or Bing.

If anyone here has the tools to investigate these IPs in more depth I'd highly appreciate it.

Thanks,

Gabriel.

PS: Sent an email to softlayer's abuse dept asking for more info.

[Grumpy]

If their requests are constantly different, scraper bot.

If their requests are constantly same, dos attack.

Just block them with firewall, no chance they're real.

The first one... Okay. no one uses chrome 5. We're at like 36 now. That's like one of the very first chrome versions. It's obviously not a real person.

The next two hosted by softlayer. There's no reason a datacenter is going to visit a website so much except for scrape/attack/proxy.

You can watch and see if anything goes weird/complains after you block them. Quite unlikely though.

[Gabriel Torres]

Just to let you guys know that I just found out that these two IP addresses, 184.172.56.203 and 173.193.181.197, are from a crawler from a partner of ours, HotWords (http://www.hotwords.com.br). These crawlers scrape our forum in order to display advertisements relevant to our contents. So, I had to unblock them.

I still have no clue about 87.238.41.135.