Sometimes I notice an abnormal amount of guests online, so I check the who's online list. When I check the who's online list I notice a lot of guests with the same IP address. Sometimes there can be upwards of 60 - 100 guests from the same ip address or sometimes even 200 - 500 guests.

On the who's online list they appear to be checking profiles, reading topics, searching, or viewing other parts of the board. I've tried banning the ip addresses, but they always come back under a different ip address.

I've also tried blocking bad bots with htaccess and blocking anything keyweb which does no good. When I did a whois on the ip address it appears to be coming from data centers in different locations for each different ip address.

I know it's not a search engine, as Google, Yahoo, and Msn don't hit websites en mass like this. I don't think it's a DDos attack either since my website experiences no downtime or hindered performance when these ip addresses hit my website en mass.

What is it and what can I do to stop it?

Be careful you don't ban yourself :) Out of my 140 guests earlier, 130 were me. I have no idea what's causing it but suspected ajax refresh at one time.

This occurred on IPB2 as well, I need the actual number of guests, bots viewing the forum for statistical purposes and not some bloated number.

I think you'll find that it is proably spiders crawling your site (even though you suggest that it isn't). Have you checked out who the IPs belong to?

http://ws.arin.net/whois/
http://www.db.ripe.net/whois

I think that there is a topic here somewhere (or maybe an article at resources) about constructing a robots.txt file that focuses spiders on topics and posts, stops them trying to reply etc.

Why is there so many from the same ip address though? Even Google, Yahoo, Msn etc doesn't index my site in excess like that, with other search engines it's usually just one ip address.

I think there are so many from one IP address because the spiders are programmed to multithread/multitask. They don't have to wait for the response from one before they fire off another request. I often have Yahoo for example hitting me simultaneously from the same IP address.

I keep getting Google showing up in my online list with 30, 40 & currently 99 Guests showing all using the same IP (213.180.209.10)

This has been all day today & yesterday.

I even placed this IP in my Ban filter yesterday but it still shows up as online???

Yahoo/Inktomi search bots are infamous for pounding on sites with 100-500 open connections to them at a time, its just how they crawl. Google and MSN usually only send one spider to your site to crawl it so its probably not those two, but if I had to guess it was any, Yahoo would be it.


Try to add a line to your robots.txt in your sites root folder that looks like



then it will usually take a day or two for that change to take effect but it should fix your problem, assuming its Yahoo

 User-agent: Slurp 

Crawl-delay: 5 

Thanks for the info, will add that right now, much appreciated

Is it ok to add the crawl delay to all bots/user agents? I added it like this in my robots.txt:

User-agent: *
Crawl-delay: 5

Then I specified the directories below I don't want search engines to index at all.