Jump to content

detect Yahoo and MSN crawlers?


Guest boeing747fp

Recommended Posts

Posted

When is IP.Board going to recognize Yahoo Slurp and MSNBot crawlers instead of just Googlebot, Archive.org, and AskJeeves??? Yahoo's crawler does wonders to one's Online List (i've just spotted 33 "guests" that were actually Yahoo Slurp crawlers [yahoo hasn't learned to keep all of it's crawler servers under 1 IP Address like Google has] ) :(

Posted

haha never noticed that. is this a new feature? SWEET!
last time i asked this in the Support dept of IPS (a year or so ago) they said it was impossible to add crawlers to the crawler detecting list... :P

Posted

haha never noticed that. is this a new feature? SWEET!


last time i asked this in the Support dept of IPS (a year or so ago) they said it was impossible to add crawlers to the crawler detecting list... :P


It has been there for quite some time now, definately in the 2.1.x releases, not sure about before.

Maybe they should add a few more main ones by default. :P

Is Google the only reliable search engine then IPS? :whistle:
Posted

Out of 33 guests almost all are the same 2 IPs for mine and they all trace to RIPE network in the Netherlands. I wonder if that is a real spoider or this moron I know from the Netherlands who does DOS attacks on forums.

Archived

This topic is now archived and is closed to further replies.

  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...