(DP42) Bot Group

[balazsp]

Hi @DawPi

It's been over a month, did you happen to make any progress? 

Thank you. 

 

[DawPi]

No, not yet. Mod works as intended and your request is beyond of scope of standard support for my mods (bugs etc).

[Randy Calvert]

@DawPi would you mind sharing what bot user agents you are detecting?  

I specifically wanted to check on AdsBot-Google-Mobile (both Android and Web), AdsBot-Google, and Mediapartners-Google.  I'm also curious about other Google ones noted at  https://developers.google.com/search/docs/advanced/crawling/overview-google-crawlers

I was wanting to move Google's Adsense and their ad quality checking bots from default Guest to Members group since there is some content that is not available to guests that Google needs to get access to.  This sounds like it could work if it detects these bots.  

 

[DawPi]

4 hours ago, Randy Calvert said:

would you mind sharing what bot user agents you are detecting?  

The same as IPS4:

		'about'			=> "Libby[_/ ]([0-9.]{1,10})",
		'adsense'		=> array( "Mediapartners-Google/([0-9.]{1,10})", "Mediapartners-Google" ),
		'ahrefs'		=> "AhrefsBot",
		'alexa'			=> "^ia_archive",
		'altavista'		=> "Scooter[ /\-]*[a-z]*([0-9.]{1,10})",
		'ask'			=> "Ask[ \-]?Jeeves",
		'baidu'			=> array( "^baiduspider\-", "baiduspider[ /]([0-9.]{1,10})" ),
		'bing'			=> array( "bingbot[ /]([0-9.]{1,10})", "msnbot(?:-media)?[ /]([0-9.]{1,10})" ),
		'brandwatch'	=> "magpie-crawler",
		'excite'		=> "Architext[ \-]?Spider",
		'google'		=> array( "Googl(?:e|ebot)(?:-Image|-Video|-News)?/([0-9.]{1,10})", "Googl(?:e|ebot)(?:-Image|-Video|-News)?/?" ),
		'googlemobile'	=> array( "Googl(?:e|ebot)(?:-Mobile)?/([0-9.]{1,10})", "Googl(?:e|ebot)(?:-Mobile)?/" ),
		'facebook'		=> "facebookexternalhit/([0-9.]{1,10})",
		'infoseek'		=> array( "SideWinder[ /]?([0-9a-z.]{1,10})", "Infoseek" ),
		'inktomi'		=> "slurp@inktomi\.com",
		'internetseer'	=> "^InternetSeer\.com",
		'look'			=> "www\.look\.com",
		'looksmart'		=> "looksmart-sv-fw",
		'lycos'			=> "Lycos_Spider_",
		'majestic'		=> "MJ12bot\/v([0-9.]{1,10})",
		'msproxy'		=> "MSProxy[ /]([0-9.]{1,10})",
		'webcrawl'		=> "webcrawl\.net",
		'websense'		=> "(?:Sqworm|websense|Konqueror/3\.(?:0|1)(?:\-rc[1-6])?; i686 Linux; 2002[0-9]{4})",
		'yahoo'			=> "Yahoo(?:.*?)(?:Slurp|FeedSeeker)",
		'yandex'		=> "Yandex(?:[^\/]+?)\/([0-9.]{1,10})",
		'seznam'		=> array( "SeznamBot[ /]([0-9.]{1,10})", "Seznam screenshot-generator ([0-9.]{1,10})" ),
		'dotbot'		=> "DotBot[ /]([0-9.]{1,10})",
		'sogou'			=> "Sogou web spider[ /]([0-9.]{1,10})",
		'isetallabot'	=> "istellabot[ /][a-z]([0-9.]{1,10})",
		'blexbot'		=> "BLEXBot[ /]([0-9.]{1,10})",
		'semrush'		=> "SemrushBot/([0-9.]{1,10})"

 

[DTL2012]

Has anyone using this experienced an uplift in organic search traffic?

[DawPi]

Thank you for being a client!  The Invision Community Marketplace is closing October 30 2023, so I am moving all of my files over to my personal site https://forum.invisionize.pl. 

• You can redeem the marketplace purchase keys provided by IPS in order to keep getting updates and support for the applications/plugins you purchased.
• You can purchase files directly on my marketplace.
• You can get support for your purchased applications/plugins directly on their new support topics.
• You can request a custom application/plugin or service.

 

Bookmark https://forum.invisionize.pl and the new Marketplace Directory www.Invisioneer.org.