Sergey_SV Posted December 7, 2018 Posted December 7, 2018 1 hour ago, Unlucky said: Coming to this topic late. Has anyone got the solution to get this working? We would really like to add adverts on the messenger pages. Thanks hmm.. this is a good topic, I just found out that Google have no problem display the Ads in the messenger and profile pages on my site. Only in forums locked for guests it is not... So it is somehow able to crawl there...
Sonya* Posted December 22, 2018 Posted December 22, 2018 (edited) I've done a "dirty" hack to allow a crawler to login into my community. Attention: This changes core login functionality for one user and will be overwritten when you update: Create a user with name "Crawler" and give him access to the areas the crawler should have access to. Open file system/Login/Login.php and edit a method authenticate(): public function authenticate( \IPS\Login\Handler $onlyCheck = NULL ) { try { if ( isset( \IPS\Request::i()->_processLogin ) ) { // Allow access for the AdSense bot if (\IPS\Request::i()->auth != 'Crawler') { \IPS\Session::i()->csrfCheck(); } ..... } Go to your Adsense account and add credentials: Be aware of the additional field _processLogin to be added to login credentials. Again, it works, but this is a very dirty solution that works till next update. I hope that IPS can add a setting so that we can exclude bot user from checking csrf in the future, so that we do not need so dirty hacks. Edited December 22, 2018 by Sonya* Owdy and lordi 2
Owdy Posted December 22, 2018 Posted December 22, 2018 (edited) I get parse errors with that. Can you show exatly how to edit Login.php? @Sonya* edit: got it. Edited December 22, 2018 by Owdy Sonya* 1
Mopar1973Man Posted December 23, 2018 Posted December 23, 2018 (edited) Could some write this up a bit clearer it creating a mess on my end. I need to know the edit on the login.php file. What to look for and what to insert where. Edited December 23, 2018 by Mopar1973Man
Nathan Explosion Posted December 27, 2018 Posted December 27, 2018 I've put together a plugin which handles this in a different location, making it hookable - have sent a new copy of it to @Mopar1973Man to try it out. If anyone would like a copy then please PM me directly. I won't be releasing it to the MarketPlace, as it does circumvent the security which IPS has built in to the product (albeit for only one configured account) - all I ask is you chuck me a beer 😉 ptprog, Morgin, Unlucky and 8 others 11
Owdy Posted January 11, 2019 Posted January 11, 2019 In Adsense settings, you give restrictrect url, like discover/unread. How about other urls? I have member areas, where are tons of topics etc. How crawler know it needs to be logged in in those areas?
Nathan Explosion Posted January 11, 2019 Posted January 11, 2019 In the same way - give the url that they can access (you can add more, not just one) with the credentials they are allowed to use (and make sure that the account has the access to view those areas)
Owdy Posted January 11, 2019 Posted January 11, 2019 1 minute ago, Nathan Explosion said: In the same way - give the url that they can access (you can add more, not just one) with the credentials they are allowed to use (and make sure that the account has the access to view those areas) Impossible, there are about 1000-2000 topics.
Nathan Explosion Posted January 11, 2019 Posted January 11, 2019 14 minutes ago, Owdy said: Impossible, there are about 1000-2000 topics. And those topics are in a container called a forum......unless things have changed since I used Adsense years ago, it will access anything in the same 'directory'. So, in theory..... Add: https://invisioncommunity.com/forums/ And it should be able to then crawl..... https://invisioncommunity.com/forums/topic/368323-adsense-crawler If it doesn't work in that way now, then you'll have to ask Google. Owdy 1
Owdy Posted January 18, 2019 Posted January 18, 2019 Google logs in, but i never see ads in /discover/unread/
Unlucky Posted January 18, 2019 Posted January 18, 2019 4 hours ago, Owdy said: Google logs in, but i never see ads in /discover/unread/ Hi, is working well for us so double check all steps again in case something has been missed. Or maybe it's the type of add you are trying to show? We are only using Text & display ads in the member logged in sections
Sonya* Posted January 18, 2019 Posted January 18, 2019 6 hours ago, Owdy said: Google logs in, but i never see ads in /discover/unread/ What YOU see on the page is not what BOT sees on the page as it is dependant on user. It's possible that you do not see the ads but your users do.
Mopar1973Man Posted January 21, 2019 Posted January 21, 2019 Thank you @Nathan Explosion . He jumped in and fixed my typo. I'm now on the working list. I'll continue to monitor the status over the next few days.
Owdy Posted February 1, 2019 Posted February 1, 2019 (edited) Now Adsense says they got Unknown error in page /discover/unread/ Edited February 1, 2019 by Owdy
Sonya* Posted February 1, 2019 Posted February 1, 2019 3 hours ago, Owdy said: Now Adsense says they got Unknown error in page /discover/unread/ Same here. I assume that the hack I have made does only work for login. But when the crawler tries to get to other pages, it still fails due to the csrf check. Probably the hack should go further, so that not only login but the session itself after login does not make csrf check. I am investigating further. Owdy 1
greek_parea Posted March 6, 2019 Posted March 6, 2019 On 2/1/2019 at 1:03 PM, Sonya* said: Same here. I assume that the hack I have made does only work for login. But when the crawler tries to get to other pages, it still fails due to the csrf check. Probably the hack should go further, so that not only login but the session itself after login does not make csrf check. I am investigating further. can you help me for this?
Sonya* Posted March 6, 2019 Posted March 6, 2019 52 minutes ago, greek_parea said: can you help me for this? I have given up. I have tried different ways to bypass csrf check for one user. But any attempt failed. I can successfully login as Crawler in browser however the crawler itself fails. The account is blocked after three failed attempts and there are no logs in IPS of what is wrong with the login. Without logs in IPS and any logs from AdSense it is like pocking in the dark. Sorry!
VladTheGreat Posted June 24, 2019 Posted June 24, 2019 I know this is an old thread but I wish I seen it sooner, the answer is in your permission settings, the problem I find is that modules and menus in IPS have separate permission tabs, you might allow a crawler to crawl the "Classified" section, but at the same time restrict the menu link to be seen to guests or visa versa etc, you must check that you allow the robot to see everything you've linked from public menues also to avoid landing page link following errors, if menu isn't visible to a crawler it can't error on a path it can't see to attempt travel on etc. Also, avoid static ads, always use auto sizing ads for ips4 :)
LemonGrenade Posted June 26, 2019 Posted June 26, 2019 How are you making it work @VladTheGreat as guests just see /discover (which displays all ads) but members see discover/unread/ (which hardly displays any)
VladTheGreat Posted July 3, 2019 Posted July 3, 2019 It's hard to explain in a sentence but if crawler accesses though guest, the bot follows links only avail to guest so if you use a crawler app to crawl your website as guest and see where it's able to go, youll see where your permissions could be at fault? Hope it helps
Stormlilly Posted July 26, 2019 Posted July 26, 2019 So just to confirm, as of today there is not a simple or effective way to enable Adsense ads to display to logged in members? Is this correct IPS staff?
bfarber Posted July 26, 2019 Posted July 26, 2019 8 hours ago, Stormlilly said: So just to confirm, as of today there is not a simple or effective way to enable Adsense ads to display to logged in members? Is this correct IPS staff? You can display ads to logged in members, however if Adsense requires that a page be guest-viewable in order for ads to display there, then any pages which guests are unable to view would not be able to display ads. Some sites lock everything down and only logged in users can access basically anything. Some sites are pretty much wide open, guests just can't post. There's a lot of variation in how a site can be configured, so it's hard to give a one size fits all answer.
Recommended Posts