tomwin Posted August 31, 2017 Share Posted August 31, 2017 Hi, In my Google webmaster tools i get many error not sure if Google maybe later will punish me for this but the problem is. I have a lot of download documents but guest cant download them only members. So in my Google webmaster tools i get many error not sure if Google maybe later will punish me for this but the problem is. Google can't search the download documents or the download button. Google search like a guest. Is there a way to stop google from crawling this? Maybe add something in my robots.txt? Hope someone can help me with this one. Link to comment Share on other sites More sharing options...
tomwin Posted September 1, 2017 Author Share Posted September 1, 2017 Very strange that no one want help here. There must be more people out there that have download files and only let register member to download. If you have then Google can't search them and you will have a lot of Denied access (403) in your Google webmaster tools. I'm not sure if Google will punish you or me later on. But if you have like me several of thousand document for download there will be a lot of error report in Google webmaster tools. Invision staff told me to ask here, i did, but no response. But the only thing i wanted to know what to add in my robots.txt any one can help here. Sorry for bad English. Link to comment Share on other sites More sharing options...
Foolboy Posted September 1, 2017 Share Posted September 1, 2017 I guess you have two options. (Sorry but I do not have Download application so my advice might be wrong) 1) If you submitted yout sitemap to Google Webmaster Tool (or now Google Search Console), you can remove it from appearing in sitemap in Admin Control Panel > Serarch Engine Optimization > Sitemap. Remove the "recommended settings" option and you will see the list of content types on your community that you can select to remove it from the sitemap. 2) You have to set permission on Download application or download categories to be unseen by guest. Since IPS applications is pretty dynamic, I think robots.txt solution will not work very well and it will work only the root URL of your application e.g. https://www.example.com/downloads, etc. Link to comment Share on other sites More sharing options...
tomwin Posted September 1, 2017 Author Share Posted September 1, 2017 Thanks for your reply. I will try this. Link to comment Share on other sites More sharing options...
sofos Posted January 2, 2018 Share Posted January 2, 2018 @tomwin Did You resolved that? Link to comment Share on other sites More sharing options...
Recommended Posts
Archived
This topic is now archived and is closed to further replies.