Alex.amazingboy Posted January 28, 2014 Share Posted January 28, 2014 Getting heaps errors from Google Crawler, how to prevent them ? For example : Link to comment Share on other sites More sharing options...
jair101 Posted January 28, 2014 Share Posted January 28, 2014 It seems you have disabled guests to be able to see user profiles, so when google tries to crawl them it gives an error. I added this to robots.txt: Disallow: /user/ Link to comment Share on other sites More sharing options...
Alex.amazingboy Posted January 28, 2014 Author Share Posted January 28, 2014 It seems you have disabled guests to be able to see user profiles, so when google tries to crawl them it gives an error. I added this to robots.txt: Disallow: /user/ Thanks mate and what about this : More than 500 errors caused by this thing :D Link to comment Share on other sites More sharing options...
jair101 Posted January 28, 2014 Share Posted January 28, 2014 Same, google is trying to use the search, which you have blocked. I have this line in my robots.txt, should help: Disallow: /*&module=search Link to comment Share on other sites More sharing options...
Alex.amazingboy Posted January 28, 2014 Author Share Posted January 28, 2014 thank you :) Link to comment Share on other sites More sharing options...
Recommended Posts
Archived
This topic is now archived and is closed to further replies.