Saurabh Jain Posted May 4, 2015 Share Posted May 4, 2015 http://mechanical-engg.com/: Googlebot can't access your siteOver the last 24 hours, Googlebot encountered 4264 errors while attempting to access your robots.txt. To ensure that we didn't crawl any pages listed in that file, we postponed our crawl. Your site's overall robots.txt error rate is 10.2%. How should I configure the same... Link to comment Share on other sites More sharing options...
不中用 Posted May 4, 2015 Share Posted May 4, 2015 http://mechanical-engg.com/: Googlebot can't access your siteOver the last 24 hours, Googlebot encountered 4264 errors while attempting to access your robots.txt. To ensure that we didn't crawl any pages listed in that file, we postponed our crawl. Your site's overall robots.txt error rate is 10.2%. How should I configure the same.... just add a robots.txt file into your root folder .. empty file .. or compare http://community.invisionpower.com/robots.txtbut I think it is an old one .. it should avoid any Googlebot errors .. . Link to comment Share on other sites More sharing options...
Saurabh Jain Posted May 4, 2015 Author Share Posted May 4, 2015 . just add a robots.txt file into your root folder .. empty file .. or compare http://community.invisionpower.com/robots.txtbut I think it is an old one .. it should avoid any Googlebot errors .. .I have a /forum http://mechanical-engg.com/forum/so should I add /Forum Link to comment Share on other sites More sharing options...
不中用 Posted May 4, 2015 Share Posted May 4, 2015 I have a /forum http://mechanical-engg.com/forum/so should I add /Forum .I think this will guide you better ..http://www.robotstxt.org/robotstxt.html . Link to comment Share on other sites More sharing options...
opentype Posted May 4, 2015 Share Posted May 4, 2015 Looks like you are using the 3.x robots.txt on a 4.0 site.As IN10TION said: Just empty that file but keep it online. You might add individual lines later if you really want to have certain areas not indexed. Link to comment Share on other sites More sharing options...
EmpireKicking Posted August 24, 2015 Share Posted August 24, 2015 I have done it the right way. But the only problem I have that it keeps on saying that the robot.txt doesn't exist Link to comment Share on other sites More sharing options...
Chris Bell Posted August 24, 2015 Share Posted August 24, 2015 you must have placed it in the wrong place then Link to comment Share on other sites More sharing options...
Recommended Posts
Archived
This topic is now archived and is closed to further replies.