robot.txt

[Saurabh Jain]

http://mechanical-engg.com/: Googlebot can't access your site

Over the last 24 hours, Googlebot encountered 4264 errors while attempting to access your robots.txt. To ensure that we didn't crawl any pages listed in that file, we postponed our crawl. Your site's overall robots.txt error rate is 10.2%.

 

 

How should I configure the same...

[不中用]

http://mechanical-engg.com/: Googlebot can't access your site

Over the last 24 hours, Googlebot encountered 4264 errors while attempting to access your robots.txt. To ensure that we didn't crawl any pages listed in that file, we postponed our crawl. Your site's overall robots.txt error rate is 10.2%.

 

 

How should I configure the same...

​.

 

just add a robots.txt file into your root folder .. empty file ..

 

or compare http://community.invisionpower.com/robots.txt

but I think it is an old one .. it should avoid any Googlebot errors ..

 

.

[Saurabh Jain]

​.

 

just add a robots.txt file into your root folder .. empty file ..

 

or compare http://community.invisionpower.com/robots.txt

but I think it is an old one .. it should avoid any Googlebot errors ..

 

.

​

I have a /forum 

http://mechanical-engg.com/forum/

so should I add /Forum

 

 

[不中用]

​

I have a /forum 

http://mechanical-engg.com/forum/

so should I add /Forum

 

 

​.

I think this will guide you better ..

http://www.robotstxt.org/robotstxt.html

 

.

[opentype]

Looks like you are using the 3.x robots.txt on a 4.0 site.

As IN10TION said: Just empty that file but keep it online. 
You might add individual lines later if you really want to have certain areas not indexed. 

[EmpireKicking]

I have done it the right way. But the only problem I have that it keeps on saying that the robot.txt doesn't exist

[Chris Bell]

you must have placed it in the wrong place then