Josiah Wallingford Posted June 13, 2018 Share Posted June 13, 2018 (edited) When I look at my 404 error logs I see 2545 404 error messages on robots.txt How do I resolve this? Why is there no robots.txt file included with invision community? If search engines keep getting 404 error messages anywhere on my site, they are going to consider the site garbage and lower its ranking, correct? Is there a best method for adding a robots.txt file? Edited June 13, 2018 by Josiah Wallingford Link to comment Share on other sites More sharing options...
opentype Posted June 13, 2018 Share Posted June 13, 2018 It doesn’t affect ranking and there is no need to have such a file. You create it yourself as soon as there are things you want to put in there. Link to comment Share on other sites More sharing options...
Josiah Wallingford Posted June 13, 2018 Author Share Posted June 13, 2018 Could I just create a robots.txt file that is blank so that at least they are not getting 404 errors? What effects would that have on them indexing the site? Link to comment Share on other sites More sharing options...
opentype Posted June 13, 2018 Share Posted June 13, 2018 You can do that to avoid the 404s. That’s the only result. I will not influence indexing/ranking. Josiah Wallingford 1 Link to comment Share on other sites More sharing options...
Joel R Posted June 13, 2018 Share Posted June 13, 2018 There are a couple of files in the MP to manage 404 files. Link to comment Share on other sites More sharing options...
bfarber Posted June 14, 2018 Share Posted June 14, 2018 We don't include a default robots.txt because doing so would overwrite any custom robots.txt you may set up. Hexsplosions 1 Link to comment Share on other sites More sharing options...
HCICT Posted June 14, 2018 Share Posted June 14, 2018 There is a robots.txt on the marketplace and it works pretty nice! ? Link to comment Share on other sites More sharing options...
Hexsplosions Posted June 14, 2018 Share Posted June 14, 2018 I got tired of Webmaster Tools complaining about index issues due to my robots.txt, so these days it is blank. Josiah Wallingford 1 Link to comment Share on other sites More sharing options...
Recommended Posts