RASEL MIAH Posted December 3 Posted December 3 Hello, how can I solve this problem? My robots.txt file is blocking the indexing of +6000 pages of my forum. I need someone to help me solve this. This is my robots.txt: https://dobet.forum/robots.txt What should I modify? Thank you.
Marc Posted December 3 Posted December 3 Are you using our standard robots.txt. If so, are there specific things you want to unblock? Just because something is blocked, it doesn't necessarily mean that is a bad thing, so its worth being careful what you edit Just taken a look since you are on cloud. You appear to be using our optimised one. If you want to use your own instead, you can certainly do so if you are fully aware what you are doing System>Site Promotion>Search engine optimisation>Crawl Management Before deciding to do that, its very very likely what is being blocked, is for good reason. I would advise on having a read through this first
Recommended Posts