April 14, 2017 in Technical Problems
In a few posts on this community I have ready about renaming robotstxt.txt to use as a robots.txt template file however I am not able to find robotstxt.txt on my installation.
Is this not used anymore? Is it advised to create a robots.txt file?
There is no need for a robots.txt. You can create one, if there is public content you don’t want indexed.
what would you do with links like these appearing on top of results page, (as the only links of your site, on that search)
Currently if i search for my domain name + something else in greek,
parents.org.gr ρωτήστε την ειδικό
those links appear exactly on top of search results, instead of a normal, (with content) link that should appear, or just the domain name.
what is your recommendation ? I have seen a 5% increase in bounce traffic since the migration.
I am discussing it right now with support, they recommend adding a specific robot.txt to hide those.
Doing a search now, to see what others are doing, ...is it just me who has this problem?.....??
You seem to allow a lot of activity for guests. That’s why these links are even crawled. In a typical setup, they don’t appear for guests. Allowing emailing for guests is pretty problematic. Every spammer could use your page as a spam distribution center.
But anyway. As long as you want to keep your current settings and permissions, then yes, you would want to set up robots.txt rules to filter out those URLs.
Thank you very much for your reply.
no i didnt get that those links were the result of that.
following and emailing.... ofcourse i dont want that for guests. these things are very new for us because of the huuuuge difference between vbulletin 3,8 and ips....!
I will go and change that asap! .
But i think those things should be something like a default setting when you migrate? ... no ?
This topic is now archived and is closed to further replies.
Started 4 hours ago
Started 54 minutes ago
Started 6 hours ago