Jump to content

Use of robots.txt

Simon Harvey

Recommended Posts

  • 1 month later...

Hello @opentype

what would you do with links like these appearing on top of results page,    (as the only links of your site, on that search)



Currently  if i search for my domain name + something else in greek, 

parents.org.gr ρωτήστε την ειδικό   


δοκιμή parents.org.gr 

those links appear  exactly on top of search results, instead of a normal, (with content) link that should appear, or just the domain name. 

what is your recommendation ?    I have seen a 5% increase in bounce traffic since the migration. 

I am discussing it right now with support, they recommend adding a specific  robot.txt to hide those.    


Doing a search now, to see what others are doing, ...is it just me who has this problem?.....??


Link to comment
Share on other sites

You seem to allow a lot of activity for guests. That’s why these links are even crawled. In a typical setup, they don’t appear for guests. Allowing emailing for guests is pretty problematic. Every spammer could use your page as a spam distribution center. 

But anyway. As long as you want to keep your current settings and permissions, then yes, you would want to set up robots.txt rules to filter out those URLs. 

Link to comment
Share on other sites

Thank you very much for your reply. 

no i didnt get that those links were the result of that.     

following and emailing.... ofcourse i dont want that for guests.      these things are very new for us  because of the huuuuge  difference  between   vbulletin 3,8  and  ips....! 

I will go and change that asap! . 

But i think those things should be something like a default setting when you migrate? ... no ? 

Link to comment
Share on other sites


This topic is now archived and is closed to further replies.

  • Recently Browsing   0 members

    • No registered users viewing this page.
  • Create New...