cheapbooks.com Posted September 28, 2007 Share Posted September 28, 2007 I see people in my referal logs finding my site by typing "registration" at google and then going to the forums to register for the purpose of posting spam.there is little reason for most webmasters to have registration pages indexed on google. I'd like to request that this page be hidden from search engine spidersI dont think this can be done with robots.txt, which doesn't support parameters to cgi applications. anyway it should be done at the application level to ensure compliance Link to comment Share on other sites More sharing options...
bfarber Posted September 28, 2007 Share Posted September 28, 2007 You can do this via robots.txt, yes. Link to comment Share on other sites More sharing options...
cheapbooks.com Posted September 29, 2007 Share Posted September 29, 2007 You can do this via robots.txt, yes.please shpw me an example. I don't think it is possible. they both use index.php. Link to comment Share on other sites More sharing options...
Michael Posted September 29, 2007 Share Posted September 29, 2007 User-agent: * Disallow: /index.php?act=reg Link to comment Share on other sites More sharing options...
cheapbooks.com Posted October 1, 2007 Share Posted October 1, 2007 User-agent: * Disallow: /index.php?act=regI have read discussions on other sites regarding this and no one says that this will work. they specifically say that this does not work. therefor, it cannot be done this way. it needs to be done at the application level. Link to comment Share on other sites More sharing options...
Michael Posted October 1, 2007 Share Posted October 1, 2007 Can you give some sources on this? I've always used this and I've never seen my registration page show up in Google (not that I've spent a great deal of time searching, but searching for specifically my site never shows me the registration page). Link to comment Share on other sites More sharing options...
bfarber Posted October 1, 2007 Share Posted October 1, 2007 http://www.robotstxt.org/wc/norobots.html#formatAccording to the specifications, it IS valid.Try it in Google webmaster tools. Link to comment Share on other sites More sharing options...
Recommended Posts
Archived
This topic is now archived and is closed to further replies.