JLS Posted December 27, 2015 Posted December 27, 2015 I need some direction with a robots.txt. I need all the standard recommended stuff blocked and then I'd also like to block user profiles unless there is good reason not too. I understand they can be viewed by the public but seems a waste... Thank you in advance for help with this
tnn Posted December 27, 2015 Posted December 27, 2015 This was graciously provided by @laltroweb.it (https://laltroweb.it/robots.txt). Further reading in this thread -> https://community.invisionpower.com/topic/422077-my-robotstxt-fpr-ipb4/ Something to get you started
JLS Posted December 27, 2015 Author Posted December 27, 2015 Cool that helped thanks. Now, I kinda think I need to allow bots in the profiles but no-index them to get those few thousand pages removed from google. Then once removed do a no-follow. Does that sound right to you or any others reading this? Thanks
Recommended Posts
Archived
This topic is now archived and is closed to further replies.