Jump to content

robots.txt

Featured Replies

Posted

Can you include robotstxt.txt file for IPS 4.0 to archive with RC version?

  • 2 weeks later...
  • Author

:cry:

  • 1 month later...

+1

Can you add an example of robots.txt to archives?

  • 1 month later...
  • Author

Bump

  • Management

We do not include a robots.txt in IPS4 as it is not needed. Guest links can be removed entirely for inaccessible sections, so there's nothing for search engines to exclude as they won't see it anyway. :)

  • 1 month later...

We do not include a robots.txt in IPS4 as it is not needed. Guest links can be removed entirely for inaccessible sections, so there's nothing for search engines to exclude as they won't see it anyway. :)

Why is it relevant for Google to index for example the ?do=reportComment pages??? 

Why is it relevant for Google to index for example the ?do=reportComment pages??? 

​It isn’t, and that’s why Google won’t usually even see that link, because it gets hidden through the guest permissions – that’s what Lindy explained. 

Hmm, in guest mode I receive indeed a 404 error (good).

Google's index is not updated yet then. The pages are cached around March this year. 

Edited by pyro888

  • 11 months later...

can you disallow google from indexing the members profiles (/profile/) from within admin setting guest restrictions?

Thanks,

On 03/05/2015 at 8:24 AM, Lindy said:

We do not include a robots.txt in IPS4 as it is not needed. Guest links can be removed entirely for inaccessible sections, so there's nothing for search engines to exclude as they won't see it anyway. :)

Yes there is - anything with a CSRF key in the URL will cause issues.

on the "Use recommended settings?" i'm on "manual" settings but when we take a look at the xml generated by the sitemap.php we can see clearly that the option as little   as no effect and yes we have open support incident on this  and other bug on the  sitemap, url issue etc etc... ^_^

 

 

3 hours ago, recifbox said:

on the "Use recommended settings?" i'm on "manual" settings but when we take a look at the xml generated by the sitemap.php we can see clearly that the option as little   as no effect and yes we have open support incident on this  and other bug on the  sitemap, url issue etc etc... ^_^

 

 

I think you should have created a new topic on your sitemap issue. This topic is about robots.txt started by Ilya Hoilik.

  • 1 year later...

hello,

@Colonel_mortis  is there a workaround on this?  what issues do you mean ?  can you please elaborate? 

 

@Lindy  hello, is there any documentatiion specifically that will explain this  to a newbie  " Guest links can be removed entirely for inaccessible sections" 

 

Archived

This topic is now archived and is closed to further replies.

Recently Browsing 0

  • No registered users viewing this page.