Noindex tags inserted into profiles after 4.4 upgrade

[sadams101]

I am seeing noindex tags like below inserted into some profile pages after upgrading:

<meta name="robots" content="noindex, follow">

Here is an example:

https://www.celiac.com/profile/95500-gege-harris/

Does anyone know why? I don't want noindex in this content.

[AlexWebsites]

This was in one of the feature announcements. Its added to profiles that have no content.

[GlenP]

See

 

Invision Community 4.4 omits containers (such as forums, blogs, etc.) that have no content (such as a new forum without any topics yet) from the sitemap, and also adds a 'noindex, follow' meta tag into the HTML source.

Google will periodically check to see if the status of the page has changed and happily slurp away when content has been added.

[sadams101]

How is this an improvement? Where is the setting to turn that off?

[Dll]

It's useful because google and other search engines have crawl budgets - they will only crawl and index so many pages on your site at any one time. So, do you want those pages to be useful ones, or do you want some of that budget to be spent crawling and indexing empty profiles with zero value? Profiles with content in them don't have the tag set, so those are still indexed.

[bfarber]

On 3/15/2019 at 8:20 PM, sadams101 said:

How is this an improvement? Where is the setting to turn that off?

We had a high profile person in the SEO industry make the recommendation for very obvious reasons, the same ones DII just provided: search engines can only crawl so many pages on your site, so if you allow them to waste time on "thin" pages, they can't pick up the actual content you want indexed. Why would you care to have an empty user profile page indexed, especially if it means that a hot topic users are actively posting in may not get indexed because the crawler is busy with those empty profile pages?

To answer your second question, there is no setting to turn it off.

[AlexWebsites]

For profile pages that do have content, I would install this plugin to help with SEO.

 

[sadams101]

So the thing is, all of my 86,000+ member profiles have been indexed already, and now I'm see warning from Google about the noindex on them. I think for things like this it is best to have an on/off when rolling them out, as some users may want them indexed.

Thank you very much for sharing the plugin, and what an absolute shock that the profiles on my site have no meta tag info by default...this is crazy, especially for a site like mine running Pages and having over 4,500 articles, and EAT in search being so important. If there was one improvement to make with this plugin it would be to use the "About Me" as the meta description for the author, if there is content in the About Me.

[bfarber]

I can appreciate your position, but it's also important to remember that the vast vast majority of clients have no idea what "meta tags" even are to be able to make such decisions. Then when SEO-related settings get adjusted and cause more harm then good, the product itself ends up being viewed as "not SEO friendly".

Google seeing the noindex tag is not a bad thing. It means Google will strip those "thin" pages (which it can actually penalize you for) and focus on your actual content. The goal isn't to have the most pages indexed - it's to have the best quality pages indexed, and indexed high. If you put up 80,000 extra pages that Google has to sift through, and they determine "well these are all the same, and there's nothing on them", that's not going to do your site any favors overall.

The missing meta tags (for profiles that DO have content) I agree with and have raised internally for consideration.

[sadams101]

I would recommend other SEO tricks rather than the noindex. For example, you could have made the cannonical link on profile pages with no content be the main site's URL, so that if there were any benefit from having those pages in Google's index, it would be transferred to the site's home page, rather than lost completely, as happens with the noindex.

[Dll]

That's really not what the canonical link is designed for. It's use is to tell search engines what the main version of a page is, not to point at an entirely different page. Blank profiles have no value in any case, it's not as if anyone will have linked to them or there's any content for google to use within them. The correct thing to do is what has been done and ask google not to index them.

The warning you are seeing from Google because of it is not an issue, and the number of pages you have in google doesn't have a positive effect on your position for searches relevant to your site. Eg, having 1000 blank profiles won't help you rank for the search terms you're trying to, in fact those blank profiles  being crawled and indexed could stop google finding,  indexing and ranking the relevant, content rich pages you want it to. 

[bfarber]

19 hours ago, sadams101 said:

I would recommend other SEO tricks rather than the noindex. For example, you could have made the cannonical link on profile pages with no content be the main site's URL, so that if there were any benefit from having those pages in Google's index, it would be transferred to the site's home page, rather than lost completely, as happens with the noindex.

Indeed, as Dll points out....that's not what a canonical tag is for, and that's not "optimizing" your site for search engines. That's an attempt to trick Google into doing something which, I would suggest, Google is smart enough not to fall for.