Matt

Ok, so it's worth rounding up what Invision Community DOES do in terms of SEO.
Auto generated meta tags for description Custom meta tag editor for finer control in a single area Uses appropriate header codes 200 for OK, 301 for redirects, 303 for 'the page is actually here', 404 for not founds, 403s for permission errors, etc Uses appropriate HTML markup to highlight important content (h1, h2, etc) Uses rewritten URLs for a cleaner structure packed with keywords Creates and submits a sitemap to show Google which URLS are important to your community Uses nofollow where appropriate to stop pages like 'Contact Us' from being crawled Uses JSON-LD micro data markup to tell Google about what data they are seeing and how it should be used Allows easy integration with Google Search Console for tracking Uses https Has a responsive theme which gets the "Mobile Friendly" badge Here's what is coming in 4.3
Meta description expanded to 300 characters Ability to rebuild your entire sitemap quickly Lastmod tag added to sitemap files Not to mention other retention tools like
Bulk emailing tool available Emailed notifications Promote to social media Share to social media There seems to be a level of worry in this topic, and while I'm happy to field any questions you have, Google is a bit mysterious and prone to changing things overnight.
We adhere to good standards and do all the right things as you can see from this list.
We are not adverse to change and adding new features, but we never do it in a panic or with a knee-jerk until we get some hard evidence which supports the reason for change.
We have been monitoring our own Google Search Console and clicks/impressions are up, indexes are down slightly, but Google has seen them and flagged them as 'discovered'. These tend to be profiles from people who have never posted (and we have about 200k of those alone).
I do not believe we are facing any crisis, or that anything is substantially wrong.
We can always do better, and we're listening. We just need a little more than a few charts to go on before we make drastic change.

This is correct, but Google will not see that as a unique link for that post. It'll see that link as a unique link for that page, which is why it says "Page 5 -".

We do not provide direct permalinks to posts because there is little value, and you create duplicate content. vBSEO did (for a while) load a single post in a new window with a permalink, but that was back in 2008 and I'm sure it's moved on since then.

Ok, so it's worth rounding up what Invision Community DOES do in terms of SEO.
Auto generated meta tags for description Custom meta tag editor for finer control in a single area Uses appropriate header codes 200 for OK, 301 for redirects, 303 for 'the page is actually here', 404 for not founds, 403s for permission errors, etc Uses appropriate HTML markup to highlight important content (h1, h2, etc) Uses rewritten URLs for a cleaner structure packed with keywords Creates and submits a sitemap to show Google which URLS are important to your community Uses nofollow where appropriate to stop pages like 'Contact Us' from being crawled Uses JSON-LD micro data markup to tell Google about what data they are seeing and how it should be used Allows easy integration with Google Search Console for tracking Uses https Has a responsive theme which gets the "Mobile Friendly" badge Here's what is coming in 4.3
Meta description expanded to 300 characters Ability to rebuild your entire sitemap quickly Lastmod tag added to sitemap files Not to mention other retention tools like
Bulk emailing tool available Emailed notifications Promote to social media Share to social media There seems to be a level of worry in this topic, and while I'm happy to field any questions you have, Google is a bit mysterious and prone to changing things overnight.
We adhere to good standards and do all the right things as you can see from this list.
We are not adverse to change and adding new features, but we never do it in a panic or with a knee-jerk until we get some hard evidence which supports the reason for change.
We have been monitoring our own Google Search Console and clicks/impressions are up, indexes are down slightly, but Google has seen them and flagged them as 'discovered'. These tend to be profiles from people who have never posted (and we have about 200k of those alone).
I do not believe we are facing any crisis, or that anything is substantially wrong.
We can always do better, and we're listening. We just need a little more than a few charts to go on before we make drastic change.

Ok, so it's worth rounding up what Invision Community DOES do in terms of SEO.
Auto generated meta tags for description Custom meta tag editor for finer control in a single area Uses appropriate header codes 200 for OK, 301 for redirects, 303 for 'the page is actually here', 404 for not founds, 403s for permission errors, etc Uses appropriate HTML markup to highlight important content (h1, h2, etc) Uses rewritten URLs for a cleaner structure packed with keywords Creates and submits a sitemap to show Google which URLS are important to your community Uses nofollow where appropriate to stop pages like 'Contact Us' from being crawled Uses JSON-LD micro data markup to tell Google about what data they are seeing and how it should be used Allows easy integration with Google Search Console for tracking Uses https Has a responsive theme which gets the "Mobile Friendly" badge Here's what is coming in 4.3
Meta description expanded to 300 characters Ability to rebuild your entire sitemap quickly Lastmod tag added to sitemap files Not to mention other retention tools like
Bulk emailing tool available Emailed notifications Promote to social media Share to social media There seems to be a level of worry in this topic, and while I'm happy to field any questions you have, Google is a bit mysterious and prone to changing things overnight.
We adhere to good standards and do all the right things as you can see from this list.
We are not adverse to change and adding new features, but we never do it in a panic or with a knee-jerk until we get some hard evidence which supports the reason for change.
We have been monitoring our own Google Search Console and clicks/impressions are up, indexes are down slightly, but Google has seen them and flagged them as 'discovered'. These tend to be profiles from people who have never posted (and we have about 200k of those alone).
I do not believe we are facing any crisis, or that anything is substantially wrong.
We can always do better, and we're listening. We just need a little more than a few charts to go on before we make drastic change.

4.3 also uses 300 characters for meta description

4.3 also uses 300 characters for meta description

Ok, right away I can see the LD is fine.
"interactionStatistic": [ { "@type": "InteractionCounter", "interactionType": "http://schema.org/ViewAction", "userInteractionCount": 80927 }, { "@type": "InteractionCounter", "interactionType": "http://schema.org/CommentAction", "userInteractionCount": 1239 }, { "@type": "InteractionCounter", "interactionType": "http://schema.org/FollowAction", "userInteractionCount": 3 } ], Testing the link using Google's tool shows the meta data is being received perfectly.

Invision Community is doing its job.
 
 

I think we need to be mindful the the sitemap is just one way that Google discovers and crawls links.
What goes in the sitemap isn't a hard rule that Google must only check out those links, so there's little point in adding too many restrictions here and there because it'll be mostly pointless. You'll submit fewer links, but Google will still pull up the ones you didn't add.
I did add a setting for profiles, because of the huge number of 'dead' profiles that stuff up the sitemap, which is just a waste.

What may or may not be in the sitemap doesn't solve why Google is shedding indexed pages.
That said, when using the new search console, the figures are totally different.
We have 92k indexed pages
We have about 400k pages that Google has either 'discovered' or 'crawled but not indexed' due to its own algorithms. These are 301 redirect links (this is OK, it has no reason to store these) and empty profiles which have almost zero content.
But it's important to realise that Google is not punishing us, it is just working harder to index content that it thinks others will find useful, and "Johnny@11" who registered in 2011 and has never posted doesn't count any more.
 
 

Again, the sitemap is not a YOU CAN ONLY LOOK AT THESE LINKS GOOGLE LOL.
The sitemap just informs Google of "important" URLs on your site. It will use these as a base to spider out from.
I have no idea why Google is not updating the meta data of your indexed URL. That's not down to the sitemap. That's down to Google not refreshing the data. Google will pull the replies meta data from the page itself.
To save me bother, what is the URL to that topic? I'd like to review the meta tags in the json LD to make sure they're correct.

Just so you know, we're watching this topic and looking at our own stats to build a better picture.
The facts we know:
1) Almost every site I've got access to (via friends, etc) have seen a massive drop since June of indexed pages. This is not exclusive to Invision Community powered sites. I've seen the same with Wordpress.
2) Google slipped in an update in 2017 to target several things, one of these things is poor backlinks and other poor quality links. It looks like this means that user profiles that have no content have been dropped from the index along with links that 301. That is fine. You don't want Google storing the 301 link, as long as it stores the real link (and it does seem to).
3) A drop in what is indexed doesn't actually correlate to the health of the site. We've seen our index volume drop, but clicks, engagement and discovery slightly increase (probably due to better quality results?)
As always, Google say nothing so we're left guessing.
We will look at stopping user profiles from being submitted. For example, we see nearly 380k links as 'discovered' but Google has chosen to not index them. Looking through the list, it's all user profiles.
This means:
1) Sitemaps are working fine. There's no massive problem with them that correlates with a drop in indexed pages
2) The cornerstones of good SEO are taken care of in the software
3) Google is being weird and mysterious as always.
What can we do in the short term?
1) Stop sending profiles with no content to the sitemap. They are now ignored and Google appears to be dropping them from its indexes
2) Add in nofollow on links that 301 so Google doesn't bother 'discovering' them at all.

Just so you know, we're watching this topic and looking at our own stats to build a better picture.
The facts we know:
1) Almost every site I've got access to (via friends, etc) have seen a massive drop since June of indexed pages. This is not exclusive to Invision Community powered sites. I've seen the same with Wordpress.
2) Google slipped in an update in 2017 to target several things, one of these things is poor backlinks and other poor quality links. It looks like this means that user profiles that have no content have been dropped from the index along with links that 301. That is fine. You don't want Google storing the 301 link, as long as it stores the real link (and it does seem to).
3) A drop in what is indexed doesn't actually correlate to the health of the site. We've seen our index volume drop, but clicks, engagement and discovery slightly increase (probably due to better quality results?)
As always, Google say nothing so we're left guessing.
We will look at stopping user profiles from being submitted. For example, we see nearly 380k links as 'discovered' but Google has chosen to not index them. Looking through the list, it's all user profiles.
This means:
1) Sitemaps are working fine. There's no massive problem with them that correlates with a drop in indexed pages
2) The cornerstones of good SEO are taken care of in the software
3) Google is being weird and mysterious as always.
What can we do in the short term?
1) Stop sending profiles with no content to the sitemap. They are now ignored and Google appears to be dropping them from its indexes
2) Add in nofollow on links that 301 so Google doesn't bother 'discovering' them at all.

Just so you know, we're watching this topic and looking at our own stats to build a better picture.
The facts we know:
1) Almost every site I've got access to (via friends, etc) have seen a massive drop since June of indexed pages. This is not exclusive to Invision Community powered sites. I've seen the same with Wordpress.
2) Google slipped in an update in 2017 to target several things, one of these things is poor backlinks and other poor quality links. It looks like this means that user profiles that have no content have been dropped from the index along with links that 301. That is fine. You don't want Google storing the 301 link, as long as it stores the real link (and it does seem to).
3) A drop in what is indexed doesn't actually correlate to the health of the site. We've seen our index volume drop, but clicks, engagement and discovery slightly increase (probably due to better quality results?)
As always, Google say nothing so we're left guessing.
We will look at stopping user profiles from being submitted. For example, we see nearly 380k links as 'discovered' but Google has chosen to not index them. Looking through the list, it's all user profiles.
This means:
1) Sitemaps are working fine. There's no massive problem with them that correlates with a drop in indexed pages
2) The cornerstones of good SEO are taken care of in the software
3) Google is being weird and mysterious as always.
What can we do in the short term?
1) Stop sending profiles with no content to the sitemap. They are now ignored and Google appears to be dropping them from its indexes
2) Add in nofollow on links that 301 so Google doesn't bother 'discovering' them at all.

10 days might be fine depending on how often Google visits your site.
Again, the frequency that Google visits your site has nothing to do with the sitemap.
In 4.3, we have added the lastmod timestamp, and added a button to rebuild your index from scratch.
Also, just double check your forum and topic permissions. Remember, if a guest cannot see the page, then Google cannot either.

I think we need to be mindful the the sitemap is just one way that Google discovers and crawls links.
What goes in the sitemap isn't a hard rule that Google must only check out those links, so there's little point in adding too many restrictions here and there because it'll be mostly pointless. You'll submit fewer links, but Google will still pull up the ones you didn't add.
I did add a setting for profiles, because of the huge number of 'dead' profiles that stuff up the sitemap, which is just a waste.

What may or may not be in the sitemap doesn't solve why Google is shedding indexed pages.
That said, when using the new search console, the figures are totally different.
We have 92k indexed pages
We have about 400k pages that Google has either 'discovered' or 'crawled but not indexed' due to its own algorithms. These are 301 redirect links (this is OK, it has no reason to store these) and empty profiles which have almost zero content.
But it's important to realise that Google is not punishing us, it is just working harder to index content that it thinks others will find useful, and "Johnny@11" who registered in 2011 and has never posted doesn't count any more.
 
 

I think we need to be mindful the the sitemap is just one way that Google discovers and crawls links.
What goes in the sitemap isn't a hard rule that Google must only check out those links, so there's little point in adding too many restrictions here and there because it'll be mostly pointless. You'll submit fewer links, but Google will still pull up the ones you didn't add.
I did add a setting for profiles, because of the huge number of 'dead' profiles that stuff up the sitemap, which is just a waste.

What may or may not be in the sitemap doesn't solve why Google is shedding indexed pages.
That said, when using the new search console, the figures are totally different.
We have 92k indexed pages
We have about 400k pages that Google has either 'discovered' or 'crawled but not indexed' due to its own algorithms. These are 301 redirect links (this is OK, it has no reason to store these) and empty profiles which have almost zero content.
But it's important to realise that Google is not punishing us, it is just working harder to index content that it thinks others will find useful, and "Johnny@11" who registered in 2011 and has never posted doesn't count any more.
 
 

Just so you know, we're watching this topic and looking at our own stats to build a better picture.
The facts we know:
1) Almost every site I've got access to (via friends, etc) have seen a massive drop since June of indexed pages. This is not exclusive to Invision Community powered sites. I've seen the same with Wordpress.
2) Google slipped in an update in 2017 to target several things, one of these things is poor backlinks and other poor quality links. It looks like this means that user profiles that have no content have been dropped from the index along with links that 301. That is fine. You don't want Google storing the 301 link, as long as it stores the real link (and it does seem to).
3) A drop in what is indexed doesn't actually correlate to the health of the site. We've seen our index volume drop, but clicks, engagement and discovery slightly increase (probably due to better quality results?)
As always, Google say nothing so we're left guessing.
We will look at stopping user profiles from being submitted. For example, we see nearly 380k links as 'discovered' but Google has chosen to not index them. Looking through the list, it's all user profiles.
This means:
1) Sitemaps are working fine. There's no massive problem with them that correlates with a drop in indexed pages
2) The cornerstones of good SEO are taken care of in the software
3) Google is being weird and mysterious as always.
What can we do in the short term?
1) Stop sending profiles with no content to the sitemap. They are now ignored and Google appears to be dropping them from its indexes
2) Add in nofollow on links that 301 so Google doesn't bother 'discovering' them at all.

Again, the sitemap is not a YOU CAN ONLY LOOK AT THESE LINKS GOOGLE LOL.
The sitemap just informs Google of "important" URLs on your site. It will use these as a base to spider out from.
I have no idea why Google is not updating the meta data of your indexed URL. That's not down to the sitemap. That's down to Google not refreshing the data. Google will pull the replies meta data from the page itself.
To save me bother, what is the URL to that topic? I'd like to review the meta tags in the json LD to make sure they're correct.

I think we need to be mindful the the sitemap is just one way that Google discovers and crawls links.
What goes in the sitemap isn't a hard rule that Google must only check out those links, so there's little point in adding too many restrictions here and there because it'll be mostly pointless. You'll submit fewer links, but Google will still pull up the ones you didn't add.
I did add a setting for profiles, because of the huge number of 'dead' profiles that stuff up the sitemap, which is just a waste.

What may or may not be in the sitemap doesn't solve why Google is shedding indexed pages.
That said, when using the new search console, the figures are totally different.
We have 92k indexed pages
We have about 400k pages that Google has either 'discovered' or 'crawled but not indexed' due to its own algorithms. These are 301 redirect links (this is OK, it has no reason to store these) and empty profiles which have almost zero content.
But it's important to realise that Google is not punishing us, it is just working harder to index content that it thinks others will find useful, and "Johnny@11" who registered in 2011 and has never posted doesn't count any more.
 
 

Just so you know, we're watching this topic and looking at our own stats to build a better picture.
The facts we know:
1) Almost every site I've got access to (via friends, etc) have seen a massive drop since June of indexed pages. This is not exclusive to Invision Community powered sites. I've seen the same with Wordpress.
2) Google slipped in an update in 2017 to target several things, one of these things is poor backlinks and other poor quality links. It looks like this means that user profiles that have no content have been dropped from the index along with links that 301. That is fine. You don't want Google storing the 301 link, as long as it stores the real link (and it does seem to).
3) A drop in what is indexed doesn't actually correlate to the health of the site. We've seen our index volume drop, but clicks, engagement and discovery slightly increase (probably due to better quality results?)
As always, Google say nothing so we're left guessing.
We will look at stopping user profiles from being submitted. For example, we see nearly 380k links as 'discovered' but Google has chosen to not index them. Looking through the list, it's all user profiles.
This means:
1) Sitemaps are working fine. There's no massive problem with them that correlates with a drop in indexed pages
2) The cornerstones of good SEO are taken care of in the software
3) Google is being weird and mysterious as always.
What can we do in the short term?
1) Stop sending profiles with no content to the sitemap. They are now ignored and Google appears to be dropping them from its indexes
2) Add in nofollow on links that 301 so Google doesn't bother 'discovering' them at all.

Just so you know, we're watching this topic and looking at our own stats to build a better picture.
The facts we know:
1) Almost every site I've got access to (via friends, etc) have seen a massive drop since June of indexed pages. This is not exclusive to Invision Community powered sites. I've seen the same with Wordpress.
2) Google slipped in an update in 2017 to target several things, one of these things is poor backlinks and other poor quality links. It looks like this means that user profiles that have no content have been dropped from the index along with links that 301. That is fine. You don't want Google storing the 301 link, as long as it stores the real link (and it does seem to).
3) A drop in what is indexed doesn't actually correlate to the health of the site. We've seen our index volume drop, but clicks, engagement and discovery slightly increase (probably due to better quality results?)
As always, Google say nothing so we're left guessing.
We will look at stopping user profiles from being submitted. For example, we see nearly 380k links as 'discovered' but Google has chosen to not index them. Looking through the list, it's all user profiles.
This means:
1) Sitemaps are working fine. There's no massive problem with them that correlates with a drop in indexed pages
2) The cornerstones of good SEO are taken care of in the software
3) Google is being weird and mysterious as always.
What can we do in the short term?
1) Stop sending profiles with no content to the sitemap. They are now ignored and Google appears to be dropping them from its indexes
2) Add in nofollow on links that 301 so Google doesn't bother 'discovering' them at all.

Just so you know, we're watching this topic and looking at our own stats to build a better picture.
The facts we know:
1) Almost every site I've got access to (via friends, etc) have seen a massive drop since June of indexed pages. This is not exclusive to Invision Community powered sites. I've seen the same with Wordpress.
2) Google slipped in an update in 2017 to target several things, one of these things is poor backlinks and other poor quality links. It looks like this means that user profiles that have no content have been dropped from the index along with links that 301. That is fine. You don't want Google storing the 301 link, as long as it stores the real link (and it does seem to).
3) A drop in what is indexed doesn't actually correlate to the health of the site. We've seen our index volume drop, but clicks, engagement and discovery slightly increase (probably due to better quality results?)
As always, Google say nothing so we're left guessing.
We will look at stopping user profiles from being submitted. For example, we see nearly 380k links as 'discovered' but Google has chosen to not index them. Looking through the list, it's all user profiles.
This means:
1) Sitemaps are working fine. There's no massive problem with them that correlates with a drop in indexed pages
2) The cornerstones of good SEO are taken care of in the software
3) Google is being weird and mysterious as always.
What can we do in the short term?
1) Stop sending profiles with no content to the sitemap. They are now ignored and Google appears to be dropping them from its indexes
2) Add in nofollow on links that 301 so Google doesn't bother 'discovering' them at all.

Just so you know, we're watching this topic and looking at our own stats to build a better picture.
The facts we know:
1) Almost every site I've got access to (via friends, etc) have seen a massive drop since June of indexed pages. This is not exclusive to Invision Community powered sites. I've seen the same with Wordpress.
2) Google slipped in an update in 2017 to target several things, one of these things is poor backlinks and other poor quality links. It looks like this means that user profiles that have no content have been dropped from the index along with links that 301. That is fine. You don't want Google storing the 301 link, as long as it stores the real link (and it does seem to).
3) A drop in what is indexed doesn't actually correlate to the health of the site. We've seen our index volume drop, but clicks, engagement and discovery slightly increase (probably due to better quality results?)
As always, Google say nothing so we're left guessing.
We will look at stopping user profiles from being submitted. For example, we see nearly 380k links as 'discovered' but Google has chosen to not index them. Looking through the list, it's all user profiles.
This means:
1) Sitemaps are working fine. There's no massive problem with them that correlates with a drop in indexed pages
2) The cornerstones of good SEO are taken care of in the software
3) Google is being weird and mysterious as always.
What can we do in the short term?
1) Stop sending profiles with no content to the sitemap. They are now ignored and Google appears to be dropping them from its indexes
2) Add in nofollow on links that 301 so Google doesn't bother 'discovering' them at all.

Just so you know, we're watching this topic and looking at our own stats to build a better picture.
The facts we know:
1) Almost every site I've got access to (via friends, etc) have seen a massive drop since June of indexed pages. This is not exclusive to Invision Community powered sites. I've seen the same with Wordpress.
2) Google slipped in an update in 2017 to target several things, one of these things is poor backlinks and other poor quality links. It looks like this means that user profiles that have no content have been dropped from the index along with links that 301. That is fine. You don't want Google storing the 301 link, as long as it stores the real link (and it does seem to).
3) A drop in what is indexed doesn't actually correlate to the health of the site. We've seen our index volume drop, but clicks, engagement and discovery slightly increase (probably due to better quality results?)
As always, Google say nothing so we're left guessing.
We will look at stopping user profiles from being submitted. For example, we see nearly 380k links as 'discovered' but Google has chosen to not index them. Looking through the list, it's all user profiles.
This means:
1) Sitemaps are working fine. There's no massive problem with them that correlates with a drop in indexed pages
2) The cornerstones of good SEO are taken care of in the software
3) Google is being weird and mysterious as always.
What can we do in the short term?
1) Stop sending profiles with no content to the sitemap. They are now ignored and Google appears to be dropping them from its indexes
2) Add in nofollow on links that 301 so Google doesn't bother 'discovering' them at all.

We've added the timestamp into the sitemap and we're looking to add a tool to quickly rebuild the sitemap on demand.

We've added the timestamp into the sitemap and we're looking to add a tool to quickly rebuild the sitemap on demand.