Jump to content

Large community? You have a problems with sitemap!


Numbered

Recommended Posts

5 hours ago, sadams101 said:

As you may recall I had some custom sitemaps developed which include a map to every post made, and a map for all comments.

I wonder, how it can help. This should result in a lot of Excluded URLs due to "Alternate page with proper canonical tag". Every post and comment has a canonical tag in the header pointing to the page, so it does not really make sense to submit them to the sitemap.

screenshot-search.google.com-2019_10.03-13-09-33.thumb.png.cc4eef4b52a9b4eb3b35431af8d808ce.png

Those pages should not be indexed. 

 

Link to comment
Share on other sites

I am not seeing a rise in "Alternate page with proper canonical tag" and in fact, am seeing a decrease from 60,927 to 54,105, or 11% over the time period shown (Chart 1). Duplicate, Google chose different canonical than user is also dropping (Chart 2).

Chart 1:

image.thumb.png.0a96be3a35c360e6a9403a72db366278.png

 

Chart 2:

image.thumb.png.a28dc3bc3347a80ffbebeb13b7ff666a.png

Link to comment
Share on other sites

My active has gone down during this time, but I have also archived ~60% of my older forum content during this time, so this content is no longer in my sitemap. So you need to actually combine two charts to get the bigger picture. Chart 1 shows a drop of 19,625 in the Submitted and indexed, which looks really bad, however, Chart 2 shows in increase of Indexed, not submitted in sitemap during this time. Overall, I don't see a big change in the total number of links indexed, it has been at ~75 - 80K for a long time, but I am seeing an increase in organic traffic over the last 3 months of around 16%.

I am wondering if it makes sense to exclude archived posts from the sitemap. There is probably a good argument to keep them in.

Chart 1:

image.thumb.png.3ebafbd8dd8a059b56e5927e519a8813.png

 

Chart 2:

image.thumb.png.77ddb160d52b3de9847ba877fedeb61e.png

 

 

Link to comment
Share on other sites

I'm not sure you read the part about my archiving over 60% of my site...but I am seeing a big increase in search clicks and organic traffic lately, so including them does not seem to be hurting in any way, and could be helping.

I would like to hear input on whether or not archived posts should be excluded from the sitemap. I understand the idea of not allowing users to reply to very old content, but I'm not sure removing them from the sitemap is helpful...it might cause issues.

Edited by sadams101
Link to comment
Share on other sites

  • 1 month later...

I wanted to post an updated on my custom posts sitemap, as there are various opinions here about whether or not it can help or hurt an IPS site. In my case I have nearly 1M posts, and none of their URL's were in the sitemap directly. Now, it appears that google has indeed indexed nearly all of them, and I do not see any issue regarding the canonical links for them on the forum category pages. Currently I see zero errors, and now only 71.5K are excluded, and zero are listed in the "Duplicate, submitted URL not selected as canonical" line.

Also, my site traffic has been slowly recovering...in the last 30 days I've seen a 17% increase over the prior 30 days. So far I have seen nothing negative about using it, and I've been running it for 6 months now. During the time I've been running it I've seen slow and steady improvement in my page rank, and more importantly, the number of key words in google's index for my site has increased dramatically (it hit an all-time low around 6 months ago--coincidence?).

 

image.thumb.png.730ea185cdda8fa50b158423ebe09bbb.png

Link to comment
Share on other sites

46 minutes ago, sadams101 said:

I wanted to post an updated on my custom posts sitemap, as there are various opinions here about whether or not it can help or hurt an IPS site. In my case I have nearly 1M posts, and none of their URL's were in the sitemap directly. Now, it appears that google has indeed indexed nearly all of them, and I do not see any issue regarding the canonical links for them on the forum category pages. Currently I see zero errors, and now only 71.5K are excluded, and zero are listed in the "Duplicate, submitted URL not selected as canonical" line.

Also, my site traffic has been slowly recovering...in the last 30 days I've seen a 17% increase over the prior 30 days. So far I have seen nothing negative about using it, and I've been running it for 6 months now. During the time I've been running it I've seen slow and steady improvement in my page rank, and more importantly, the number of key words in google's index for my site has increased dramatically (it hit an all-time low around 6 months ago--coincidence?).

 

image.thumb.png.730ea185cdda8fa50b158423ebe09bbb.png

You have 0 valid pages on your screen :wacko:

Link to comment
Share on other sites

It is certainly possible that the full category maps, with ALL pages are having a bigger impact than the posts map, but I've seen only positive results after implementing both. Perhaps someone can explain why the extra pages in a category are not in a sitemap, and why that's a good idea (it is where all of the canonical links reside)?

Link to comment
Share on other sites

@Sonya* and @sadams101 you are both right.

Sonya right in the part of using posts items instead of topics wrong way to the right SEO.

Sadams101 right in the part of the pagination problem. Thank you for get attention to the pagination problem. I try to do a little bit of research and found that IPS might be better with that important part. For example, this page has a canonical tag:

<link rel="canonical" href="https://invisioncommunity.com/forums/topic/442742-large-community-you-have-a-problems-with-sitemap/">

This tag said that Google should stick to the first page of the series. And the cannonical tag links to the 'only' or 'primary' version of the same page. But other pages are not the same as first - it's wrong. Anyway, Google can understand that the other pages haven't the same content and it might ignore the canonical tag. But there Google doing that scans contrary to, and not by a clear intent.

There is no way to said Google about pagination. But somebody recommends to use rel=prev/next. This tags exist here, but with strange params: 

<link rel="next" href="https://invisioncommunity.com/forums/topic/442742-large-community-you-have-a-problems-with-sitemap/page/2/">
<link rel="last" href="https://invisioncommunity.com/forums/topic/442742-large-community-you-have-a-problems-with-sitemap/page/12/">
<link rel="canonical" href="https://invisioncommunity.com/forums/topic/442742-large-community-you-have-a-problems-with-sitemap/">

It seems like the 'next' param didn't work for the current page and put params depends only on first page. And there is no 'prev' page - I think for the same reason.

Some other example from SEO websites and cannonical tag:

Quote

For example:


<link rel="next" href="https://www.example.com/category?page=2&order=newest" />
<link rel="canonical" href="https://www.example.com/category?page=2" />

Doing so will indicate a clear relationship between the pages and prevent the potential of duplicate content.

I think anybody can search this tags and found the same information that I wrote.

To be clear - this topic for the current page (12) should have this tags:

<link rel="first" href="https://invisioncommunity.com/forums/topic/442742-large-community-you-have-a-problems-with-sitemap/">
<link rel="prev" href="https://invisioncommunity.com/forums/topic/442742-large-community-you-have-a-problems-with-sitemap/page/11/">
<link rel="last" href="https://invisioncommunity.com/forums/topic/442742-large-community-you-have-a-problems-with-sitemap/page/12/">
<link rel="canonical" href="https://invisioncommunity.com/forums/topic/442742-large-community-you-have-a-problems-with-sitemap/page/12/">

page 10 should have this tags:

<link rel="first" href="https://invisioncommunity.com/forums/topic/442742-large-community-you-have-a-problems-with-sitemap/">
<link rel="prev" href="https://invisioncommunity.com/forums/topic/442742-large-community-you-have-a-problems-with-sitemap/page/9/">
<link rel="next" href="https://invisioncommunity.com/forums/topic/442742-large-community-you-have-a-problems-with-sitemap/page/11/">
<link rel="last" href="https://invisioncommunity.com/forums/topic/442742-large-community-you-have-a-problems-with-sitemap/page/12/">
<link rel="canonical" href="https://invisioncommunity.com/forums/topic/442742-large-community-you-have-a-problems-with-sitemap/page/10/">

 

 

Edited by Upgradeovec
Link to comment
Share on other sites

Maybe. I'm not 100% sure that John said fact. If I were him I'll think twice to say that next/prev working for preventing stupid SEO hacks and panic. (for Google absolutely better if the webmasters wouldn't use these helpings than they use them in the wrong way)

And there is no Google only search engine. And point about the wrong canonical still present. Yes, Google easily can ignore them. But better to help him do the right things than confuse him.

P.S. IPS doesn't love me again (writing from TOR)

image.thumb.png.25eaffeafc3378cafa0bf7c9d77c1b5d.png

 

Edited by Upgradeovec
Link to comment
Share on other sites

34 minutes ago, Upgradeovec said:

There is no way to said Google about pagination. But somebody recommends to use rel=prev/next.

Unfortunately Google does not support Rel=prev/next any more.

I cannot see an issue you with these tags though on this community. This is what I see on this page:

<link rel="first" href="https://invisioncommunity.com/forums/topic/442742-large-community-you-have-a-problems-with-sitemap/" />
<link rel="prev" href="https://invisioncommunity.com/forums/topic/442742-large-community-you-have-a-problems-with-sitemap/page/11/" />
<link rel="canonical" href="https://invisioncommunity.com/forums/topic/442742-large-community-you-have-a-problems-with-sitemap/page/12/" />

This is what I see on the previous page:

<link rel="first" href="https://invisioncommunity.com/forums/topic/442742-large-community-you-have-a-problems-with-sitemap/" />
<link rel="prev" href="https://invisioncommunity.com/forums/topic/442742-large-community-you-have-a-problems-with-sitemap/page/10/" />
<link rel="next" href="https://invisioncommunity.com/forums/topic/442742-large-community-you-have-a-problems-with-sitemap/page/12/" />
<link rel="last" href="https://invisioncommunity.com/forums/topic/442742-large-community-you-have-a-problems-with-sitemap/page/12/" />
<link rel="canonical" href="https://invisioncommunity.com/forums/topic/442742-large-community-you-have-a-problems-with-sitemap/page/11/" />

There is neither wrong canonical nor false pagination :wacko:

Link to comment
Share on other sites

I think the pagination tags are now fixed, but those pages still are not in your site maps. They are in my custom maps. Here is my Pages custom map which shows all pages in my categories:

https://www.celiac.com/sitemap_cmscategories_pages.php

and here is my forum map that does the same:

https://www.celiac.com/sitemap_pages.php

I lost track of Google's blog post which basically said that if you want specific URLs indexed and in their search engine, just put them in your site map. Another way of looking at this is that if they are not in your sitemap Google might not think they are important enough for their search engine. Certainly with their now huge focus on "crawl budget," why would they waste their resources by indexing and crawling pages that you don't think are important enough to put into your site map?

Link to comment
Share on other sites

  • 4 weeks later...
  • 1 year later...
  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...