Sonya* Posted October 3, 2019 Share Posted October 3, 2019 5 hours ago, sadams101 said: As you may recall I had some custom sitemaps developed which include a map to every post made, and a map for all comments. I wonder, how it can help. This should result in a lot of Excluded URLs due to "Alternate page with proper canonical tag". Every post and comment has a canonical tag in the header pointing to the page, so it does not really make sense to submit them to the sitemap. Those pages should not be indexed. Daniel F and bfarber 2 Link to comment Share on other sites More sharing options...
sadams101 Posted October 3, 2019 Share Posted October 3, 2019 I am not seeing a rise in "Alternate page with proper canonical tag" and in fact, am seeing a decrease from 60,927 to 54,105, or 11% over the time period shown (Chart 1). Duplicate, Google chose different canonical than user is also dropping (Chart 2). Chart 1: Chart 2: Link to comment Share on other sites More sharing options...
AlexWebsites Posted October 3, 2019 Share Posted October 3, 2019 @sadams101 what about your active urls, have they gone up? Can you post a screen shot of that section? That and organic traffic interest me the most. Link to comment Share on other sites More sharing options...
sadams101 Posted October 3, 2019 Share Posted October 3, 2019 My active has gone down during this time, but I have also archived ~60% of my older forum content during this time, so this content is no longer in my sitemap. So you need to actually combine two charts to get the bigger picture. Chart 1 shows a drop of 19,625 in the Submitted and indexed, which looks really bad, however, Chart 2 shows in increase of Indexed, not submitted in sitemap during this time. Overall, I don't see a big change in the total number of links indexed, it has been at ~75 - 80K for a long time, but I am seeing an increase in organic traffic over the last 3 months of around 16%. I am wondering if it makes sense to exclude archived posts from the sitemap. There is probably a good argument to keep them in. Chart 1: Chart 2: Link to comment Share on other sites More sharing options...
Sonya* Posted October 3, 2019 Share Posted October 3, 2019 41 minutes ago, sadams101 said: in increase of Indexed, not submitted in sitemap It literally means, your custom sitemaps are not really helpful. You have increase in pages that are not listed in sitemap Link to comment Share on other sites More sharing options...
sadams101 Posted October 3, 2019 Share Posted October 3, 2019 (edited) I'm not sure you read the part about my archiving over 60% of my site...but I am seeing a big increase in search clicks and organic traffic lately, so including them does not seem to be hurting in any way, and could be helping. I would like to hear input on whether or not archived posts should be excluded from the sitemap. I understand the idea of not allowing users to reply to very old content, but I'm not sure removing them from the sitemap is helpful...it might cause issues. Edited October 3, 2019 by sadams101 Link to comment Share on other sites More sharing options...
Fast Lane! Posted October 4, 2019 Share Posted October 4, 2019 FYI. My sitemap doesn't include all my topics and is often days behind because I'm told of a known issue for large sites where the task process is too slow. Link to comment Share on other sites More sharing options...
sadams101 Posted November 11, 2019 Share Posted November 11, 2019 I wanted to post an updated on my custom posts sitemap, as there are various opinions here about whether or not it can help or hurt an IPS site. In my case I have nearly 1M posts, and none of their URL's were in the sitemap directly. Now, it appears that google has indeed indexed nearly all of them, and I do not see any issue regarding the canonical links for them on the forum category pages. Currently I see zero errors, and now only 71.5K are excluded, and zero are listed in the "Duplicate, submitted URL not selected as canonical" line. Also, my site traffic has been slowly recovering...in the last 30 days I've seen a 17% increase over the prior 30 days. So far I have seen nothing negative about using it, and I've been running it for 6 months now. During the time I've been running it I've seen slow and steady improvement in my page rank, and more importantly, the number of key words in google's index for my site has increased dramatically (it hit an all-time low around 6 months ago--coincidence?). AlexWebsites 1 Link to comment Share on other sites More sharing options...
sadams101 Posted November 11, 2019 Share Posted November 11, 2019 PS - The forum's category pages, beyond the initial category page, are also not in the sitemap in both the forums and pages apps, and we've also created site maps to include all pages, for example /page/2/, /page/3/, /page/4/. AlexWebsites 1 Link to comment Share on other sites More sharing options...
Sonya* Posted November 11, 2019 Share Posted November 11, 2019 46 minutes ago, sadams101 said: I wanted to post an updated on my custom posts sitemap, as there are various opinions here about whether or not it can help or hurt an IPS site. In my case I have nearly 1M posts, and none of their URL's were in the sitemap directly. Now, it appears that google has indeed indexed nearly all of them, and I do not see any issue regarding the canonical links for them on the forum category pages. Currently I see zero errors, and now only 71.5K are excluded, and zero are listed in the "Duplicate, submitted URL not selected as canonical" line. Also, my site traffic has been slowly recovering...in the last 30 days I've seen a 17% increase over the prior 30 days. So far I have seen nothing negative about using it, and I've been running it for 6 months now. During the time I've been running it I've seen slow and steady improvement in my page rank, and more importantly, the number of key words in google's index for my site has increased dramatically (it hit an all-time low around 6 months ago--coincidence?). You have 0 valid pages on your screen Link to comment Share on other sites More sharing options...
sadams101 Posted November 11, 2019 Share Posted November 11, 2019 You can take it or leave it...my site did not begin any type of recovery until I implemented these custom sitemaps, which I make no money from--I was the client who paid a consultant to make them: Link to comment Share on other sites More sharing options...
Sonya* Posted November 11, 2019 Share Posted November 11, 2019 Just now, sadams101 said: You can take it or leave it... I leave it. We are running extended sitemap for the pages (2,3,4 and so on) as well. It make indeed sense. Your posts sitemap does not make sense. And your screen is a prove -> sitemap for posts does contain zero valid pages. Numbered 1 Link to comment Share on other sites More sharing options...
sadams101 Posted November 11, 2019 Share Posted November 11, 2019 It is certainly possible that the full category maps, with ALL pages are having a bigger impact than the posts map, but I've seen only positive results after implementing both. Perhaps someone can explain why the extra pages in a category are not in a sitemap, and why that's a good idea (it is where all of the canonical links reside)? Link to comment Share on other sites More sharing options...
Numbered Posted November 12, 2019 Author Share Posted November 12, 2019 (edited) @Sonya* and @sadams101 you are both right. Sonya right in the part of using posts items instead of topics wrong way to the right SEO. Sadams101 right in the part of the pagination problem. Thank you for get attention to the pagination problem. I try to do a little bit of research and found that IPS might be better with that important part. For example, this page has a canonical tag: <link rel="canonical" href="https://invisioncommunity.com/forums/topic/442742-large-community-you-have-a-problems-with-sitemap/"> This tag said that Google should stick to the first page of the series. And the cannonical tag links to the 'only' or 'primary' version of the same page. But other pages are not the same as first - it's wrong. Anyway, Google can understand that the other pages haven't the same content and it might ignore the canonical tag. But there Google doing that scans contrary to, and not by a clear intent. There is no way to said Google about pagination. But somebody recommends to use rel=prev/next. This tags exist here, but with strange params: <link rel="next" href="https://invisioncommunity.com/forums/topic/442742-large-community-you-have-a-problems-with-sitemap/page/2/"> <link rel="last" href="https://invisioncommunity.com/forums/topic/442742-large-community-you-have-a-problems-with-sitemap/page/12/"> <link rel="canonical" href="https://invisioncommunity.com/forums/topic/442742-large-community-you-have-a-problems-with-sitemap/"> It seems like the 'next' param didn't work for the current page and put params depends only on first page. And there is no 'prev' page - I think for the same reason. Some other example from SEO websites and cannonical tag: Quote For example: <link rel="next" href="https://www.example.com/category?page=2&order=newest" /> <link rel="canonical" href="https://www.example.com/category?page=2" /> Doing so will indicate a clear relationship between the pages and prevent the potential of duplicate content. I think anybody can search this tags and found the same information that I wrote. To be clear - this topic for the current page (12) should have this tags: <link rel="first" href="https://invisioncommunity.com/forums/topic/442742-large-community-you-have-a-problems-with-sitemap/"> <link rel="prev" href="https://invisioncommunity.com/forums/topic/442742-large-community-you-have-a-problems-with-sitemap/page/11/"> <link rel="last" href="https://invisioncommunity.com/forums/topic/442742-large-community-you-have-a-problems-with-sitemap/page/12/"> <link rel="canonical" href="https://invisioncommunity.com/forums/topic/442742-large-community-you-have-a-problems-with-sitemap/page/12/"> page 10 should have this tags: <link rel="first" href="https://invisioncommunity.com/forums/topic/442742-large-community-you-have-a-problems-with-sitemap/"> <link rel="prev" href="https://invisioncommunity.com/forums/topic/442742-large-community-you-have-a-problems-with-sitemap/page/9/"> <link rel="next" href="https://invisioncommunity.com/forums/topic/442742-large-community-you-have-a-problems-with-sitemap/page/11/"> <link rel="last" href="https://invisioncommunity.com/forums/topic/442742-large-community-you-have-a-problems-with-sitemap/page/12/"> <link rel="canonical" href="https://invisioncommunity.com/forums/topic/442742-large-community-you-have-a-problems-with-sitemap/page/10/"> Edited November 12, 2019 by Upgradeovec Link to comment Share on other sites More sharing options...
Daniel F Posted November 12, 2019 Share Posted November 12, 2019 It seems that Google isn’t using this stuff since years Link to comment Share on other sites More sharing options...
Numbered Posted November 12, 2019 Author Share Posted November 12, 2019 (edited) Maybe. I'm not 100% sure that John said fact. If I were him I'll think twice to say that next/prev working for preventing stupid SEO hacks and panic. (for Google absolutely better if the webmasters wouldn't use these helpings than they use them in the wrong way) And there is no Google only search engine. And point about the wrong canonical still present. Yes, Google easily can ignore them. But better to help him do the right things than confuse him. P.S. IPS doesn't love me again (writing from TOR) Edited November 12, 2019 by Upgradeovec Link to comment Share on other sites More sharing options...
Sonya* Posted November 12, 2019 Share Posted November 12, 2019 34 minutes ago, Upgradeovec said: There is no way to said Google about pagination. But somebody recommends to use rel=prev/next. Unfortunately Google does not support Rel=prev/next any more. I cannot see an issue you with these tags though on this community. This is what I see on this page: <link rel="first" href="https://invisioncommunity.com/forums/topic/442742-large-community-you-have-a-problems-with-sitemap/" /> <link rel="prev" href="https://invisioncommunity.com/forums/topic/442742-large-community-you-have-a-problems-with-sitemap/page/11/" /> <link rel="canonical" href="https://invisioncommunity.com/forums/topic/442742-large-community-you-have-a-problems-with-sitemap/page/12/" /> This is what I see on the previous page: <link rel="first" href="https://invisioncommunity.com/forums/topic/442742-large-community-you-have-a-problems-with-sitemap/" /> <link rel="prev" href="https://invisioncommunity.com/forums/topic/442742-large-community-you-have-a-problems-with-sitemap/page/10/" /> <link rel="next" href="https://invisioncommunity.com/forums/topic/442742-large-community-you-have-a-problems-with-sitemap/page/12/" /> <link rel="last" href="https://invisioncommunity.com/forums/topic/442742-large-community-you-have-a-problems-with-sitemap/page/12/" /> <link rel="canonical" href="https://invisioncommunity.com/forums/topic/442742-large-community-you-have-a-problems-with-sitemap/page/11/" /> There is neither wrong canonical nor false pagination Link to comment Share on other sites More sharing options...
Numbered Posted November 12, 2019 Author Share Posted November 12, 2019 Wow! I see that too @Sonya*! I think IPS fixed that right now! Sonya* 1 Link to comment Share on other sites More sharing options...
Daniel F Posted November 12, 2019 Share Posted November 12, 2019 5 minutes ago, Sonya* said: Unfortunately Google does not support Rel=prev/next any more. Thanks for the link 🙂 Link to comment Share on other sites More sharing options...
bfarber Posted November 12, 2019 Share Posted November 12, 2019 6 hours ago, Upgradeovec said: Wow! I see that too @Sonya*! I think IPS fixed that right now! No, we didn't. I think you might have paginated from page 1 using AJAX and ended up looking at the canonical tag that was initially loaded, or simply made a mistake. 😛 Link to comment Share on other sites More sharing options...
sadams101 Posted November 12, 2019 Share Posted November 12, 2019 I think the pagination tags are now fixed, but those pages still are not in your site maps. They are in my custom maps. Here is my Pages custom map which shows all pages in my categories: https://www.celiac.com/sitemap_cmscategories_pages.php and here is my forum map that does the same: https://www.celiac.com/sitemap_pages.php I lost track of Google's blog post which basically said that if you want specific URLs indexed and in their search engine, just put them in your site map. Another way of looking at this is that if they are not in your sitemap Google might not think they are important enough for their search engine. Certainly with their now huge focus on "crawl budget," why would they waste their resources by indexing and crawling pages that you don't think are important enough to put into your site map? Link to comment Share on other sites More sharing options...
Caputo Posted December 5, 2019 Share Posted December 5, 2019 (edited) Hello I also have a problem, who can tell me why this error occurred? it may be a problem that I don't have a sitemap like : sitemap_pages.php Edited December 5, 2019 by Caputo Link to comment Share on other sites More sharing options...
SeNioR- Posted June 5, 2021 Share Posted June 5, 2021 (edited) Bump! 🙂 In these two years, the sitemap has been improved a bit. Let me know if you have better results in the search engine. Edited June 5, 2021 by SeNioR- AlexWebsites 1 Link to comment Share on other sites More sharing options...
Fast Lane! Posted July 7, 2023 Share Posted July 7, 2023 Is this still needed? bump bump!! Link to comment Share on other sites More sharing options...
SeNioR- Posted July 7, 2023 Share Posted July 7, 2023 12 minutes ago, Fast Lane! said: Is this still needed? bump bump!! no Link to comment Share on other sites More sharing options...
Recommended Posts