ProSkill Posted June 15, 2018 Share Posted June 15, 2018 I have been saying for a long time that there seems to be an issue with the IPB sitemap generator. I have been told by one member in particular whose name I shall not mention that the problem is with my site and not the sitemap, despite the fact that many others are reporting similar results. Before I go on, I am running 4.3.3 on a dedicated server. At one point two years ago I had almost 400k indexed pages in Google, however, after the Fred update and 4.2 upgrade I lost close to 300k indexed pages, at the lowest point I was down to about 92k indexed pages. After doing the 4.3 update I ran the rebuild sitemap tool hoping that it would update the sitemap and submit new/moved/edited pages and that the sitemap would be larger, however, the opposite appears to have happened my sitemap went down from approximately 78k pages to about 11k pages and I still don't understand why. In any case, I saw in another thread someone mentioned that they were using a paid sitemap service and that they had seen good results so I decided to try it. Here are my results so far, I started this on June 1st. Here are my indexed pages, they were trending down, however, notice on or about 6/1 the number of indexed pages shot up significantly. On 5/13 I had only 92k pages indexed, by 6/10 that rocketed to more than 146k indexed pages. Here are the details from my forum generated sitemap, notice how it only has 10,857 pages Here are the results from the paid sitemap service, 92,222 submitted pages, with 40k already indexed. Here are the pages processed by the sitemap service: ` Conclusion, the default IPB sitemap has issues. Secondly, to those who say that the sitemap doesn't matter, I would argue that these results show pretty clearly that the sitemap does matter. What do you think? Link to comment Share on other sites More sharing options...
opentype Posted June 15, 2018 Share Posted June 15, 2018 3 hours ago, ProSkill said: Conclusion, the default IPB sitemap has issues. Link to comment Share on other sites More sharing options...
ProSkill Posted June 15, 2018 Author Share Posted June 15, 2018 7 hours ago, opentype said: How do you explain these results? You seem to think that the sitemap doesn't matter even though I've just provided conclusive evidence that it does. A larger more thorough sitemap, equals more indexed pages, which equals more pages showing up in SERPS, which equals more traffic. Link to comment Share on other sites More sharing options...
AlexWebsites Posted June 15, 2018 Share Posted June 15, 2018 Your paid site map service is including links to posts and other page links as separate urls most likely. The question is, has your organic traffic increased at all as your indexes pages have? If it has, there’s something to say about that then. I have a site that used to be in vbulletin and vbseo. Had thousands of urls in my forum site map for topics and posts. Moved my site to IPS and SSL and my traffic dropped 1/2, even with redirects in place. This was last October. Now in May my traffic is the same as my highest point ever with this site and in June it’s surpassed that. Most traffic is organic from Google. I cleaned up my site by having IPS and I belive my content is being weighted better. I’ll also add that I started to use IPS built in promotion tools with Facebook and Twitter regularly and my traffic started to get better. I think there might be a correlation. It looks like you have way more urls in your 3rd party solution but the important part is whether it’s bringing you added traffic. If it is, I am for it and maybe IPS should add an option to include more urls for those that want to do this. Link to comment Share on other sites More sharing options...
opentype Posted June 15, 2018 Share Posted June 15, 2018 1 hour ago, ProSkill said: even though I've just provided conclusive evidence that it does. No, you did not. Evidence would need to demonstrate the CONNECTION between your sitemap and your Google index stats. You don’t even provide a hint, let alone “evidence” for that, nor can you even explain why and how those things would be connected this way in the first place. You literally have nothing to support your case. You just repeat an unsupported ASSERTION. “My Google stats go up and do – so I blame the IPS sitemap”. That’s not “conclusive evidence”. It’s just an assertion. But I explained all that ten times already … Link to comment Share on other sites More sharing options...
ProSkill Posted June 15, 2018 Author Share Posted June 15, 2018 4 hours ago, opentype said: No, you did not. Evidence would need to demonstrate the CONNECTION between your sitemap and your Google index stats. You don’t even provide a hint, let alone “evidence” for that, nor can you even explain why and how those things would be connected this way in the first place. You literally have nothing to support your case. You just repeat an unsupported ASSERTION. “My Google stats go up and do – so I blame the IPS sitemap”. That’s not “conclusive evidence”. It’s just an assertion. But I explained all that ten times already … I am starting to think that you're just here to troll. You never have anything constructive to say and you're unnecessarily combative. The evidence is clear, my indexed pages were going down for almost year, and my sitemap was much smaller after 4.3. Then I guess in your opinion, completely coincidentally, the week that I start using a paid sitemap service 81k plus pages are added to my sitemap, and magically my indexed pages in Google go up by almost 50k. I get that you're an IPS fan boy, I am too to some extent, but there is clearly an issue here. Link to comment Share on other sites More sharing options...
ProSkill Posted June 15, 2018 Author Share Posted June 15, 2018 4 hours ago, AlexWebsites said: Your paid site map service is including links to posts and other page links as separate urls most likely. The question is, has your organic traffic increased at all as your indexes pages have? If it has, there’s something to say about that then. I have a site that used to be in vbulletin and vbseo. Had thousands of urls in my forum site map for topics and posts. Moved my site to IPS and SSL and my traffic dropped 1/2, even with redirects in place. This was last October. Now in May my traffic is the same as my highest point ever with this site and in June it’s surpassed that. Most traffic is organic from Google. I cleaned up my site by having IPS and I belive my content is being weighted better. I’ll also add that I started to use IPS built in promotion tools with Facebook and Twitter regularly and my traffic started to get better. I think there might be a correlation. It looks like you have way more urls in your 3rd party solution but the important part is whether it’s bringing you added traffic. If it is, I am for it and maybe IPS should add an option to include more urls for those that want to do this. 1. It is not including links to posts and pages separately. My forum has has over 800k posts, and 8k+ topics. 2. I don't know yet, I only started this two weeks ago. This is also the traditionally slow time of year for my site. I will report back later with more results. Though I am quite confident that having an additional 80k indexed pages in Google will bring in more traffic. 3. My traffic has been steady overall, it's actually up over this time last year. Regardless it's rather concerning when you see that your indexed pages are dropping by hundreds or thousands every few weeks. My forum has been around for 5+ years and we have an active member base so we aren't as dependent on organic search engine traffic. 4. Again, I don't know what the result will be. However, the more indexed pages the better, as it means that your site is showing up in more places in Google. It also more accurately represents the size of the site, a 400k page site will have more authority than a 4k site (at least in theory) Link to comment Share on other sites More sharing options...
Surendra.S Posted June 15, 2018 Share Posted June 15, 2018 @ProSkill Whats the paid sitemap service you are using ? Any link ? Link to comment Share on other sites More sharing options...
AlexWebsites Posted June 15, 2018 Share Posted June 15, 2018 3 hours ago, ProSkill said: 1. It is not including links to posts and pages separately. My forum has has over 800k posts, and 8k+ topics. 2. I don't know yet, I only started this two weeks ago. This is also the traditionally slow time of year for my site. I will report back later with more results. Though I am quite confident that having an additional 80k indexed pages in Google will bring in more traffic. 3. My traffic has been steady overall, it's actually up over this time last year. Regardless it's rather concerning when you see that your indexed pages are dropping by hundreds or thousands every few weeks. My forum has been around for 5+ years and we have an active member base so we aren't as dependent on organic search engine traffic. 4. Again, I don't know what the result will be. However, the more indexed pages the better, as it means that your site is showing up in more places in Google. It also more accurately represents the size of the site, a 400k page site will have more authority than a 4k site (at least in theory) I agree that it can't hurt to have more pages indexed. If you have 8k+ topics, where are you getting 92k+ submitted urls from? What type of additional urls. The first screenshot showed 10k+ urls and those are probably direct IPS links to pages, just curious if its not posts, what's driving the new 92k+ count of urls in the sitemap? Link to comment Share on other sites More sharing options...
ProSkill Posted June 16, 2018 Author Share Posted June 16, 2018 5 hours ago, Surendra.S said: @ProSkill Whats the paid sitemap service you are using ? Any link ? https://pro-sitemaps.com/ Link to comment Share on other sites More sharing options...
Duken Posted June 17, 2018 Share Posted June 17, 2018 Pro sitemaps.com is really expensive ? im using a local sitemap tool for a test. a1 sitemap generator It taken me more then 1.5 days now to download all the links ? but I will add this created sitemap next to the ipb one. I have the same feeling about missing out visitors. Maby the ipb sitemap is the best official way. But if I don't try I'll never really know. The thing is, I upgraded to 4.3 and more links are getting indexed then 4.2. so I think something's changed on the background. But there was no sitemap "issue" on 4.2. What are you having for payed service @ProSkill? Link to comment Share on other sites More sharing options...
AlexWebsites Posted June 24, 2018 Share Posted June 24, 2018 @ProSkill I'm still curious how you got to 92k urls in your sitemap vs the default ips sitemap. That service must be picking up additional urls from topic pages or actual post links, right? The million dollar question is, has your organic traffic increased with having more urls off the sitemap indexed? Link to comment Share on other sites More sharing options...
Sheffielder Posted June 30, 2018 Share Posted June 30, 2018 It's turning into quite a debate I'm really interested in how traffic has been affected by this sitemap thing being introduced To be honest if it radically increases traffic then I'll be straight there to try it too Link to comment Share on other sites More sharing options...
ProSkill Posted July 2, 2018 Author Share Posted July 2, 2018 Now up to 195,259 indexed pages. Link to comment Share on other sites More sharing options...
Joel R Posted July 2, 2018 Share Posted July 2, 2018 17 hours ago, ProSkill said: Now up to 195,259 indexed pages. What's the change in your traffic or search rankings? Link to comment Share on other sites More sharing options...
AlexWebsites Posted July 3, 2018 Share Posted July 3, 2018 On 6/30/2018 at 9:13 AM, Sheffielder said: It's turning into quite a debate I'm really interested in how traffic has been affected by this sitemap thing being introduced To be honest if it radically increases traffic then I'll be straight there to try it too I would be inclined to as well. I’m very curious @ProSkill how your traffic is doing and if there is an increase in correlation to the increase in urls being indexed. Link to comment Share on other sites More sharing options...
Sheffielder Posted July 10, 2018 Share Posted July 10, 2018 On 7/2/2018 at 7:11 PM, Joel R said: What's the change in your traffic or search rankings? This is what I'm keen to know too Link to comment Share on other sites More sharing options...
AlexWebsites Posted July 20, 2018 Share Posted July 20, 2018 @ProSkill any update on your traffic vs increase in sitemap urls? Link to comment Share on other sites More sharing options...
AlexWebsites Posted August 7, 2018 Share Posted August 7, 2018 I guess no difference @ProSkill or is your traffic so high, you forgot about this topic...? Link to comment Share on other sites More sharing options...
kihon Posted August 8, 2018 Share Posted August 8, 2018 I've been following this thread for a while and it looks like I have an issue with my site's sitemap. I have 155,000 errors in google search console. The issue is the date formatting. How do I fix this? What are the right settings to have in the control panel under the sitemap settings? We are currently using the recommended settings which resulted in these errors so what should the settings be instead? The google indexing is 0 because of this error. Here is the error from webmaster tools: Parent tag: url Tag: lastmod Value: 1969-12-31T19:00:00-05:00 Errors Invalid date An invalid date was found. Please fix the date or formatting before resubmitting. 155,202 Link to comment Share on other sites More sharing options...
Stuart Silvester Posted August 8, 2018 Share Posted August 8, 2018 23 minutes ago, kihon said: I've been following this thread for a while and it looks like I have an issue with my site's sitemap. I have 155,000 errors in google search console. The issue is the date formatting. How do I fix this? What are the right settings to have in the control panel under the sitemap settings? We are currently using the recommended settings which resulted in these errors so what should the settings be instead? The google indexing is 0 because of this error. Here is the error from webmaster tools: Parent tag: url Tag: lastmod Value: 1969-12-31T19:00:00-05:00 Errors Invalid date An invalid date was found. Please fix the date or formatting before resubmitting. 155,202 Please submit a ticket, that does not seem right. Link to comment Share on other sites More sharing options...
kihon Posted August 8, 2018 Share Posted August 8, 2018 9 minutes ago, Stuart Silvester said: Please submit a ticket, that does not seem right. Thanks! Submitted a ticket Link to comment Share on other sites More sharing options...
ProSkill Posted October 15, 2018 Author Share Posted October 15, 2018 Update: I am now up to 277,077 indexed pages Link to comment Share on other sites More sharing options...
ProSkill Posted October 15, 2018 Author Share Posted October 15, 2018 According to Google Analytics there was a 25.98% increase in users over the same period last year. Link to comment Share on other sites More sharing options...
ProSkill Posted December 21, 2018 Author Share Posted December 21, 2018 Just a quick update. My site is now on page 1 for the primary keyword that we target. Link to comment Share on other sites More sharing options...
Recommended Posts
Archived
This topic is now archived and is closed to further replies.