Jump to content

How do you explain these sitemap/index results?


ProSkill

Recommended Posts

I have been saying for a long time that there seems to be an issue with the IPB sitemap generator. I have been told by one member in particular whose name I shall not mention that the problem is with my site and not the sitemap, despite the fact that many others are reporting similar results. Before I go on, I am running 4.3.3 on a dedicated server.  At one point  two years ago I had almost 400k indexed pages in Google, however, after the Fred update and 4.2 upgrade I lost close to 300k indexed pages, at the lowest point I was down to about 92k indexed pages. After doing the 4.3 update I ran the rebuild sitemap tool hoping that it would update the sitemap and submit new/moved/edited pages and that the sitemap would be larger, however, the opposite appears to have happened my sitemap went down from approximately 78k pages to about 11k pages and I still don't understand why. In any case, I saw in another thread someone mentioned that they were using a paid sitemap service and that they had seen good results so I decided to try it. Here are my results so far, I started this on June 1st. 

 

Here are my indexed pages, they were trending down, however, notice on or about 6/1 the number of indexed pages shot up significantly. On 5/13 I had only 92k pages indexed, by 6/10 that rocketed to more than 146k indexed pages. 

1408660637_ScreenShot06-14-18at10_26PM.thumb.PNG.50c7f19bf443e14b6e9a8b84d41ac556.PNG

 

Here are the details from my forum generated sitemap, notice how it only has 10,857 pages

 

985997794_ScreenShot06-14-18at10_28PM.PNG.75beef6ac69e6ddd34a178c6b58f213d.PNG

 

Here are the results from the paid sitemap service, 92,222 submitted pages, with 40k already indexed. 

1651143521_ScreenShot06-14-18at10.28PM001.thumb.PNG.f20b1944907c141a73d3baedd09ede50.PNG

Here are the pages processed by the sitemap service:

`1219072117_ScreenShot06-14-18at10_47PM.PNG.f8abefa652e4be724ad1e86ab0258380.PNG

 

 

Conclusion, the default IPB sitemap has issues. Secondly, to those who say that the sitemap doesn't matter, I would argue that these results show pretty clearly that the sitemap does matter.  What do you think? 

 

 

Edited by ProSkill
Link to comment
Share on other sites

7 hours ago, opentype said:

1077.gif

How do you explain these results? You seem to think that the sitemap doesn't matter even though I've just provided conclusive evidence that it does.  A larger more thorough sitemap, equals more indexed pages, which equals more pages showing up in SERPS, which equals more traffic. 

Link to comment
Share on other sites

Your paid site map service is including links to posts and other page links as separate urls most likely. 

The question is, has your organic traffic increased at all as your indexes pages have? If it has, there’s something to say about that then.

I have a site that used to be in vbulletin and vbseo. Had thousands of urls in my forum site map for topics and posts. Moved my site to IPS and SSL and my traffic dropped 1/2, even with redirects in place. This was last October. Now in May my traffic is the same as my highest point ever with this site and in June it’s surpassed that.

Most traffic is organic from Google. I cleaned up my site by having IPS and I belive my content is being weighted better. I’ll also add that I started to use IPS built in promotion tools with Facebook and Twitter regularly and my traffic started to get better. I think there might be a correlation.

It looks like you have way more urls in your 3rd party solution but the important part is whether it’s bringing you added traffic. If it is, I am for it and maybe IPS should add an option to include more urls for those that want to do this.

Link to comment
Share on other sites

1 hour ago, ProSkill said:

 even though I've just provided conclusive evidence that it does.  

No, you did not. Evidence would need to demonstrate the CONNECTION between your sitemap and your Google index stats. You don’t even provide a hint, let alone “evidence” for that, nor can you even explain why and how those things would be connected this way in the first place. You literally have nothing to support your case. You just repeat an unsupported ASSERTION. “My Google stats go up and do – so I blame the IPS sitemap”. That’s not “conclusive evidence”. It’s just an assertion. But I explained all that ten times already …

Link to comment
Share on other sites

4 hours ago, opentype said:

No, you did not. Evidence would need to demonstrate the CONNECTION between your sitemap and your Google index stats. You don’t even provide a hint, let alone “evidence” for that, nor can you even explain why and how those things would be connected this way in the first place. You literally have nothing to support your case. You just repeat an unsupported ASSERTION. “My Google stats go up and do – so I blame the IPS sitemap”. That’s not “conclusive evidence”. It’s just an assertion. But I explained all that ten times already …

I am starting to think that you're just here to troll. You never have anything constructive to say and you're unnecessarily combative. The evidence is clear, my indexed pages were going down for almost year, and my sitemap was much smaller after 4.3. Then I guess in your opinion, completely coincidentally, the week that I start using a paid sitemap service 81k plus pages are added to my sitemap, and magically my indexed pages in Google go up by almost 50k. I get that you're an IPS fan boy, I am too to some extent, but there is clearly an issue here. 

Edited by ProSkill
Link to comment
Share on other sites

4 hours ago, AlexWebsites said:

Your paid site map service is including links to posts and other page links as separate urls most likely. 

The question is, has your organic traffic increased at all as your indexes pages have? If it has, there’s something to say about that then.

I have a site that used to be in vbulletin and vbseo. Had thousands of urls in my forum site map for topics and posts. Moved my site to IPS and SSL and my traffic dropped 1/2, even with redirects in place. This was last October. Now in May my traffic is the same as my highest point ever with this site and in June it’s surpassed that.

Most traffic is organic from Google. I cleaned up my site by having IPS and I belive my content is being weighted better. I’ll also add that I started to use IPS built in promotion tools with Facebook and Twitter regularly and my traffic started to get better. I think there might be a correlation.

It looks like you have way more urls in your 3rd party solution but the important part is whether it’s bringing you added traffic. If it is, I am for it and maybe IPS should add an option to include more urls for those that want to do this.

1. It is not including links to posts and pages separately. My forum has has over 800k posts, and 8k+ topics.  

2. I don't know yet, I only started this two weeks ago. This is also the traditionally slow time of year for my site. I will report back later with more results. Though I am quite confident that having an additional 80k indexed pages in Google will bring in more traffic. 

3.  My traffic has been steady overall, it's actually up over this time last year. Regardless it's rather concerning when you see that your indexed pages are dropping by hundreds or thousands every few weeks.  My forum has been around for 5+ years and we have an active member base so we aren't as dependent on organic search engine traffic. 

4. Again, I don't know what the result will be. However, the more indexed pages the better, as it means that your site is showing up in more places in Google. It also more accurately represents the size of the site,  a 400k page site will have more authority than a 4k site (at least in theory)

Link to comment
Share on other sites

3 hours ago, ProSkill said:

1. It is not including links to posts and pages separately. My forum has has over 800k posts, and 8k+ topics.  

2. I don't know yet, I only started this two weeks ago. This is also the traditionally slow time of year for my site. I will report back later with more results. Though I am quite confident that having an additional 80k indexed pages in Google will bring in more traffic. 

3.  My traffic has been steady overall, it's actually up over this time last year. Regardless it's rather concerning when you see that your indexed pages are dropping by hundreds or thousands every few weeks.  My forum has been around for 5+ years and we have an active member base so we aren't as dependent on organic search engine traffic. 

4. Again, I don't know what the result will be. However, the more indexed pages the better, as it means that your site is showing up in more places in Google. It also more accurately represents the size of the site,  a 400k page site will have more authority than a 4k site (at least in theory)

I agree that it can't hurt to have more pages indexed. If you have 8k+ topics, where are you getting 92k+ submitted urls from? What type of additional urls. The first screenshot showed 10k+ urls and those are probably direct IPS links to pages, just curious if its not posts, what's driving the new 92k+ count of urls in the sitemap?

Link to comment
Share on other sites

Pro sitemaps.com is really expensive ? im using a local sitemap tool for a test. 

a1 sitemap generator

It taken me more then 1.5 days now to download all the links ? but I will add this created sitemap next to the ipb one. 

I have the same feeling about missing out visitors. Maby the ipb sitemap is the best official way. But if I don't try I'll never really know.

The thing is, I upgraded to 4.3 and more links are getting indexed then 4.2. so I think something's changed on the background. But there was no sitemap "issue" on 4.2. 

What are you having for payed service @ProSkill?

Link to comment
Share on other sites

On 6/30/2018 at 9:13 AM, Sheffielder said:

 

It's turning into quite a debate

I'm really interested in how traffic has been affected by this sitemap thing being introduced

To be honest if it radically increases traffic then I'll be straight there to try it too

I would be inclined to as well. I’m very curious @ProSkill how your traffic is doing and if there is an increase in correlation to the increase in urls being indexed.

Link to comment
Share on other sites

  • 2 weeks later...
  • 3 weeks later...

I've been following this thread for a while and it looks like I have an issue with my site's sitemap. I have 155,000 errors in google search console. The issue is the date formatting. How do I fix this? What are the right settings to have in the control panel under the sitemap settings? We are currently using the recommended settings which resulted in these errors so what should the settings be instead? The google indexing is 0 because of this error.

Here is the error from webmaster tools:

 
Parent tag: url
Tag: lastmod
Value: 1969-12-31T19:00:00-05:00

 

Errors    Invalid date    An invalid date was found. Please fix the date or formatting before resubmitting.    155,202

     
Edited by kihon
Link to comment
Share on other sites

23 minutes ago, kihon said:

I've been following this thread for a while and it looks like I have an issue with my site's sitemap. I have 155,000 errors in google search console. The issue is the date formatting. How do I fix this? What are the right settings to have in the control panel under the sitemap settings? We are currently using the recommended settings which resulted in these errors so what should the settings be instead? The google indexing is 0 because of this error.

Here is the error from webmaster tools:

 
Parent tag: url
Tag: lastmod
Value: 1969-12-31T19:00:00-05:00

 

Errors    Invalid date    An invalid date was found. Please fix the date or formatting before resubmitting.    155,202

     

Please submit a ticket, that does not seem right.

Link to comment
Share on other sites

  • 2 months later...
  • 2 months later...
  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...