Requesting Google Sitemap

[Maxxius]

I believe a lot of people over time have asked IPS to make sitemap creation a core feature. Yet I reiterate this once more.

As you know CSEO was rendered useless, CSEO itself realized that and closed its doors. It seems that IPS really picked up on SEO but one thing is left unattended - sitemap creation.

I know that at least 3 mods exist to serve that function to some extent.

Why IPS can't create a sitemap generator so it would have a complete package for SEO. So the end-user wouldn't have to do anything in regards to SEO stuff.

[tekguru]

Seconded a needed part of the system that is missing!

[Axel Wers]

[Maxxius]

Its a "latest topics and stuff" sitemap generator - not all the topics and stuff are indexed.

[Ichirō]

I agree , why only do half a job ? As ips have now sown up the seo , why not go the last yard and make sitemaps a core feature. Is there a reason why it hasnt been done or is it in the pipeline for future releases?

[Axel Wers]

Its a "latest topics and stuff" sitemap generator - not all the topics and stuff are indexed.

All topics ARE generated depending on your private settings. Sitemap generator is still in development, so in near future will be generated gallery albums, blog entries etc.

[.Ian]

All topics ARE generated depending on your private settings. Sitemap generator is still in development, so in near future will be generated gallery albums, blog entries etc.

I think it only covers the last 12,000 topics, so certainly not all on larger forums.

I think the OP is saying that it should be included as default - with a third party mod, even by a staff member there is no guarantee it will be extended to gallery, blogs etc., or released as an official part of the IPS suite. I hope it will, but unlikely until after the gallery has an overhaul.

After all there is also an arcade and classified add-ons run by staff, but these are unlikely to be official, so the fear of the OP is that the sitemap will be the same.

[blair]

None of the current IPB sitemaps to do a proper job of marking change frequency, or more importantly priority. They mark them all the SAME. Search engines see this as an error, resulting in a lower "trust" factor for the sitemap, and slower indexing. Priority and change frequency should be dynamic, based on last mod, topic views, last view, etc. It can't be hard to use existing data to create a simple algorithm.

CSEO's implementation wasn't perfect, but it was pretty good (and the code is open sourced). vBSEO's sitemap does an excellent job with vB forums. Not to mention a complete sitemap implementation should have not just an XML, but an HTML sitemap as well.

If they taught SEO in college, a proper sitemap may not be a 101 course, but it wouldn't be more than 201. Basic SEO. Too bad IPS doesn't appreciate the value. vB4 included XML sitemaps.

[stoo2000]

None of the current IPB sitemaps to do a proper job of marking change frequency, or more importantly priority. They mark them all the SAME. Search engines see this as an error, resulting in a lower "trust" factor for the sitemap, and slower indexing. Priority and change frequency should be dynamic, based on last mod, topic views, last view, etc. It can't be hard to use existing data to create a simple algorithm.

That's not exactly true now is it.

I know for a fact that the task I did generates the change frequency by looking at how long ago the last post to the topic was, priority is increased if said topic is pinned.

[blair]

My apologies, I thought I had tested all 3 sitemaps, but I did not test yours. As I recall it was because it said it supported a max of 50K topics, which is more like 20K in practice, and I have 10X that amount. Having already been dissapointed by Dan's I skipped it.

However, pinned is not a very good indicator of priority. If you look at vbseo's they define it to the hundreths (x.xx) per topic.

[Shadow82x]

I agree, this should definitely be in 3.2.

[Dave Legg]

I would also like to see this. Would be great to get a staff members point of view on this.

[bonesoul]

+1, ipb should feature a sitemap by default.

[majdi]

there is stoo's app which works perfectly

[Ikadon]

+1

[Guest]

The next release of Sitemap Generator is coming, and will be released with Invision Power Services as the author. There is, admittedly, a lot of work to be done on it before it could be a "core" feature of IP.Board or included by default, but it's as official as you're going to get at the moment.

Some things I'll be changing in upcoming releases though:

• Blog, Gallery, Nexus and Downloads will be supported when their next versions are released.
• I'll be adding support for generating multiple sitemaps. Each app will be able to generate as many as they need to. This will mean we can support more like 50,000 topics rather than the current 25,000 limit.
• Basic support for priorities and change frequencies. Followed later by more in-depth support.
  

Feel free to leave feedback about it either here or in the topic, and I'll try to include any requests that are feasible. Ideas on how you'd like to see priorities generated would be great.

[Guest]

Oh, I'd also like to follow up on one particular point in this topic, the idea of having all topics in the sitemap. There's a couple of reasons we're unlikely to ever expand it to do this:

1. The main reason I chose to write this sitemap generator in the first place is that existing solutions either didn't check permissions properly, or they were not developed with big boards in mind. The latter part is what matters here, as anything we (IPS) release has to take into account our entire customer base. On some of our customer's forums, if we generated all topics it would take literally hours to run the sitemap generator, there'd be hundreds of files and they would take up gigabytes of space and bandwidth.

2. More importantly than the above, I truly do not believe it to be necessary to include all of your content in a sitemap that is generated and sent to search engines more than once a day. I don't know any forums that generate more than 25,000 new topics a week, which means that you can expect your new content to be served to the search engines for at least that long. This gives them ample time to index the new content, and if they have not done so by then, they probably never will. Remember also that we ping the search engines with your sitemap to let them know that new content is available, so it's not like there is a discovery period to worry about.


I'll expand it to 50,000 as soon as the new generation method is in place, but that's likely it for a while at least.

[stoo2000]

After speaking with Dan yesterday and exchanging a couple of ideas for a sitemap product. I have decided that my Sitemap Task will receive no further updates.

[Maxxius]

I got a question and an idea:

#1

The next release of Sitemap Generator is coming, and will be released with Invision Power Services as the author

Is this mod going to be released as a mod like IP.Tracker/IP.Shoutbox community type of project or a simple regular mod.

#2 - you said that it takes a lot of resources to remake a sitemap, etc. But what IF

- Lets take IP.Board forum. Lets make a sitemap which indexes topics on one sitemap.xml file BY TOPIC ID from 1-10000-20000, another sitemap file sitemap2.xml - with topics which IDs are sitemap3.xml and so on. Sitemaps for each topic ID range are set to be run at different times. Once in 2 weeks or so.

And when it comes to generating the sitemap to the current range of topic IDs.

For example you have 50,000 topics on the forum. The sitemap.xml file for topic ids for range 50,000 to 60,000 is rebuilt once 24 hours at night.

I know that I'm not that good at describing this right now. But perhaps you understood what I'm trying do say here.

[Guest]

Is this mod going to be released as a mod like IP.Tracker/IP.Shoutbox community type of project or a simple regular mod.

It'll just be a regular modification, still maintained by myself and not shipped with IP.Board as standard. The only difference is that as I now work at IPS, we thought it would be best to release it that way. :)

[.Nuno.]

Please add a mobile sitemap too.

[Maxxius]

Alright Dan, what about #2 suggestion?

[Guest]

Please add a mobile sitemap too.

I'm not really sure what the point would be, the entire sitemap would need to be duplicated as everything it supports also supports the mobile skin.

I'd leave it up to Google to hit the site with both mobile and non-mobile crawlers and detect the difference itself.

[Guest]

Alright Dan, what about #2 suggestion?

As I said in the post about this, anything is possible, but in my opinion there is simply no point. Additionally, there are greater concerns than whether or not it can be generated, such as disk space and bandwidth issues.

[Guest]

CSEO's implementation wasn't perfect, but it was pretty good (and the code is open sourced). vBSEO's sitemap does an excellent job with vB forums. Not to mention a complete sitemap implementation should have not just an XML, but an HTML sitemap as well.

If they taught SEO in college, a proper sitemap may not be a 101 course, but it wouldn't be more than 201. Basic SEO. Too bad IPS doesn't appreciate the value. vB4 included XML sitemaps.

IPS does appreciate the value of SEO. I've been involved in making sure IP.Board is good for SEO since long before I was employed here, and continue to be now, my previous two jobs have been in SEO agencies.

As for HTML sitemaps, I completely disagree. That's not an SEO feature, at best it constitutes a usability feature, but that's pushing it considering the sheer quantity of links it would have in it.