Invision Community 4: SEO, prepare for v5 and dormant account notifications By Matt Monday at 02:04 PM
stoo2000 Posted September 21, 2009 Posted September 21, 2009 File Name: XML Sitemap Generator Task v1.0.5File Submitter: stoo2000File Submitted: 04 Aug 2009File Updated: 23 Jul 2010File Category: Miscellaneous XML FilesIP.Board 3 Sitemap Generator Task v1.0.5About the Sitemap Generator Task This task was originally updated to accommodate my own forums, due to the demand and the fact that no one has made a new task to address the demand I have decided to enhance this task and release it for use. This task uses built in IP.Board methods for generating FURLS, this means that this task is compatible with custom FURL templates. The task is Improved and built on Google Sitemap Generator for IPB 2.2 by CheetahShrk.Version 1.0.5Fixes Version 1.0.4Enhancements [*]Corrected minor typo in gzip process. Version 1.0.3Fixes [*]Added option to ping ASK! with sitemaps. Version 1.0.2Fixes [*]Localisation bug introduced in 1.0.2 - affects locales that use , as a number seperator. Version 1.0.1Fixes [*]Removed use of old class_XML.php to use the new IP.Board 3, classXML.php method. Version 1Fixes [*]Fixed an issue where URLS with no SEO Title generated would put old style URLS in sitemap, and not use FURLS. [*]PING function altered to work properly. [*]Added option to choose if GZIP should be used [*]Upgraded to work with IP.Board 3.0.x Enhancements [*]Now uses the database class correctly – uses prefixes correctly. [*]Easier to use settings, with forum multi-select [*]Supports IP.Board 3.0.x FURL’s & legacy urls IMPORTANT!There is a bug in IPB 3.0.4 that will prevent FURLS being generated for your sitemap, as such any sitemaps generated will include legacy urls, if you would like to continue to use FURLS you need to run the task manually, then disable it. This bug is fixed in 3.0.5.Click here to download this file [*]Bing Ping added
Mercury_ Posted October 5, 2009 Posted October 5, 2009 Hello. Thank you for this great modification. I've got it installed, but It generates a timeout and locks the task. My forum has around 20.000 topics. Is it possible to improve the performance of this mod? Or a better way in my opinion: - Add a possibility to create multiple sitemaps. Lets say .. I want an sitemap(X) for all topics in the forum 1, 2 and 3. This task runs at 1am. Then I set up an other sitemap(Y) for all topics in forums 4, 5 and 6 and this task runs at 1.20am ... and so on. You can add easily multiple sitemaps in Google. Or add all created sitemaps to the indexsitemap.xml. In this way you can split al the load. One more question: Is it possible for you to add all pages in a topic to the sitemap? Let me know what you are thinking about my opinion. :) Thanks.
stoo2000+ Posted October 9, 2009 Posted October 9, 2009 Hi Mercury. I don't seem to have access to this thread from my usual acccount... No matter how many sitemaps you create you are still working with that data in one request, so it will still timeout. Could you possibly send me an excerpt of your error_log detailing the problem ? The solution may be to extend the max execution time whilst processing the data, although I would need to look into how that affects shared hosts. As you mention splitting it into multiple tasks would be an idea but an avenue I'd like to avoid at the moment.
Mercury_ Posted October 12, 2009 Posted October 12, 2009 Hi. If I select all forums, I'll get a 504-error after 30 seconds. There is no sql-error or a log entry in the Task-Manager. It simply times out. Now what I did ... instead of selecting all forums I selected only 3 of them. Then I startet to run the task and it successfully finished the sitemap in less than 5 seconds. That is all good. But how can I bring all the other forum in? That is where I pointet at. Give the possibility in your mod to create more tasks (propably only by import the xml with some ID-changes). Now you have multiple tasksk which can run at diffrent times (leave 5 minutes time between). Sitemap-Task 1 ... includes forums 1-5 and creates topicsitemap1.xml Sitemap-Task 2 ... includes forums 6-11 and creates topicsitemap2.xml Sitemap-Task 3 ... includes forums 11-15 and creates topicsitemap2.xml And so on ... By this sheme to split the work load you prevent a timeout. I don't think that this is a lot of work. You just need to use some more variables.
stoo2000+ Posted October 18, 2009 Posted October 18, 2009 It's really something that needs some looking into really, if processing a large amount of data is causing a time out, and 50,000 item limit per forum might mean that it still times out even if it is split up. Give this a shot. Find (Line 274): while ( $r = $this->DB->fetch() ) { $content = array(); replace with: while ( $r = $this->DB->fetch() ) { // lets see if this solves some timing out issues. set_time_limit(30); $content = array();
stoo2000 Posted October 26, 2009 Author Posted October 26, 2009 I'm not so sure that is related to the sitemap generator, Have a quick search for 504 error online, it seems it could be an issue with an upstream provider i.e your web hosting.
InvisionHQ Posted November 4, 2009 Posted November 4, 2009 Many warnings on the topicsitemap, all related to "Tag: priority"
stoo2000 Posted November 4, 2009 Author Posted November 4, 2009 [quote name='InvisionHQ' date='04 November 2009 - 08:44 AM' timestamp='1257324255' post='1875351'] Many warnings on the topicsitemap, all related to "Tag: priority" Could you PM me some examples please, and link me to your topicsitemap file, I've had this version live on one of my own forums for a few days, without any Google reported issues.
stoo2000 Posted November 4, 2009 Author Posted November 4, 2009 Looks like it is a Bug/Feature in PHP that replaces . with , in integers depending on your locale, Will correct it and send you a copy to test.
dyelton Posted November 5, 2009 Posted November 5, 2009 Community SEO (http://www.communityseo.com) handles sitemaps just fine, but I had to stop using their software as they were giving me extremely high server loads (not related to sitemaps). I have a very large forum (1+ million posts) and this sitemap task also times out for me. I'm using version 1.0.3.
stoo2000 Posted November 5, 2009 Author Posted November 5, 2009 [quote name='dyelton' date='05 November 2009 - 02:17 PM' timestamp='1257430664' post='1875858'] Community SEO (http://www.communityseo.com) handles sitemaps just fine, but I had to stop using their software as they were giving me extremely high server loads (not related to sitemaps). I have a very large forum (1+ million posts) and this sitemap task also times out for me. I'm using version 1.0.3. That's one reason why it's also limited to 50,000 items, I do want to look in the future at indexing over time, and not at once. You could try adding set_time_limit(30) into the topic loop, that may prevent the time out, but it will still take a small while to construct the XML file.
AlexJ Posted November 11, 2009 Posted November 11, 2009 In the docs it says CHMOD the above files to 0777. Does it also applies to task sitemap.php? More or less if I run task it only generates 2 or 3 links thats it.
stoo2000 Posted November 11, 2009 Author Posted November 11, 2009 You only need to chmod the xml/xml.gz files. Make sure you select the forums you want to put in the sitemap (in the sitemap task settings)
AlexJ Posted November 11, 2009 Posted November 11, 2009 Thanks got it working. :) For some reason it didn't worked for first time. I did run task again back and it worked well. We have only 3000 topics so it worked out pretty fast. Just to double check indexsitemap has only 2 links in it. forumsitemap.xml.gz and for topicsitemap.xml.gz. Is that how it should be? Is their anyway to show sitemap on forums as a link?
stoo2000 Posted November 11, 2009 Author Posted November 11, 2009 Yep the indexsitemap.xml.gz is just a location to let search engines know about the different sitemaps that are generated. The XML Sitemaps are intended for Search engines, you won't get any benefits by linking to it, since it informs search engines when it has been generated :)
AlexJ Posted November 11, 2009 Posted November 11, 2009 Cool thanks. And one more last question. Does it makes huge difference if my website uses www.domain.com extension and forum uses .domain.com extension? Does it effects page ranking? or Google analytics tracking?
stoo2000 Posted November 11, 2009 Author Posted November 11, 2009 [quote name='AlexJ' date='11 November 2009 - 11:51 PM' timestamp='1257983516' post='1878343'] Cool thanks. And one more last question. Does it makes huge difference if my website uses www.domain.com extension and forum uses .domain.com extension? Does it effects page ranking? or Google analytics tracking? Nope that doesn't matter at all, obviously you would want two Google Analytics accounts so you can track forum and website traffic.
AlexJ Posted November 11, 2009 Posted November 11, 2009 Oh i was using same. :( I thought all sub domains and folders are included in one main profile of Google analytics.
TheMac Posted November 19, 2009 Posted November 19, 2009 Hello, I have this: Sitemap Generator v1.0.3 (LOCKED) But when I unlock they lock other time...
stoo2000+ Posted November 19, 2009 Posted November 19, 2009 That means that there was a problem running the task, you should unlock the task and run it manually. If you see any errors please send them to me in a PM.
Enkidu Posted November 26, 2009 Posted November 26, 2009 I'm getting a pile of re-direct errors and they all have this format: Now where did this showtopic come from? I'm using FURL and never ever switched back to showtopic :blink: where did this come from?http://tabee3i.com/index.php?showtopic=729
stoo2000 Posted November 26, 2009 Author Posted November 26, 2009 [quote name='Enkidu' date='26 November 2009 - 12:47 AM' timestamp='1259196460' post='1882513'] I'm getting a pile of re-direct errors and they all have this format: Now where did this showtopic come from? I'm using FURL and never ever switched back to showtopic :blink: where did this come from? The task uses built in IP.B functions to come up with the FURL, do you have any other SEO mods installed ?http://tabee3i.com/index.php?showtopic=729
Recommended Posts
Archived
This topic is now archived and is closed to further replies.