Invision Community 4: SEO, prepare for v5 and dormant account notifications By Matt November 11, 2024
Luke Posted June 14, 2007 Posted June 14, 2007 The thought of running a tool that cycles 500 posts at a time is a very scary thought to me... I have 600k+ posts on my forum, and that will take a very long time. It's ok for smaller boards, but for larger boards it's going to be a nightmare...I was thinking that it might be better to run a php script through shell instead of piping it through apache (or whatever web service is installed). You can also use set_time_limit to keep the script from timing out...Basically... You know how you can do manual sql upgrades with SSH? Would be nice if there was something similar for the rebuild tools...
sparc Posted June 15, 2007 Posted June 15, 2007 it was running super slow for us. The second time i ran it, it blank paged on me.just one of those weird quirks. Luckily it doesn't have to be run often and you can eventually push it through.I think there should be a knowledgebase article for this that estimates how it should run and how fast. And more importantly, what to do if you are getting timeouts or slowdowns.Also, i would propose that the rebuild info gets logged in the ACP as to whether it gets completed. And if it doesn't get completed, it provides a link to restart at the exact spot it stopped. That way you don't lose any time.
Logan Posted June 15, 2007 Posted June 15, 2007 I agree also, I'd love to see rebuild tool info get logged... and if it doesn't finish, a link to start from where it left off, ex: "Rebuilt Post Content, at 212,500 posts - 550,500 left to go... click here to finish, or click here to cancel (delete saved spot)". Although if it fails you can just refresh your browser yes, and relogin if need be... but I like the idea of logging and would be easier for those that didn't know you could just refresh.
rm5b.2bs Posted June 15, 2007 Posted June 15, 2007 I also get blank pages, all I do is refresh the page to fix it up.But yeh, doing it via SSH would be much better.
Luke Posted June 15, 2007 Posted June 15, 2007 I would like to upgrade to 2.3.0. I really would... But I cannot and will not upgrade until there is a solid and fool proof way of updating posts on a large board. I know you can't simply run SQL commands in SSH for this type of process, but you can run a php script directly in ssh and bypass the web server.... All you'd need to do to keep the script running is use set_time_limit to extend the 30 second time limit. You can do this by setting it to a very large number, or set do "set_time_limit(30)" on every loop.When called, set_time_limit() restarts the timeout counter from zero. In other words, if the timeout is the default 30 seconds, and 25 seconds into script execution a call such as set_time_limit(20) is made, the script will run for a total of 45 seconds before timing out.
Breed Posted June 15, 2007 Posted June 15, 2007 I had to increase the timeout of my DB server in order to complete the rebuilding process without getting errors. This is one of the most stupid ideas IPS ever had and I'm still pissed about this crap. :angry:
Dll Posted June 15, 2007 Posted June 15, 2007 You don't need to update all the posts unless you have custom bb code as far as I'm aware. The message about updating all posts has been left on the installer since the initial change to 2.2.
mortenz Posted June 15, 2007 Posted June 15, 2007 Last time I upgraded, it took more than 24 hours of hard work for the server to upgrade the db. We have +800.000 posts. Any upgrade that requires serious db-work is a real pain and it would be very helpful if more information and tools where provided to accomodate this.
Keith J. Kacin Posted June 15, 2007 Posted June 15, 2007 A shell script has been written, and is being tested to ensure it works on large boards.It should be released shortly. :)
Management Matt Posted June 15, 2007 Management Posted June 15, 2007 The shell tool is running on this board right now.I've already used it to rebuild calendar events, PMs, announcements and signatures. It's half way through the posts now. It's pretty fast. It did 3600 posts in 20 seconds on my test board.
Management Matt Posted June 15, 2007 Management Posted June 15, 2007 I had to increase the timeout of my DB server in order to complete the rebuilding process without getting errors. This is one of the most stupid ideas IPS ever had and I'm still pissed about this crap. :angry:Oh, we've had far sillier ideas, trust me.
Nimdock Posted June 15, 2007 Posted June 15, 2007 Oh, we've had far sillier ideas, trust me.Hehehehe, that was good.I guess I'll wait for the shell tool before upgrading. :)
Logan Posted June 15, 2007 Posted June 15, 2007 I had no idea a shell tool to rebuild content was in the works, that's awesome news! Can't wait for the release, that's going to make upgrading so much easier :)
rm5b.2bs Posted June 15, 2007 Posted June 15, 2007 Why do we have to rebuild our post content after upgrading? :huh:
Breed Posted June 15, 2007 Posted June 15, 2007 Oh, we've had far sillier ideas, trust me.Haha I believe you on this one. :PA shell script is a good idea, but now it's too late for me since I rebuilded everything eyesterday. Maybe the next time. :ph34r:
Management Matt Posted June 15, 2007 Management Posted June 15, 2007 To improve efficiency we now parse once then save the data to the database.Previously, we parsed the post on every page view.In a future version (and please don't quote me on this) we're considering abstracting the BBcode handling to allow for different methods (cached, not-cached, partially cached, etc).
John__ Posted June 15, 2007 Posted June 15, 2007 Looking forward to this toolWe have just over 2 million posts
Ziv Grosu Posted June 15, 2007 Posted June 15, 2007 To improve efficiency we now parse once then save the data to the database.Previously, we parsed the post on every page view.In a future version (and please don't quote me on this) we're considering abstracting the BBcode handling to allow for different methods (cached, not-cached, partially cached, etc).I advise you to use the new methods :) The new software of BBCODE would be ingenious. I think when the 3.0.0 release, people will be crazy. :)Unfortunately , I thought there will be additional things to the newest board released.
bfarber Posted June 15, 2007 Posted June 15, 2007 Unfortunately , I thought there will be additional things to the newest board released.We had publicly stated several times prior to the release that there were nearly no new features, the entire release was a performance improvement release. :)
AtariAge Posted June 16, 2007 Posted June 16, 2007 I'm also looking forward to a shell script as I have over 1.3 million posts on my forum and it takes forever to rebuild the posts (something I had to do for the last upgrade). Thanks for listening and writing one.. ..Al
kafloo Posted June 18, 2007 Posted June 18, 2007 when can we download this script pleasewe have 1.5M posts
Management Matt Posted June 18, 2007 Management Posted June 18, 2007 I'm going to add in a little last minute feature so you can choose to rebuild posts from the last X days.This will be useful if you've just added a BBCode or you've just changed a BBCode and only want current posts to be updated.
kafloo Posted June 18, 2007 Posted June 18, 2007 hope you could release it soonI'm being pressured by my members to fix the broken bbcode, snapbacks, etc. on old posts:)Thanks in advance
Management Matt Posted June 18, 2007 Management Posted June 18, 2007 We'll likely make an 'official' release later today. In the mean time feel free to give it a go.It's been used here successfully.(Zip removed, see further down)
Recommended Posts
Archived
This topic is now archived and is closed to further replies.