Luke Posted June 14, 2007 Share Posted June 14, 2007 The thought of running a tool that cycles 500 posts at a time is a very scary thought to me... I have 600k+ posts on my forum, and that will take a very long time. It's ok for smaller boards, but for larger boards it's going to be a nightmare...I was thinking that it might be better to run a php script through shell instead of piping it through apache (or whatever web service is installed). You can also use set_time_limit to keep the script from timing out...Basically... You know how you can do manual sql upgrades with SSH? Would be nice if there was something similar for the rebuild tools... Link to comment Share on other sites More sharing options...
sparc Posted June 15, 2007 Share Posted June 15, 2007 it was running super slow for us. The second time i ran it, it blank paged on me.just one of those weird quirks. Luckily it doesn't have to be run often and you can eventually push it through.I think there should be a knowledgebase article for this that estimates how it should run and how fast. And more importantly, what to do if you are getting timeouts or slowdowns.Also, i would propose that the rebuild info gets logged in the ACP as to whether it gets completed. And if it doesn't get completed, it provides a link to restart at the exact spot it stopped. That way you don't lose any time. Link to comment Share on other sites More sharing options...
Logan Posted June 15, 2007 Share Posted June 15, 2007 I agree also, I'd love to see rebuild tool info get logged... and if it doesn't finish, a link to start from where it left off, ex: "Rebuilt Post Content, at 212,500 posts - 550,500 left to go... click here to finish, or click here to cancel (delete saved spot)". Although if it fails you can just refresh your browser yes, and relogin if need be... but I like the idea of logging and would be easier for those that didn't know you could just refresh. Link to comment Share on other sites More sharing options...
rm5b.2bs Posted June 15, 2007 Share Posted June 15, 2007 I also get blank pages, all I do is refresh the page to fix it up.But yeh, doing it via SSH would be much better. Link to comment Share on other sites More sharing options...
Luke Posted June 15, 2007 Share Posted June 15, 2007 I would like to upgrade to 2.3.0. I really would... But I cannot and will not upgrade until there is a solid and fool proof way of updating posts on a large board. I know you can't simply run SQL commands in SSH for this type of process, but you can run a php script directly in ssh and bypass the web server.... All you'd need to do to keep the script running is use set_time_limit to extend the 30 second time limit. You can do this by setting it to a very large number, or set do "set_time_limit(30)" on every loop.When called, set_time_limit() restarts the timeout counter from zero. In other words, if the timeout is the default 30 seconds, and 25 seconds into script execution a call such as set_time_limit(20) is made, the script will run for a total of 45 seconds before timing out. Link to comment Share on other sites More sharing options...
Breed Posted June 15, 2007 Share Posted June 15, 2007 I had to increase the timeout of my DB server in order to complete the rebuilding process without getting errors. This is one of the most stupid ideas IPS ever had and I'm still pissed about this crap. :angry: Link to comment Share on other sites More sharing options...
Dll Posted June 15, 2007 Share Posted June 15, 2007 You don't need to update all the posts unless you have custom bb code as far as I'm aware. The message about updating all posts has been left on the installer since the initial change to 2.2. Link to comment Share on other sites More sharing options...
mortenz Posted June 15, 2007 Share Posted June 15, 2007 Last time I upgraded, it took more than 24 hours of hard work for the server to upgrade the db. We have +800.000 posts. Any upgrade that requires serious db-work is a real pain and it would be very helpful if more information and tools where provided to accomodate this. Link to comment Share on other sites More sharing options...
Keith J. Kacin Posted June 15, 2007 Share Posted June 15, 2007 A shell script has been written, and is being tested to ensure it works on large boards.It should be released shortly. :) Link to comment Share on other sites More sharing options...
rm5b.2bs Posted June 15, 2007 Share Posted June 15, 2007 Sweet.Thanks Keith. :) Link to comment Share on other sites More sharing options...
Management Matt Posted June 15, 2007 Management Share Posted June 15, 2007 The shell tool is running on this board right now.I've already used it to rebuild calendar events, PMs, announcements and signatures. It's half way through the posts now. It's pretty fast. It did 3600 posts in 20 seconds on my test board. Link to comment Share on other sites More sharing options...
Management Matt Posted June 15, 2007 Management Share Posted June 15, 2007 I had to increase the timeout of my DB server in order to complete the rebuilding process without getting errors. This is one of the most stupid ideas IPS ever had and I'm still pissed about this crap. :angry:Oh, we've had far sillier ideas, trust me. Link to comment Share on other sites More sharing options...
Nimdock Posted June 15, 2007 Share Posted June 15, 2007 Oh, we've had far sillier ideas, trust me.Hehehehe, that was good.I guess I'll wait for the shell tool before upgrading. :) Link to comment Share on other sites More sharing options...
Logan Posted June 15, 2007 Share Posted June 15, 2007 I had no idea a shell tool to rebuild content was in the works, that's awesome news! Can't wait for the release, that's going to make upgrading so much easier :) Link to comment Share on other sites More sharing options...
rm5b.2bs Posted June 15, 2007 Share Posted June 15, 2007 Why do we have to rebuild our post content after upgrading? :huh: Link to comment Share on other sites More sharing options...
Breed Posted June 15, 2007 Share Posted June 15, 2007 Oh, we've had far sillier ideas, trust me.Haha I believe you on this one. :PA shell script is a good idea, but now it's too late for me since I rebuilded everything eyesterday. Maybe the next time. :ph34r: Link to comment Share on other sites More sharing options...
Management Matt Posted June 15, 2007 Management Share Posted June 15, 2007 To improve efficiency we now parse once then save the data to the database.Previously, we parsed the post on every page view.In a future version (and please don't quote me on this) we're considering abstracting the BBcode handling to allow for different methods (cached, not-cached, partially cached, etc). Link to comment Share on other sites More sharing options...
John__ Posted June 15, 2007 Share Posted June 15, 2007 Looking forward to this toolWe have just over 2 million posts Link to comment Share on other sites More sharing options...
Ziv Grosu Posted June 15, 2007 Share Posted June 15, 2007 To improve efficiency we now parse once then save the data to the database.Previously, we parsed the post on every page view.In a future version (and please don't quote me on this) we're considering abstracting the BBcode handling to allow for different methods (cached, not-cached, partially cached, etc).I advise you to use the new methods :) The new software of BBCODE would be ingenious. I think when the 3.0.0 release, people will be crazy. :)Unfortunately , I thought there will be additional things to the newest board released. Link to comment Share on other sites More sharing options...
bfarber Posted June 15, 2007 Share Posted June 15, 2007 Unfortunately , I thought there will be additional things to the newest board released.We had publicly stated several times prior to the release that there were nearly no new features, the entire release was a performance improvement release. :) Link to comment Share on other sites More sharing options...
AtariAge Posted June 16, 2007 Share Posted June 16, 2007 I'm also looking forward to a shell script as I have over 1.3 million posts on my forum and it takes forever to rebuild the posts (something I had to do for the last upgrade). Thanks for listening and writing one.. ..Al Link to comment Share on other sites More sharing options...
kafloo Posted June 18, 2007 Share Posted June 18, 2007 when can we download this script pleasewe have 1.5M posts Link to comment Share on other sites More sharing options...
Management Matt Posted June 18, 2007 Management Share Posted June 18, 2007 I'm going to add in a little last minute feature so you can choose to rebuild posts from the last X days.This will be useful if you've just added a BBCode or you've just changed a BBCode and only want current posts to be updated. Link to comment Share on other sites More sharing options...
kafloo Posted June 18, 2007 Share Posted June 18, 2007 hope you could release it soonI'm being pressured by my members to fix the broken bbcode, snapbacks, etc. on old posts:)Thanks in advance Link to comment Share on other sites More sharing options...
Management Matt Posted June 18, 2007 Management Share Posted June 18, 2007 We'll likely make an 'official' release later today. In the mean time feel free to give it a go.It's been used here successfully.(Zip removed, see further down) Link to comment Share on other sites More sharing options...
Recommended Posts
Archived
This topic is now archived and is closed to further replies.