Management Matt Posted April 9, 2015 Management Posted April 9, 2015 The background processes are designed to do just that - run in the background. There should be no need to leave your site offline until they complete.
Nathan Explosion Posted April 9, 2015 Posted April 9, 2015 I'll be choosing to keep it offline (thankfully, my busy periods are tied into the UK football season, so a week offline after end of season isn't going to be an issue) so that I don't have to constantly field questions as to why posts look strange. However, is it possible for the number of posts that are rebuilt each time to be increased? If yes, then great....if no, then fine (by me) Would just like to know either way.
SeaTea Posted April 9, 2015 Posted April 9, 2015 The background processes are designed to do just that - run in the background. There should be no need to leave your site offline until they complete.What will be unavailable when these processes are not yet finished ? Just statistics or also topics/messages ? And if so, will the newer topics handled first ?
Tanja Posted April 9, 2015 Posted April 9, 2015 Since members will be complaining about their posts, signatures, about me fields in profiles and personal massages not being formatted as they intended it to be and some of them might get to changing it in order to be presented in a way they want it. From our perspective, this is unnecessary stress for both us running the community and the members who have no idea why that's happening. So we'll be offline up until that gets rebuilt.In our case around 640.000 posts get rebuild in about 24 hours (we run it manually right away), it's not a big deal, but does it really have to take that long?
bfarber Posted April 9, 2015 Posted April 9, 2015 Nothing is "unavailable", content is just not formatted properly.It takes a long time because of all the overhead of actually performing the content rebuilding and parsing it. Rebuilding content is an intensive process, which is why we try to avoid doing it if possible (going from 3.4 to 4.0 is a big jump and requires it)
Tanja Posted April 9, 2015 Posted April 9, 2015 Just to be clear, we'd rather it took so long and everything gets done properly (and it does ) then it not rebuilding correctly. We look at it as a one time fee that is OK
Ocean West Posted April 9, 2015 Posted April 9, 2015 How many bugs have been filed because the background process was working and they cleared up after it was complete.It's not benign as users can't search until its complete. Admin's can't really get into redecorating or doing any sort of spring cleaning while they are operating. (deleted forums or other ACP settings)adding or removing apps / add ons - your mileage may vary - i have had some refuse to install or not function as intended because background processes were not completeprobably not good idea to attempt to install sphinx or other server side settings ( caching - or moving of files to different locations ) until the system is complete.
Management Lindy Posted April 9, 2015 Management Posted April 9, 2015 We are going to build a degree of intelligence to the rebuild routine that will gauge how much your server can handle based on how fast it can run a cycle. This will be completed next week.
Izaya Orihara Posted April 9, 2015 Posted April 9, 2015 A degree of intelligence? No clue what that means but sounds good to me
simonle Posted April 9, 2015 Posted April 9, 2015 We are going to build a degree of intelligence to the rebuild routine that will gauge how much your server can handle based on how fast it can run a cycle. This will be completed next week. Great, I'll wait till then to upgrade my forums.
Nathan Explosion Posted April 10, 2015 Posted April 10, 2015 2.5 million (and counting) posts to rebuild here - on my last test upgrade, it took 8 days on my dedicated server. That question I asked above, if answered at any time, will help me further with this one.So some good news - the above test was performed with RC6. I'm 1 hour into the background tasks on a test upgrade (to 4.0.0) of my forum now, and the post rebuild is already 4.5% through....so I'm already looking at an improvement on the 8 days.
Ocean West Posted April 10, 2015 Posted April 10, 2015 I upgraded last night and hit the RUN NOW button well after midnight - i woke up at about 8 am to watch all background processes compete.
AutoItScript Posted April 11, 2015 Posted April 11, 2015 I don't think people should get their hopes up about any major time improvements here - whatever tweaks are made. It's an intensive process. I modded my board to update 500 posts at a time (which I imagine will be way higher than any official tweaks were mentioned earlier.). It really didn't change the time at all - a few seconds here and there. In 500 post chunks the rebuild speed was 100k posts per hour.When I upgrade for real and given that the best rebuild time for my board is projected to be 12 hours I'm just going to reopen it straight away. I'll pre-warn members the day before about what is going to happen.What would help me though is if the rebuild process could be made to go backwards. This would mean that the active posts are converted first so that most people wouldn't notice there is an issue unless they went hunting for archive threads.
Hexsplosions Posted April 11, 2015 Posted April 11, 2015 A degree of intelligence? No clue what that means but sounds good to me Don't do it @Lindy, you'll kill us all!
Nathan Explosion Posted April 11, 2015 Posted April 11, 2015 What would help me though is if the rebuild process could be made to go backwards. This would mean that the active posts are converted first so that most people wouldn't notice there is an issue unless they went hunting for archive threads. Funny you should mention that, as the same thing occurred to me earlier - have made a file change on my test site to see if it has the effect I think it will. EDIT: Yeah...that didn't have the effect I thought...oops.
AutoItScript Posted April 11, 2015 Posted April 11, 2015 EDIT: Yeah...that didn't have the effect I thought...oops. Ha
JarredB Posted April 11, 2015 Posted April 11, 2015 Don't do it @Lindy, you'll kill us all! All your forum are belong to Lindy.His expression is perfect.... too perfect...
VoronAlex Posted April 12, 2015 Posted April 12, 2015 I found a way around to resolve the stack in rebuilding background tasks without turning on the cron in ACP:in the dashboard below all background processes there is a link"Alternatively, you can manually run them now and wait until they all complete. "which didn't work for 2 days showing only 92% stack in rebuilding posts since we upgraded last Friday, and I opened it many times in the current tab in FireFox, showing "starting...." and then just a blank page in that tab, a few minutes ago I opened it with a right mouse button in a New Window AND - voila! it's working right now and the progress is on! Yahooooo!All tasks took about 3 hours to complete
Management Charles Posted April 17, 2015 Management Posted April 17, 2015 For those of you who are experiencing slow task execution please try uploading this file. Simply upload it and then wait for your tasks to run as normal.applications/core/tasks/ queue.phpUnsupported, use at your own risk, backup, and so on. This is designed to work on 4.0.1. If it helps then we will release it in 4.0.2. Please report your experience here.
Nathan Explosion Posted April 17, 2015 Posted April 17, 2015 Thanks - nice timing, as I started another test upgrade yesterday and processing flew from 0 to 30% in just under 24 hours but then started crawling and has only risen about 3% in the last 6 hours.Will report back soon.
Nathan Explosion Posted April 17, 2015 Posted April 17, 2015 Already seeing a slight speed up - prior to putting the new file in place, the % indicator for rebuilding posts was 'climbing' at a rate of .01% each task run (I've got 3 cron jobs in place running every minute, but using "sleep xx;" to get a 20 second interval between each job) but seeing that change to a .04% incremenet with the new file.
Management Charles Posted April 17, 2015 Management Posted April 17, 2015 Already seeing a slight speed up - prior to putting the new file in place, the % indicator for rebuilding posts was 'climbing' at a rate of .01% each task run (I've got 3 cron jobs in place running every minute, but using "sleep xx;" to get a 20 second interval between each job) but seeing that change to a .04% incremenet with the new file. Would you mind doing a normal cron setup to simulate how clients would normally do it? Just a normal one running once a minute. I fear your setup may cause problems with the new queue logic. But either way you're already seeing an improvement of 4 times per cycle which is good
Nathan Explosion Posted April 17, 2015 Posted April 17, 2015 Of course - back to once a minute exclusively now.Since my last post, the % has jumped up by 4 so definitely seeing an improvement.
Management Charles Posted April 17, 2015 Management Posted April 17, 2015 @mikesound @Ocean West @Ziggs or others in this topic: if you could try it out too it would be appreciated. Thanks!
Ocean West Posted April 18, 2015 Posted April 18, 2015 i installed it even though my tasks have completed - unless this also does other tasks like moving file directories?
Recommended Posts
Archived
This topic is now archived and is closed to further replies.