Migration Background Processes

[dwarren]

Hello all!

I just recently joined Invision Cloud, and have started migrating content from a phpBB 3.2 forum.   The initial migration data import seemed to go well and it pulled over the content seemingly without issue (100k+ posts, 50k+ PMs).  There were some formatting issues with the posts, but those seem to be resolving as the background processes go through the rebuilding process.

However, the background processes (rebuilding posts, PMs, etc.) seem like they are going to take at least another day or two before completion.   In fact, a few of them seem to be stalled for the last hour or so (posts have been rebuilding for the last 5 hours, and it is still at only 5.77%).  I have two questions I'm hoping you all can help with:

1. Is it normal for the background processes to proceed at a very slow rate and stall for long periods?
2. Is it possible to let existing members into the forums while these background processes run, or should we hold off opening the forums until all of these background processes complete?

Thanks for any guidance you can provide!
 

[Daniel F]

9 minutes ago, dwarren said:

• Is it normal for the background processes to proceed at a very slow rate and stall for long periods?

If you have many background tasks with the same priority, it's indeed looking like some of them stopped at X percent, but that's not really the case.
So what happens is: The first process is called and executed. Once this finished, the next item is called based on the priority. But if you have more then 1 item with the same priority, just a random one of the items with the same priority is called. So at this stage, task1 stopped at 2% and then MySQL picked probably just another task with them same priority.

If you have a huge board with many different content items, it indeed looks like it's not processing, but trust me, it is:) If you're still worried, please submit a ticket and I'll be more then happy to take a look at this:)

I agree that this can be confusing and have raised an internal suggestion to improve this.
 

[dwarren]

4 minutes ago, Daniel F said:

If you have many background tasks with the same priority, it's indeed looking like some of them stopped at X percent, but that's not really the case.
So what happens is: The first process is called and executed. Once this finished, the next item is called based on the priority. But if you have more then 1 item with the same priority, just a random one of the items with the same priority is called. So at this stage, task1 stopped at 2% and then MySQL picked probably just another task with them same priority.

If you have a huge board with many different content items, it indeed looks like it's not processing, but trust me, it is:) If you're still worried, please submit a ticket and I'll be more then happy to take a look at this:)

I agree that this can be confusing and have raised an internal suggestion to improve this.
 

Thank you for the quick response! It is good to know that processes are still running.

Is there any drawback to letting members in while these tasks are processing? We aren't sure if it is okay to let everyone in to start posting or if it is better to wait.  Also, is it okay for us to start doing things like setting permissions on forums while these tasks are running?

We just don't want to get ahead of ourselves and screw something up due to lack of patience...but if there is no risk involved, I know our members would like to dive in to their new forums.

[Daniel F]

9 minutes ago, dwarren said:

Is there any drawback to letting members in while these tasks are processing?

No, it should be fine.

Background tasks are built to work while the board is online:)