What it is
Queue extensions allow your application to process certain functionality in the background. When deciding whether you need to use a task or a queue extension, your consideration should often come down to how much data needs to be processed. If you are running through 10 items on a regular schedule, you should use a task. If you need to loop through 50,000 items as quickly as possible, but this can be done in the background instead of immediately, you should use a Queue extension.
Queue extensions are used to send follow notifications (when there are more than a specific number of recipients of the notification) as well as rebuilding images, among many other things.
How it works
When you create a new instance of this extension, there will be 4 methods in the class template.
/**
* Parse data before queuing
*
* @param array $data
* @return array
*/
public function preQueueData( $data )
{
return $data;
}
The preQueueData() method allows the Queue extension to calculate and store any data necessary prior to initiating the background queue task. Often times you may need to capture the highest ID available in the database at the time the task initiates to ensure you do not rebuild any data beyond that ID. Or you may need to capture the number of items in a database table for use with the progress meter later. You can simply return the $data unmodified, or you can add new entries to the $data array and return it.
/**
* Run Background Task
*
* @param mixed $data Data as it was passed to \IPS\Task::queue()
* @param int $offset Offset
* @return int New offset
* @throws \IPS\Task\Queue\OutOfRangeException Indicates offset doesn't exist and thus task is complete
*/
public function run( $data, $offset )
{
}
The run() method accepts the data returned from preQueueData(), as well as an offset, and then runs the task. You will generally want to process a limited number of items each run (how many depends on how intensive the processing is, but typically somewhere between 50 and 500), and then either return the new offset (the number of items processed + the initial offset), or the new ID to start at. You can use this method if the database table you are working with is extremely large and you would prefer to run a query like "WHERE id > X LIMIT 500" instead of "LIMIT 50000, 500", which can get slow over time.
If you have finished processing all of the items for this task, you should throw new \IPS\Task\Queue\OutOfRangeException;, however returning NULL will also cause the task to cease running as well.
/**
* Get Progress
*
* @param mixed $data Data as it was passed to \IPS\Task::queue()
* @param int $offset Offset
* @return array( 'text' => 'Doing something...', 'complete' => 50 ) Text explaining task and percentage complete
* @throws \OutOfRangeException Indicates offset doesn't exist and thus task is complete
*/
public function getProgress( $data, $offset )
{
return array( 'text' => 'Doing something...', 'complete' => 50 );
}
The getProgress() method returns the title of the task to show on the AdminCP dashboard. You will be returning an array with key 'text' being the text to display and 'complete' being the percentage completed so far for this task. You can reference the offset against the total number of items in a database table to determine the percentage completed so far.
/**
* Perform post-completion processing
*
* @param array $data
* @return void
*/
public function postComplete( $data )
{
}
The optional postComplete() method allows your extension to perform any post-completion cleanup. For example, if your task was moving files from one area to another, you may wish to delete the original files (or container folder) after the files have finished moving.
To initiate the queue task after you have created it, you call \IPS\Task::queue()
\IPS\Task::queue( 'myapp', 'ExtensionKey', array( 'initial_data' => true ), 3, array( 'initial_data' ) );
The first parameter is the application key and the second parameter is the extension key (e.g. the class name or file name of the Queue extension you wish to initiate). The third parameter is the default value for the $data array, which is then passed to preQueueData() and later to run() and getProgress(). If you have no initial data to supply you can pass an empty array or NULL, and you can still populate this variable within preQueueData() as needed.
The fourth parameter is the queue task's priority. The lower priority, the higher the priority - that is to say, tasks with a priority of 1 will be ran before tasks with a priority of 2 (or higher).
The last parameter is an array of keys from the data array that can be checked for duplicates, resulting in the task not being duplicated. For example, if you call the following:
\IPS\Task::queue( 'myapp', 'ExtensionKey' );
\IPS\Task::queue( 'myapp', 'ExtensionKey' );
There will be two background tasks created. If, however, you call the following:
\IPS\Task::queue( 'myapp', 'ExtensionKey', array( 'initial_data' => true ) );
\IPS\Task::queue( 'myapp', 'ExtensionKey', array( 'initial_data' => true ), 3, array( 'initial_data' ) );
There will only be one background task. This is because we instructed the queue() method on the second call that if there was already a queue task with the same application key and extension key which also had a data key 'initial_data' that matches our current value (true), do not duplicate the task, but replace it instead.
If instead we had this
\IPS\Task::queue( 'myapp', 'ExtensionKey', array( 'initial_data' => false ), 3, array( 'initial_data' ) );
\IPS\Task::queue( 'myapp', 'ExtensionKey', array( 'initial_data' => true ), 3, array( 'initial_data' ) );
Then we would again have two tasks, because the 'initial_data' value from the first and second tasks does not match.
Queue tasks are used extensively throughout the Invision Community software and are a great way to offload intensive processing to the background so that administrators can move on with their tasks and not have to wait for a mundane unimportant task to finish.
Report Document