Processing long jobs with the task queue
App Engine provides a mechanism called request timer to ensure that requests from a client have a finite lifespan, avoiding infinite loops and preventing an overly aggressive use of the resources from an application. In particular, the request timer raises a DeadlineExceededError
error whenever a request takes more than 60 seconds to complete. We have to take this into consideration if our application provides functionalities that involve complex queries, I/O operations, or image processing. This is the case of the ShrinkHandler
class from the previous paragraph: the number of notes to be loaded and the attached images to be processed could be big enough to make the request last more than 60 seconds. In such cases, we can use the task queue, which is a service provided by App Engine that lets us execute operations outside the request / response cycle with a wider time limit of 10 minutes.
There are two types of task queue: push queues, which are used...