Storing results in a database and object storage
In the previous section, we showed how to implement a background worker to do the heavy computation and an API to schedule tasks on this worker. However, we are still missing two important aspects: the user doesn’t have any way to know the progress of the task nor to retrieve the final result. Let’s fix this!
Sharing data between the worker and the API
As we’ve seen, the worker is a program running in the background executing the computations the API has asked it to do. However, the worker doesn’t have any way to talk with the API server. That’s expected: since there could be any number of server processes, and since they could even run on different physical servers, processes cannot communicate directly. It’s always the same problem of having a central data source on which processes can write and read data.
Actually, the first approach to solve the lack of communication between the...
