Processes, threads, or a single thread?
Now that we know how to use multiprocessing
, threading
and concurrent.futures
, which should you choose for your case?
Since concurrent.futures
implements both threading,
and multiprocessing
, you can mentally exchange threading
in this section with concurrent.futures.ThreadPoolExecutor
. The same goes for multiprocessing
and concurrent.futures.ProcessPoolExecutor
, of course.
When we consider the choice between single-threaded, multithreaded, and multiprocess, there are multiple factors that we can consider.
The first and most important question you should ask yourself is whether you really need to use threading
or multiprocessing
. Often, code is fast enough and you should ask yourself if the cost of dealing with the potential side effects of memory sharing and such is worth it. Not only does writing code become more complicated when parallel processing is involved, but the complexity of debugging is multiplied as well.
Second,...