To parallelize computations efficiently, you need to also understand when to use processes to perform computation and when threads are the better tool for the job. Long story short, if your only target is to actually parallelize work, then it's best to start with adding extra threads up to the point where they don't bring extra benefits. At such a point, add more processes on other machines in your network, each with multiple threads too.
Why is that? Because processes are more heavyweight than threads. Spawning a process and switching between them takes longer than creating and switching between threads. Each process requires its own memory space, while threads within the same process share their memory. Also, inter-process communication is slower than just passing variables between threads. Working with threads is easier than it is with processes, so the development will be faster too.
Processes, however, also have...
