Clustering and parallel programming

The term clustering may have a number of operational definitions depending on the situation that one is facing or the organization that one is working with. In this chapter, we will use the term in a very general sense to indicate a system of computing nodes across which a task may be split and whose parts may be executed in parallel with all the system nodes. We won't specify what nodes are, as they may be anything from a collection of processes on the same machine or a computer network to virtual machines or physical computers on a network.

The word "cluster" alludes to a logical collection, but in our definition there is a more important word—parallel. For our purposes, clusters exist to make running code in parallel more efficient or convenient. The topic of parallel computing is a vast one and has an interesting history. However, it rose to greater prominence in 2003 due to the physical limitations that were encountered by the chip-making industry...