Clustering with IPython
As explained in the IPython documentation for parallel computing, IPython has built-in support for parallelism. This came as a result of the architectural overhaul that IPython received when the project finished migrating to ZeroMQ in 2011. The architecture that resulted can be summarized with the following components, all of which are present in the IPython.parallel package:
The IPython engine: This is a Python interpreter that accepts Python commands over a network connection. Multiple engines form the basis of IPython's parallel computing capabilities.
The IPython hub: This is the process that keeps track of engine connections, schedulers, clients, task requests, and results. Its primary purpose is to facilitate queries that are made from the cluster state.
The IPython schedulers: The actions that can be performed on an engine go through a scheduler. They also provide a fully asynchronous interface to a set of engines.
The controller client: This is the user interface...
