Distributing tasks on several cores with IPython.parallel

In the previous sections, we covered a few methods to accelerate Python code. Here, we will see how to run multiple tasks in parallel on a multicore computer. IPython implements highly-powerful and user-friendly facilities for interactive parallel computing in the Notebook.

We first need to install ipyparallel (also called IPython.parallel) with conda install ipyparallel. Next, let's import NumPy and ipyparallel:

In [1]: import numpy as np
        # ipyparallel was IPython.parallel before IPython 4.0
        from ipyparallel import Client

To use IPython.parallel, we need to launch a few engines.

The first way to do it is to run ipcluster start in the terminal.

You can also launch engines from the Notebook dashboard. However, you first need to add c.NotebookApp.server_extensions.append('ipyparallel.nbextension') in the file ~/.jupyter/jupyter_notebook_config.py (you may need to create this file). Then, from the Notebook dashboard (accessible...