Multiprocessing with IPython

In data analysis, we often perform processing tasks which are computationally expensive. In these cases we will need multiprocessing tools that enable us to improve the performance. Multiprocessing in IPython is a big enough topic to have its own chapter. In this section, we only show how we can run a map function into parallel processes with the Pool object in Wakari.

Pool

The Pool class is the easiest way to run a parallel process into a Wakari IPython Notebook. In this case, we will create a function that will be applied to each element on a numpy array by using the map_async method, which is a variant of the map method that delivers the result asynchronously.

In the following screenshot, we can see the result of the map_async function of the Pool object. With the get method, we will get the result when it arrives:

Tip

You can find the multiprocessing module documentation at http://docs.python.org/2/library/multiprocessing.html.