Multiprocessor and distributed environments

There are a variety of multiprocessor and distributed environment possibilities. The most common reason for using more than one processor is, of course, to make models run faster. The time it takes to load MNIST—a relatively tiny dataset of 60,000 images—to memory is not significant. However, consider the situation where we have giga or terabytes of data, or if the data is distributed across multiple servers. The situation is even more complex when we consider online models, where data is being harvested from multiple servers in real time. Clearly, some sort of parallel processing capability is required.

Using a GPU

The simplest way to make a model run faster is to...