Accelerating Deep Learning
Big data and large-scale networks require massive operations in deep learning. We have used CPUs for calculations so far, but CPUs alone are not sufficient to tackle deep learning. In fact, many deep learning frameworks support Graphics Processing Units (GPUs) to process a large number of operations quickly. Recent frameworks are starting to support distributed learning by using multiple GPUs or machines. This section describes accelerating calculations in deep learning. Our implementations of deep learning ended in section 8.1. We will not implement the acceleration (such as support of GPUs) described here.
Challenges to Overcome
Before discussing the acceleration of deep learning, let's see what processes take time in deep learning. The pie charts in Figure 8.14 show the time spent on each class in the forward processing of AlexNet:
