Deep Neural Networks (DNNs) are structured in a very uniform manner, such that, at each layer of a network thousands of identical artificial neurons perform the same computation. Therefore, DNN's architecture fits quite well with the kinds of computation that a GPU can efficiently perform.
GPU have additional advantages over CPU; these include having more computational units and having a higher bandwidth to retrieve from memory.
Furthermore, in many deep learning applications that require a lot of computational effort, GPU graphics specific capabilities can be exploited to further speed up calculations.
This chapter is organized as follows:
- GPGPU computing
- GPGPU history
- The CUDA architecture
- GPU programming model
- TensorFlow GPU set up
- TensorFlow GPU management
- Assigning a single GPU on a multi-GPU system
- Using multiple GPUs
