Improving Scalability – GPU Execution

For the case studies described in this book, we have used relatively small datasets and small networks. This allowed us to train the networks within hours using only CPU-based execution. However, training tasks that take minutes or hours on small datasets can easily take days or weeks on larger datasets; small network architectures can quickly increase in size and execution times can quickly become prohibitive. In general, when working with deep neural networks, the training phase is the most resource-intensive task.

GPUs have been designed to handle multiple computations simultaneously. This paradigm suits the intensive computations required to train a deep learning network. Hence, GPUs are an alternative option to train large deep learning networks efficiently and effectively on large datasets.

Some Keras libraries can exploit the computational power of NVIDIA®-compatible GPUs via the TensorFlow paradigms. As a consequence,...