Improving Training Performance with MXNet

In previous chapters, we have leveraged MXNet capabilities to solve computer vision and Natural Language Processing (NLP) tasks. In those chapters, the focus was on obtaining the maximum performance out of pre-trained models, leveraging the Model Zoos from GluonCV and GluonNLP. We trained these models using different approaches: from scratch, transfer learning, and fine-tuning. In this chapter, we will focus on improving the performance of the training process itself and accelerating how we can obtain those results.

To achieve the objective of optimizing the performance of our training loops, MXNet contains different features. We have already briefly used some of those features such as the concept of lazy evaluation, which was introduced in Chapter 1. We will revisit it in this chapter, in combination with automatic parallelization. Moreover, we will optimize how to access data efficiently, leveraging Gluon DataLoaders in different contexts...