Introducing the data loader in Python
The data loader is a concept that is fairly unique to deep learning. In statistical machine learning, you still see many models using gradient updating, which requires mini-batches, but the loading aspect is more hidden – more integrated with the algorithm itself. PyTorch leaned into this concept from the early days, explicitly offering a data loader object and exposing the entire training loop to the developer. While somewhat more complex than early TensorFlow, this actually enabled developers to have a lot more flexibility and control over the training process, which helped them more easily develop custom solutions. This was a part of the reason more and more research projects eventually embraced PyTorch over TensorFlow as their deep learning framework of choice. Now, the majority of models I encounter are first implemented in PyTorch, and occasionally in TensorFlow.
What is a data loader? A data loader hydrates your training loop with...
