The API guide also makes the parallel between data pipelines for training and Extract, Transform, Load (ETL) processes. ETL is a common paradigm for data processing in computer science. In computer vision, ETL pipelines in charge of feeding models with training data usually look like this:
The extraction step consists of selecting data sources and extracting their content. These sources may be listed explicitly by a document (for instance, a CSV file containing the filenames for all the images), or implicitly (for instance, with all the dataset's images already stored in a specific folder). Sources may be stored on different devices (local or remote), and it is also the task of the extractor to list these different sources and extract their content. For example, it is common in computer vision to have datasets so big that they have to be stored on multiple hard...
