It’s All About Data – Options to Store and Transform ML Datasets

The real work on a machine learning project only starts once the required data is available in the project development environment. Sometimes, when the data changes very frequently or the use case requires real-time data, we may need to set up some data pipelines to ensure that the required data is always available for analysis and modeling purposes. The best way to transfer, store, or transform data also depends on the size, type, and nature of the underlying data. Raw data, as collected in the real world, is often massive in size and may belong to multiple types, such as text, audio, images, videos, and so on. Due to the varying nature, size, and type of real-world data, it becomes really important to set up the correct infrastructure for storing, transferring, transforming, and analyzing the data at scale.

In this chapter, we will learn about the different options for moving data to the Google Cloud...