Chapter 7: Data and Feature Management

In this chapter, we will add a feature management data layer to the machine learning platform being built. We will leverage the features of the MLflow Projects module to structure our data pipeline.

Specifically, we will look at the following sections in this chapter:

• Structuring your data pipeline project
• Acquiring stock data
• Checking data quality
• Managing features

In this chapter, we will acquire relevant data to provide datasets for training. Our primary resource will be the Yahoo Finance Data for BTC dataset. Alongside that data, we will acquire the following extra datasets.

Leveraging our productionization architecture introduced in Chapter 6, Introducing ML Systems Architecture, represented in Figure 7.1, the feature and data component is responsible for acquiring data from sources and making the data available in a format consumable by the different components of the platform:

...