Creating a hyperparameter tuning pipeline in Pachyderm

In this section, we will explore our hyperparameter tuning pipeline and will create all the required attributes in Pachyderm to run our example.

Example overview

The house price prediction challenge is one of the classic ML examples of hyperparameter tunning optimization. It might not sound that complicated and may even be easy to predict based on your empirical experience. Likely, you know the area where you live pretty well and can estimate the price of houses based on square footage, number of rooms, adjacent land plot, and other parameters.

This information can be represented in a form of a two-dimensional (2D) array or a table with mentioned parameters. Here is an example of such a table:

Figure 9.4 – Sample housing data

Based on this information, you can predict the price of a house with similar characteristics without using any ML model and just by looking at these numbers.

But...