What is hyperparameter tuning?
Hyperparameter tuning is a process whereby we search for the best set of hyperparameters of an ML model from all of the candidate sets. It is the process of optimizing the technical metrics we care about. The goal of hyperparameter tuning is simply to get the maximum evaluation score on the validation set without causing an overfitting issue.
Hyperparameter tuning is one of the model-centric approaches to optimizing a model's performance. In practice, it is suggested to prioritize data-centric approaches over a model-centric approach when it comes to optimizing a model's performance. Data-centric means that we are focusing on cleaning, sampling, augmenting, or modifying the data, while model-centric means that we are focusing on the model and its configuration.
To understand why data-centric is prioritized over model-centric, let's say you are a cook in a restaurant. When it comes to cooking, no matter how expensive and fancy your...
