When learning a new algorithm, you can get discouraged by the number of hyperparameters and find it hard to decide where to start. Therefore, I suggest we start by answering the following two questions:
- How has the algorithm been architected?
- How does the algorithm train?
In the following sections, we are going to answer both of these questions and learn about the corresponding hyperparameters one by one.
Understanding the algorithm's architecture
Luckily, the knowledge we gained about linear models inChapter 3, Making Decisions with Linear Equations, will give us a good headstart here. In brief, linear models can be outlined in the following diagram:
Each of the input features (xi) is multiplied by a weight (wi), and the sum of these products is the output of the model (y). Additionally, we sometimes add an extra bias (threshold), along with its weight...
