Decision Trees
A decision tree has leaves, branches, and nodes. Nodes are where a decision is made. A decision tree consists of rules that we use to formulate a decision (or prediction) on the prediction of a data point.
Every node of the decision tree represents a feature, while every edge coming out of an internal node represents a possible value or a possible interval of values of the tree. Each leaf of the tree represents a label value of the tree.
This may sound complicated, but let's look at an application of this.
Suppose we have a dataset with the following features and the response variable is determining whether a person is creditworthy or not:
Figure 4.1: Sample dataset to formulate the rules
A decision tree, remember, is just a group of rules. Looking at the dataset in Figure 4.1, we can come up with the following rules:
- All people with house loans are determined as creditworthy.
- If debtors are employed and studying, then...
