Tree-structure models for classification

When we build our model, we have to find the informative attributes or the features contained in our data. A feature that never changes in our data set is not a very informative attribute. Some features will be more informative and they will influence the outcome more than the others. In other words, we can say that such features provide more information gain than the others. Once we have identified the features providing the information and sorted them in order of information gain, we can arrange them in the form of an inverted tree to create our model. The leaf nodes of the tree denote the outcome. Other nodes denote the features or attributes that provide information gain to reach the final outcome.

In the tree-based learning approach, we recursively divide the historical data. During the training of the model, various dividing criteria will be tried. If a feature has a non-numeric value then it is converted into binary. A classification tree can...