A decision tree is the arrangement of data in a tree structure where, at each node, data is separated into different branches according to the value of the attribute at the node.
To construct a decision tree, we will use a standard ID3 learning algorithm that chooses an attribute that classifies data samples in the best possible way to maximize the information gain—a measure based on information entropy.
In this chapter, we will cover the following topics:
- What a decision tree is and how to represent data in a decision tree through the swim preference example
- The concepts of information entropy and information gain, theoretically in the first instance, before applying the swim preference example in practical terms in the Information theory section
- How to use a ID3 algorithm to construct a decision tree from the training data, and its implementation in Python...
