Summary

In this chapter, we started with the fundamental concepts of decision trees, and then built a simple classification tree and a regression tree from scratch. We went over the details and checked the consistency with the scikit-learn library API.

You may notice that tree methods do tend to overfit and might fail to reach the optimal model. In the next chapter, we will explore the so-called ensemble learning. They are meta-algorithms that can be used on top of many other machine learning algorithms as well.