In this chapter, we will learn about two new classes of machine learning (ML) models: decision trees and random forests. We will see how decision trees learn rules from data that encodes non-linear relationships between the input and the output variables. We will illustrate how to train a decision tree and use it for prediction for regression and classification problems, visualize and interpret the rules learned by the model, and tune the model's hyperparameters to optimize the bias-variance tradeoff and prevent overfitting. Decision trees are not only important standalone models but are also frequently used as components in other models.
In the second part of this chapter, we will introduce ensemble models that combine multiple individual models to produce a single aggregate prediction with lower prediction-error variance. We will illustrate...