Ensemble learning involves a combination of techniques that allows multiple machine learning models, called base learners (or, sometimes, weak learners), to consolidate their predictions and output a single, optimal prediction, given their respective inputs and outputs.
In this chapter, we will give an overview of the main problems that ensembles try to solve, namely, bias and variance, as well as the relationship between them. This will help us understand the motivation behind identifying the root cause of an under-performing model and using an ensemble to address it. Furthermore, we will go over the basic categories of the methodologies available, as well as the difficulties we can expect to encounter when implementing ensembles.
The main topics covered in this chapter are the following:
- Bias, variance, and the trade-off between the two...