This chapter describes the bias and variance effects and their pathological cases, which usually appear when training machine learning (ML) models. For example, the high variance effect, also known as overfitting, is a phenomenon in ML where the constructed model explains the examples from the training set but works relatively poorly on the examples that did not participate in the training process. This occurs because while training a model, random patterns will start appearing that are normally absent from the general population. The opposite of overfitting is known as underfitting. This happens when the trained model becomes unable to predict patterns in new data or even in the training data. Such an effect can be the result of a limited training dataset or weak model design.
In this chapter, we will learn how to deal with overfitting...