Let's focus on how noise can affect the results. Noise is nothing but missing data, outliers, or too many predictors that try to confuse the model with unnecessary predictions.
Decision tree models don't have noise because of too many predictors, as by default, they eliminate the predictors that they don't use for predictions as opposed to other statistical and machine learning models.
Having too many predictors in a model causes the following problems:
- Additional noise in the data that affects the overall accuracy of the model
- The model becomes much more complex than it should be
- If new data is to be added for new predictions, we need to collect data even for the variables that are not important and are not really required for the predictions, because our model uses them up to a certain extent.
If these kinds of predictors are...