Garbage in, garbage out—throughout this book, we will see this pattern when applying machine learning methods to data. Looking back, we can see that the most interesting machine learning challenges always involved some sort of feature engineering, where we tried to use our insight into the problem to carefully craft additional features that the model hopefully would pick up.
In this chapter, we will go in the opposite direction with dimensionality reduction, cutting away features that are irrelevant or redundant. Removing features might seem counter-intuitive at first thought, as more information always seems to be better than less information. Also, even if we had redundant features in our dataset, wouldn't the learning algorithm be able to quickly figure it out and set their weights to 0? There are, indeed, good reasons for trimming down...
