Proper preprocessing is important because it conditions data for downstream work and allows users to trust their downstream results. This step is where many practitioners spend the majority of their time, so you should also get comfortable with spending your time on the methods that are discussed here. This chapter will start with cleaning and filtering data input, and then move onto feature selection and dimensional reduction. Feature selection involves searching for relationships and quantifying data/variable quality. So, for all intents and purposes, the mining begins here.
The following topics will be covered in this chapter:
- Cleaning input data
- Working with missing values
- Normalization and standardization
- Handling categorical data
- High-dimensional data and the curse of dimensionality
- Feature selection with filter and wrapper methods...