Up until this point, we've explored how to load data into Python and process it to create a bidimensional NumPy array containing numerical values (your dataset). Now, we are ready to be immersed fully in data science, extract meaning from data, and develop potential data products. This chapter on data treatment and transformations and the next one on machine learning are the most challenging sections of this entire book.
In this chapter, you will learn how to do the following:
- Briefly explore data and create new features
- Reduce the dimensionality of data
- Spot and treat outliers
- Decide on the best score or loss metrics for your project
- Apply scientific methodology and effectively test the performance of your machine learning hypothesis
- Reduce the complexity of the data science problem by decreasing the number of features
- Optimize your learning parameters...