Book Image

Python Data Science Essentials

By : Alberto Boschetti, Luca Massaron
Book Image

Python Data Science Essentials

By: Alberto Boschetti, Luca Massaron

Overview of this book

The book starts by introducing you to setting up your essential data science toolbox. Then it will guide you across all the data munging and preprocessing phases. This will be done in a manner that explains all the core data science activities related to loading data, transforming and fixing it for analysis, as well as exploring and processing it. Finally, it will complete the overview by presenting you with the main machine learning algorithms, the graph analysis technicalities, and all the visualization instruments that can make your life easier in presenting your results. In this walkthrough, structured as a data science project, you will always be accompanied by clear code and simplified examples to help you understand the underlying mechanics and real-world datasets.
Table of Contents (13 chapters)

Feature creation

Sometimes, you'll find yourself in a situation where features and target variables are not really related. In this case, you can modify the input dataset, apply linear or nonlinear transformations that can improve the accuracy of the system, and so on. It's a very important step of the process because it completely depends on the skills of the data scientist, who is the one responsible for artificially changing the dataset and shaping the input data for a better fit with the classification model.

For example, if you're trying to predict the value of a house and you just know the height, width, and the length of each room, you can artificially build a feature that represents the volume of the house. This is strictly not an observed feature, but it's a feature built on the top of the existing ones. Let's start with some code:

In: import numpy as np
from sklearn import datasets
from sklearn.cross_validation import train_test_split
from sklearn.metrics import mean_squared_error...