Book Image

Python Data Science Essentials

By : Alberto Boschetti, Luca Massaron
Book Image

Python Data Science Essentials

By: Alberto Boschetti, Luca Massaron

Overview of this book

The book starts by introducing you to setting up your essential data science toolbox. Then it will guide you across all the data munging and preprocessing phases. This will be done in a manner that explains all the core data science activities related to loading data, transforming and fixing it for analysis, as well as exploring and processing it. Finally, it will complete the overview by presenting you with the main machine learning algorithms, the graph analysis technicalities, and all the visualization instruments that can make your life easier in presenting your results. In this walkthrough, structured as a data science project, you will always be accompanied by clear code and simplified examples to help you understand the underlying mechanics and real-world datasets.
Table of Contents (13 chapters)

The detection and treatment of outliers

In data science, examples are at the core of learning from data processes. If unusual, inconsistent, or erroneous data is fed into the learning process, the resulting model may be unable to correctly generalize the accommodating of any new data. An unusually high value present in a variable may not only skew descriptive measures such as the mean and variance, but it may also distort how many algorithms learn from data, thus exposing them to unusual values and expecting unusual responses from them.

When a data point deviates markedly from the others in a sample, it is called an outlier. Any other expected observation is labeled as an inlier.

A point may be an outlier due to the following three general causes (each one implies different remedies):

  • The point represents a rare occurrence, but it is yet a possible value, given the available data of the data distribution is just a sample. In such an occurrence, the generative underlying process is the same...