Book Image

Python for Data Science For Dummies - Second Edition

By : John Paul Mueller, Luca Massaron
Book Image

Python for Data Science For Dummies - Second Edition

By: John Paul Mueller, Luca Massaron

Overview of this book

Python is a general-purpose programming language created in the late 1980s — and named after Monty Python — that's used by thousands of people to do things from testing microchips at Intel to powering Instagram to building video games with the PyGame library. The book begins by discussing how Python can make data science easy. You’ll learn how to work with the Anaconda tool suite that makes coding in Python easy. You’ll also learn to write code using Google Colab. As you progress, you'll discover how to perform interesting calculations and data manipulations using various Python libraries, such as pandas and NumPy. You’ll learn how to create data visualizations with MatPlotLib. While learning the advanced concepts, you’ll learn how to wrangle data by using techniques, such as hierarchical clustering. Finally, you’ll learn how to work with decision trees and use machine learning to make predictions. By the end of the book, you’ll have the skills and the knowledge that’s needed to write code in Python and extract information from data.
Table of Contents (13 chapters)
Free Chapter
1
Cover
9
Index
10
About the Authors
11
Advertisement Page
12
Connect with Dummies
13
End User License Agreement

Chapter 16

Detecting Outliers in Data

IN THIS CHAPTER

Bullet Understanding what is an outlier

Bullet Distinguishing between extreme values and novelties

Bullet Using simple statistics for catching outliers

Bullet Finding out most tricky outliers by advanced techniques

Errors happen when you least expect, and that’s also true in regard to your data. In addition, data errors are difficult to spot, especially when your dataset contains many variables of different types and scale. Data errors can take a number of forms. For example, the values may be systematically missing on certain variables, erroneous numbers could appear here and there, and the data could include outliers. A red flag has to be raised when:

  • Missing values on certain groups of cases or variables imply that some specific cause is generating the error.
  • Erroneous values depend on how the application has produced or manipulated the data. For instance, you need to know whether the application has obtained data from a measurement instrument...