7. Dealing with Outliers Using Python | Big Data Visualization

Outliers

In this chapter, we want to deal with the manipulation of big data sources to address data outliers. So let's have a quick reminder for the reader:

Outliers can be defined as:

A data point that is way out of keeping with the others
That piece of data that doesn't fit
Either a very high value or a very low value
Unusual observations within the data
An observation point that is distant from all others

Options for outliers

The options that are generally accepted for dealing with found outliers in big data are:

Delete: This includes the outlier values or even the actual variable where the outliers exist
Transform: This includes the values or the variable itself

Delete

If you have just a few outliers, you may decide to simply delete those outlying values (they then become blank or missing values, which usually are easier to deal with in a visualization). Also, if the variable just doesn't make sense, or if there are just too many outliers in that variable (or maybe you just don't need the variable...

Tech Concepts

Programming languages

Tech Tools

Unlimited access to the largest independent learning library in tech of over 8,000 expert-authored tech books and videos.

Innovative learning tools, including AI book assistants, code context explainers, and text-to-speech.

50+ new titles added per month and exclusive early access to books as they are being written.

Big Data Visualization

By : James D. Miller

Big Data Visualization

By: James D. Miller

Overview of this book

Outliers

Options for outliers

Delete

Big Data Visualization

By : James D. Miller

Big Data Visualization

By: James D. Miller

Overview of this book

Outliers

Options for outliers

Delete

Confirmation

Buy this book with your credits?

Submit Your Feedback

Create a Free Account To Continue Reading

Sign in to activate your 7-day free access