Book Image

Hands-On Exploratory Data Analysis with Python

By : Suresh Kumar Mukhiya, Usman Ahmed
Book Image

Hands-On Exploratory Data Analysis with Python

By: Suresh Kumar Mukhiya, Usman Ahmed

Overview of this book

Exploratory Data Analysis (EDA) is an approach to data analysis that involves the application of diverse techniques to gain insights into a dataset. This book will help you gain practical knowledge of the main pillars of EDA - data cleaning, data preparation, data exploration, and data visualization. You’ll start by performing EDA using open source datasets and perform simple to advanced analyses to turn data into meaningful insights. You’ll then learn various descriptive statistical techniques to describe the basic characteristics of data and progress to performing EDA on time-series data. As you advance, you’ll learn how to implement EDA techniques for model development and evaluation and build predictive models to visualize results. Using Python for data analysis, you’ll work with real-world datasets, understand data, summarize its characteristics, and visualize it for business intelligence. By the end of this EDA book, you’ll have developed the skills required to carry out a preliminary investigation on any dataset, yield insights into data, present your results with visual aids, and build a model that correctly predicts future outcomes.

Related Content you might be interested in

Current Title:

Small book image
Hands-On Exploratory Data Analysis with Python
Suresh Kumar Mukhiya | Usman Ahmed
Mar, 2020 | 352 pages
Small book image
Machine Learning with scikit-learn Quick Start Guide
Kevin Jolly
Oct, 2018 | 172 pages
Small book image
Become a Python Data Analyst
Alvaro Fuentes
Aug, 2018 | 178 pages
Small book image
Hands-On Data Analysis with Pandas
Stefanie Molin
Apr, 2021 | 788 pages
Table of Contents (17 chapters)
Preface
Preface
Who this book is for
What this book covers
To get the most out of this book
Get in touch
1
Section 1: The Fundamentals of EDA
Section 1: The Fundamentals of EDA
Free Chapter
2
Exploratory Data Analysis Fundamentals
Exploratory Data Analysis Fundamentals
Understanding data science
The significance of EDA
Making sense of data
Comparing EDA with classical and Bayesian analysis
Software tools available for EDA
Getting started with EDA
Summary
Further reading
3
Visual Aids for EDA
Visual Aids for EDA
Technical requirements
Line chart
Bar charts
Scatter plot
Area plot and stacked plot
Pie chart
Table chart
Polar chart
Histogram
Lollipop chart
Choosing the best chart
Other libraries to explore
Summary
Further reading
4
EDA with Personal Email
EDA with Personal Email
Technical requirements
Loading the dataset
Data transformation
Data analysis
Summary
Further reading
5
Data Transformation
Data Transformation
Technical requirements
Background
Merging database-style dataframes
Transformation techniques
Benefits of data transformation
Summary
Further reading
6
Section 2: Descriptive Statistics
Section 2: Descriptive Statistics
7
Descriptive Statistics
Descriptive Statistics
Technical requirements
Understanding statistics
Measures of central tendency
Measures of dispersion
Summary
Further reading
8
Grouping Datasets
Grouping Datasets
Technical requirements
Understanding groupby()
Groupby mechanics
Data aggregation
Pivot tables and cross-tabulations
Summary
Further reading
9
Correlation
Correlation
Technical requirements
Introducing correlation
Types of analysis
Discussing multivariate analysis using the Titanic dataset
Outlining Simpson's paradox
Correlation does not imply causation
Summary
Further reading
10
Time Series Analysis
Time Series Analysis
Technical requirements
Understanding the time series dataset
TSA with Open Power System Data
Summary
Further reading
11
Section 3: Model Development and Evaluation
Section 3: Model Development and Evaluation
12
Hypothesis Testing and Regression
Hypothesis Testing and Regression
Technical requirements
Hypothesis testing
p-hacking
Understanding regression
Model development and evaluation
Summary
Further reading
13
Model Development and Evaluation
Model Development and Evaluation
Technical requirements
Types of machine learning
Understanding supervised learning
Understanding unsupervised learning
Understanding reinforcement learning
Unified machine learning workflow
Summary
Further reading
14
EDA on Wine Quality Data Analysis
EDA on Wine Quality Data Analysis
Technical requirements
Disclosing the wine quality dataset
Analyzing red wine
Analyzing white wine
Model development and evaluation
Summary
Further reading
15
Other Books You May Enjoy
Other Books You May Enjoy
Leave a review - let other readers know what you think
Appendix
Appendix
String manipulation
Using pandas vectorized string functions
Using regular expressions
Further reading

TSA with Open Power System Data

In this section, we are going to use Open Power System Data to understand TSA. We'll look at the time series data structures, time-based indexing, and several ways to visualize time series data.

We will start by importing the dataset. Look at the code snippet given here:

# load time series dataset
df_power = pd.read_csv("https://raw.githubusercontent.com/jenfly/opsd/master/opsd_germany_daily.csv")
df_power.columns

The output of the preceding code is given here:

Index(['Consumption', 'Wind', 'Solar', 'Wind+Solar'], dtype='object')

The columns of the dataframe are described here:

  • Date: The date is in the format yyyy-mm-dd.
  • Consumption: This indicates electricity consumption in GWh.
  • Solar: This indicates solar power production in GWh.
  • Wind+Solar: This represents the sum of solar and...
Previous Section
End of Section 4
Next Section