Book Image

The Pandas Workshop

By : Blaine Bateman, Saikat Basak, Thomas V. Joseph, William So
5 (1)
Book Image

The Pandas Workshop

5 (1)
By: Blaine Bateman, Saikat Basak, Thomas V. Joseph, William So

Overview of this book

The Pandas Workshop will teach you how to be more productive with data and generate real business insights to inform your decision-making. You will be guided through real-world data science problems and shown how to apply key techniques in the context of realistic examples and exercises. Engaging activities will then challenge you to apply your new skills in a way that prepares you for real data science projects. You’ll see how experienced data scientists tackle a wide range of problems using data analysis with pandas. Unlike other Python books, which focus on theory and spend too long on dry, technical explanations, this workshop is designed to quickly get you to write clean code and build your understanding through hands-on practice. As you work through this Python pandas book, you’ll tackle various real-world scenarios, such as using an air quality dataset to understand the pattern of nitrogen dioxide emissions in a city, as well as analyzing transportation data to improve bus transportation services. By the end of this data analytics book, you’ll have the knowledge, skills, and confidence you need to solve your own challenging data science problems with pandas.

Related Content you might be interested in

Current Title:

Small book image
The Pandas Workshop
Blaine Bateman | Saikat Basak | Thomas V. Joseph | William So
Jun, 2022 | 744 pages
Small book image
Learning pandas.
Michael Heydt
Jun, 2017 | 446 pages
Small book image
Mastering Exploratory Analysis with pandas
Harish Garg
Sep, 2018 | 140 pages
Small book image
The Supervised Learning Workshop
Blaine Bateman | Ashish Ranjan Jha | Benjamin Johnston | Ishita Mathur
Feb, 2020 | 532 pages
Table of Contents (21 chapters)
Preface
Preface
Who this book is for
What this book covers
To get the most out of this book
Download the example code files
Download the color images
Conventions used
Get in touch
Share Your Thoughts
1
Part 1 – Introduction to pandas
Part 1 – Introduction to pandas
Free Chapter
2
Chapter 1: Introduction to pandas
Chapter 1: Introduction to pandas
Introduction to the world of pandas
Exploring the history and evolution of pandas
Components and applications of pandas
Understanding the basic concepts of pandas
Activity 1.01 – comparing sales data for two stores
Summary
3
Chapter 2: Working with Data Structures
Chapter 2: Working with Data Structures
Introduction to data structures
The need for data structures
Indexes and columns
Series
Activity 2.01 – Working with pandas data structures
Summary
4
Chapter 3: Data I/O
Chapter 3: Data I/O
The world of data
Exploring data sources
Fundamental formats
Additional text formats
Manipulating SQL data
Activity 3.01 – using SQL data for pandas analytics
Summary
5
Chapter 4: Pandas Data Types
Chapter 4: Pandas Data Types
Introducing pandas dtypes
Missing data types
Activity 4.01 – optimizing memory usage by converting into the appropriate dtypes
Subsetting by data types
Summary
6
Part 2 – Working with Data
Part 2 – Working with Data
7
Chapter 5: Data Selection – DataFrames
Chapter 5: Data Selection – DataFrames
Introduction to DataFrames
Data selection in pandas DataFrames
Activity 5.01 – Creating a multi-index from columns
Bracket and dot notation
Changing DataFrame values using bracket or dot notation
Summary
8
Chapter 6: Data Selection – Series
Chapter 6: Data Selection – Series
Introduction to pandas Series
The Series index
Preparing Series from DataFrames and vice versa
Activity 6.01 – Series data selection
Understanding the differences between base Python and pandas data selection
Activity 6.02 – DataFrame data selection
Summary
9
Chapter 7: Data Exploration and Transformation
Chapter 7: Data Exploration and Transformation
Introduction to data transformation
Dealing with messy data
Dealing with missing data
Summarizing data
Activity 7.01 – data analysis using pivot tables
Summary
10
Chapter 8: Understanding Data Visualization
Chapter 8: Understanding Data Visualization
Introduction to data visualization
Understanding the basics of pandas visualization
Exploring matplotlib
Visualizing data of different types
Activity 8.01 – Using data visualization for exploratory data analysis
Summary
11
Part 3 – Data Modeling
Part 3 – Data Modeling
12
Chapter 9: Data Modeling – Preprocessing
Chapter 9: Data Modeling – Preprocessing
An introduction to data modeling
Exploring dependent and independent variables
Understanding data scaling and normalization
Activity 9.01 – Data splitting, scaling, and modeling
Summary
13
Chapter 10: Data Modeling – Modeling Basics
Chapter 10: Data Modeling – Modeling Basics
Introduction to data modeling
Learning the modeling basics
Predicting future values of time series
Activity 10.01 – Normalizing and smoothing data
Summary
14
Chapter 11: Data Modeling – Regression Modeling
Chapter 11: Data Modeling – Regression Modeling
An introduction to regression modeling
Exploring regression modeling
Model diagnostics
Activity 11.01 – Multiple regression with non-linear models
Summary
15
Part 4 – Additional Use Cases for pandas
Part 4 – Additional Use Cases for pandas
16
Chapter 12: Using Time in pandas
Chapter 12: Using Time in pandas
Introduction to time series
What are datetimes?
Activity 12.01 – understanding power usage
Datetime math operations
Summary
17
Chapter 13: Exploring Time Series
Chapter 13: Exploring Time Series
The time series as an index
Resampling, grouping, and aggregation by time
Activity 13.01 – Creating a time series model
Summary
18
Chapter 14: Applying pandas Data Processing for Case Studies
Chapter 14: Applying pandas Data Processing for Case Studies
Introduction to the case studies and datasets
Recap of the preprocessing steps
Activity 14.01 – analyzing air quality data
Summary
19
Chapter 15: Appendix
Solution 1.1
Solution 2.1
Solution 3.1
Solution 4.1
Solution 5.1
Solution 6.1
Solution 6.2
Solution 7.1
Solution 8.1
Solution 9.1
Solution 10.1
Solution 11.1
Solution 12.1
Solution 13.1
Solution 14.1
Why subscribe?
20
Other Books You May Enjoy
Other Books You May Enjoy
Packt is searching for authors like you
Share Your Thoughts

Solution 11.1

As part of a research effort to improve metallic-oxide semiconductor sensors for the toxic gas CO (carbon monoxide), you are asked to investigate models of the sensor response for an array of sensors. You will review the data, perform some feature engineering for non-linear features, and then compare a baseline linear regression approach to a random forest model.

Perform the following steps to complete the activity:

  1. For this exercise, you will need the pandas and numpy libraries, and three modules from sklearn, matplotlib, and seaborn. Load them in the first cell of the notebook:
    import pandas as pd
import numpy as np
from sklearn.linear_model import LinearRegression as OLS
from sklearn.ensemble import RandomForestRegressor
from sklearn.preprocessing import StandardScaler
import matplotlib.pyplot as plt
import seaborn as sns
  2. As we have done before, create a utility function to plot a grid of histograms given the data, which variables to plot, the rows and...
Previous Section
End of Section 12
Next Section