Book Image

The Pandas Workshop

By : Blaine Bateman, Saikat Basak, Thomas V. Joseph, William So
5 (1)
Book Image

The Pandas Workshop

5 (1)
By: Blaine Bateman, Saikat Basak, Thomas V. Joseph, William So

Overview of this book

The Pandas Workshop will teach you how to be more productive with data and generate real business insights to inform your decision-making. You will be guided through real-world data science problems and shown how to apply key techniques in the context of realistic examples and exercises. Engaging activities will then challenge you to apply your new skills in a way that prepares you for real data science projects. You’ll see how experienced data scientists tackle a wide range of problems using data analysis with pandas. Unlike other Python books, which focus on theory and spend too long on dry, technical explanations, this workshop is designed to quickly get you to write clean code and build your understanding through hands-on practice. As you work through this Python pandas book, you’ll tackle various real-world scenarios, such as using an air quality dataset to understand the pattern of nitrogen dioxide emissions in a city, as well as analyzing transportation data to improve bus transportation services. By the end of this data analytics book, you’ll have the knowledge, skills, and confidence you need to solve your own challenging data science problems with pandas.

Related Content you might be interested in

Current Title:

Small book image
The Pandas Workshop
Blaine Bateman | Saikat Basak | Thomas V. Joseph | William So
Jun, 2022 | 744 pages
Small book image
Learning pandas.
Michael Heydt
Jun, 2017 | 446 pages
Small book image
Mastering Exploratory Analysis with pandas
Harish Garg
Sep, 2018 | 140 pages
Small book image
The Supervised Learning Workshop
Blaine Bateman | Ashish Ranjan Jha | Benjamin Johnston | Ishita Mathur
Feb, 2020 | 532 pages
Table of Contents (21 chapters)
Preface
Preface
Who this book is for
What this book covers
To get the most out of this book
Download the example code files
Download the color images
Conventions used
Get in touch
Share Your Thoughts
1
Part 1 – Introduction to pandas
Part 1 – Introduction to pandas
Free Chapter
2
Chapter 1: Introduction to pandas
Chapter 1: Introduction to pandas
Introduction to the world of pandas
Exploring the history and evolution of pandas
Components and applications of pandas
Understanding the basic concepts of pandas
Activity 1.01 – comparing sales data for two stores
Summary
3
Chapter 2: Working with Data Structures
Chapter 2: Working with Data Structures
Introduction to data structures
The need for data structures
Indexes and columns
Series
Activity 2.01 – Working with pandas data structures
Summary
4
Chapter 3: Data I/O
Chapter 3: Data I/O
The world of data
Exploring data sources
Fundamental formats
Additional text formats
Manipulating SQL data
Activity 3.01 – using SQL data for pandas analytics
Summary
5
Chapter 4: Pandas Data Types
Chapter 4: Pandas Data Types
Introducing pandas dtypes
Missing data types
Activity 4.01 – optimizing memory usage by converting into the appropriate dtypes
Subsetting by data types
Summary
6
Part 2 – Working with Data
Part 2 – Working with Data
7
Chapter 5: Data Selection – DataFrames
Chapter 5: Data Selection – DataFrames
Introduction to DataFrames
Data selection in pandas DataFrames
Activity 5.01 – Creating a multi-index from columns
Bracket and dot notation
Changing DataFrame values using bracket or dot notation
Summary
8
Chapter 6: Data Selection – Series
Chapter 6: Data Selection – Series
Introduction to pandas Series
The Series index
Preparing Series from DataFrames and vice versa
Activity 6.01 – Series data selection
Understanding the differences between base Python and pandas data selection
Activity 6.02 – DataFrame data selection
Summary
9
Chapter 7: Data Exploration and Transformation
Chapter 7: Data Exploration and Transformation
Introduction to data transformation
Dealing with messy data
Dealing with missing data
Summarizing data
Activity 7.01 – data analysis using pivot tables
Summary
10
Chapter 8: Understanding Data Visualization
Chapter 8: Understanding Data Visualization
Introduction to data visualization
Understanding the basics of pandas visualization
Exploring matplotlib
Visualizing data of different types
Activity 8.01 – Using data visualization for exploratory data analysis
Summary
11
Part 3 – Data Modeling
Part 3 – Data Modeling
12
Chapter 9: Data Modeling – Preprocessing
Chapter 9: Data Modeling – Preprocessing
An introduction to data modeling
Exploring dependent and independent variables
Understanding data scaling and normalization
Activity 9.01 – Data splitting, scaling, and modeling
Summary
13
Chapter 10: Data Modeling – Modeling Basics
Chapter 10: Data Modeling – Modeling Basics
Introduction to data modeling
Learning the modeling basics
Predicting future values of time series
Activity 10.01 – Normalizing and smoothing data
Summary
14
Chapter 11: Data Modeling – Regression Modeling
Chapter 11: Data Modeling – Regression Modeling
An introduction to regression modeling
Exploring regression modeling
Model diagnostics
Activity 11.01 – Multiple regression with non-linear models
Summary
15
Part 4 – Additional Use Cases for pandas
Part 4 – Additional Use Cases for pandas
16
Chapter 12: Using Time in pandas
Chapter 12: Using Time in pandas
Introduction to time series
What are datetimes?
Activity 12.01 – understanding power usage
Datetime math operations
Summary
17
Chapter 13: Exploring Time Series
Chapter 13: Exploring Time Series
The time series as an index
Resampling, grouping, and aggregation by time
Activity 13.01 – Creating a time series model
Summary
18
Chapter 14: Applying pandas Data Processing for Case Studies
Chapter 14: Applying pandas Data Processing for Case Studies
Introduction to the case studies and datasets
Recap of the preprocessing steps
Activity 14.01 – analyzing air quality data
Summary
19
Chapter 15: Appendix
Solution 1.1
Solution 2.1
Solution 3.1
Solution 4.1
Solution 5.1
Solution 6.1
Solution 6.2
Solution 7.1
Solution 8.1
Solution 9.1
Solution 10.1
Solution 11.1
Solution 12.1
Solution 13.1
Solution 14.1
Why subscribe?
20
Other Books You May Enjoy
Other Books You May Enjoy
Packt is searching for authors like you
Share Your Thoughts

Solution 9.1

Perform the following steps to complete the activity:

  1. For this activity, all you will need is the pandas library, some modules from sklearn, and numpy. Load them in the first cell of the notebook:
    import pandas as pd
import numpy as np
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LinearRegression as OLS
from sklearn.metrics import mean_squared_error
  2. Use the power_plant.csv dataset: 'Datasets\\power_plant.csv'. Read the data into a pandas DataFrame, print out the shape, and list the first five rows.

The independent variables are as follows:

  • AT – ambient temperature
  • V – exhaust vacuum level
  • AP – ambient pressure
  • RH – relative humidity

The dependent variable is the following:

  • EP – electrical power produced
    power_data = pd.read_csv('Datasets\\power_plant.csv')
print(power_data.shape...
Previous Section
End of Section 10
Next Section