Browse Library
Sign In Start Free Trial
Close icon Search icon
Account
Browse Library Sign In Start Free Trial

Add to playlist

Create a Playlist

Modal Close icon
You need to login to use this feature.
  • Book Overview & Buying Pandas Cookbook
  • Table Of Contents Toc
Pandas Cookbook

Pandas Cookbook - Third Edition

By : William Ayd, Matthew Harrison
4.9 (10)
Buy this Book
close
close
Pandas Cookbook

Pandas Cookbook

4.9 (10)
By: William Ayd, Matthew Harrison
Buy this Book

Overview of this book

Unlock the full power of pandas 2.x with this hands-on cookbook, designed for Python developers, data analysts, and data scientists who need fast, efficient solutions for real-world data challenges. This book provides practical, ready-to-use recipes to streamline your workflow. With step-by-step guidance, you'll master data wrangling, visualization, performance optimization, and scalable data analysis using pandas’ most powerful features. From importing and merging large datasets to advanced time series analysis and SQL-like operations, this cookbook equips you with the tools to analyze, manipulate, and visualize data like a pro. Learn how to boost efficiency, optimize memory usage, and seamlessly integrate pandas with NumPy, PyArrow, and databases. This book will help you transform raw data into actionable insights with ease. *Email sign-up and proof of purchase required
Table of Contents (14 chapters)
close
close
close
Preface
Icon
Preface
Icon
Who this book is for
Icon
What this book covers
Icon
To get the most out of this book
Icon
Sections
Icon
Get in touch
Icon
Reviews
Icon
Leave a Review!
Icon
Making the Most Out of This Book – Get to Know Your Free Benefits
Lock Free Chapter
1
pandas Foundations
Icon
pandas Foundations
Icon
Importing pandas
Icon
Series
Icon
DataFrame
Icon
Index
Icon
Series attributes
Icon
DataFrame attributes
2
Selection and Assignment
Icon
Selection and Assignment
Icon
Basic selection from a Series
Icon
Basic selection from a DataFrame
Icon
Position-based selection of a Series
Icon
Position-based selection of a DataFrame
Icon
Label-based selection from a Series
Icon
Label-based selection from a DataFrame
Icon
Mixing position-based and label-based selection
Icon
DataFrame.filter
Icon
Selection by data type
Icon
Selection/filtering via Boolean arrays
Icon
Selection with a MultiIndex – A single level
Icon
Selection with a MultiIndex – Multiple levels
Icon
Selection with a MultiIndex – a DataFrame
Icon
Item assignment with .loc and .iloc
Icon
DataFrame column assignment
3
Data Types
Icon
Data Types
Icon
Integral types
Icon
Floating point types
Icon
Boolean types
Icon
String types
Icon
Missing value handling
Icon
Categorical types
Icon
Temporal types – datetime
Icon
Temporal types – timedelta
Icon
Temporal PyArrow types
Icon
PyArrow List types
Icon
PyArrow decimal types
Icon
NumPy type system, the object type, and pitfalls
4
The pandas I/O System
Icon
The pandas I/O System
Icon
CSV – basic reading/writing
Icon
CSV – strategies for reading large files
Icon
Microsoft Excel – basic reading/writing
Icon
Microsoft Excel – finding tables in non-default locations
Icon
Microsoft Excel – hierarchical data
Icon
SQL using SQLAlchemy
Icon
SQL using ADBC
Icon
Apache Parquet
Icon
JSON
Icon
HTML
Icon
Pickle
Icon
Third-party I/O libraries
5
Algorithms and How to Apply Them
Icon
Algorithms and How to Apply Them
Icon
Basic pd.Series arithmetic
Icon
Basic pd.DataFrame arithmetic
Icon
Aggregations
Icon
Transformations
Icon
Map
Icon
Apply
Icon
Summary statistics
Icon
Binning algorithms
Icon
One-hot encoding with pd.get_dummies
Icon
Chaining with .pipe
Icon
Selecting the lowest-budget movies from the top 100
Icon
Calculating a trailing stop order price
Icon
Finding the baseball players best at…
Icon
Understanding which position scores the most per tea
6
Visualization
Icon
Visualization
Icon
Creating charts from aggregated data
Icon
Plotting distributions of non-aggregated data
Icon
Further plot customization with Matplotlib
Icon
Exploring scatter plots
Icon
Exploring categorical data
Icon
Exploring continuous data
Icon
Using seaborn for advanced plots
7
Reshaping DataFrames
Icon
Reshaping DataFrames
Icon
Concatenating pd.DataFrame objects
Icon
Merging DataFrames with pd.merge
Icon
Joining DataFrames with pd.DataFrame.join
Icon
Reshaping with pd.DataFrame.stack and pd.DataFrame.unstack
Icon
Reshaping with pd.DataFrame.melt
Icon
Reshaping with pd.wide_to_long
Icon
Reshaping with pd.DataFrame.pivot and pd.pivot_table
Icon
Reshaping with pd.DataFrame.explode
Icon
Transposing with pd.DataFrame.T
8
Group By
Icon
Group By
Icon
Group by basics
Icon
Grouping and calculating multiple columns
Icon
Group by apply
Icon
Window operations
Icon
Selecting the highest rated movies by year
Icon
Comparing the best hitter in baseball across years
9
Temporal Data Types and Algorithms
Icon
Temporal Data Types and Algorithms
Icon
Timezone handling
Icon
DateOffsets
Icon
Datetime selection
Icon
Resampling
Icon
Aggregating weekly crime and traffic accidents
Icon
Calculating year-over-year changes in crime by category
Icon
Accurately measuring sensor-collected events with missing values
10
General Usage and Performance Tips
chevron up
Icon
General Usage and Performance Tips
Icon
Avoid dtype=object
Icon
Be cognizant of data sizes
Icon
Use vectorized functions instead of loops
Icon
Avoid mutating data
Icon
Dictionary-encode low cardinality data
Icon
Test-driven development features
11
The pandas Ecosystem
Icon
The pandas Ecosystem
Icon
Foundational libraries
Icon
Exploratory data analysis
Icon
Data validation
Icon
Visualization
Icon
Data science
Icon
Databases
Icon
Other DataFrame libraries
12
Other Books You May Enjoy
Icon
Other Books You May Enjoy
13
Index
Icon
Index

Be cognizant of data sizes

As your datasets grow larger, you may find that you have to pick more optimal data types to ensure your pd.DataFrame can still fit into memory.

Back in Chapter 3, Data Types, we discussed the different integral types and how they are a trade-off between memory usage and capacity. When dealing with untyped data sources like CSV and Excel files, pandas will err on the side of using too much memory as opposed to picking the wrong capacity. This conservative approach can lead to inefficient usage of your system’s memory, so knowing how to optimize that can make the difference between loading a file and receiving an OutOfMemory error.

How to do it

To illustrate the impact of picking proper data types, let’s start with a relatively large pd.DataFrame composed of Python integers:

df = pd.DataFrame({
    "a": [0] * 100_000,
    "b": [2 ** 8] * 100_000,
    "c": [2 ** 16] * 100_000,
    "d": [2...
CONTINUE READING
83
Tech Concepts
36
Programming languages
73
Tech Tools
Icon Unlimited access to the largest independent learning library in tech of over 8,000 expert-authored tech books and videos.
Icon Innovative learning tools, including AI book assistants, code context explainers, and text-to-speech.
Icon 50+ new titles added per month and exclusive early access to books as they are being written.
Pandas Cookbook
End of Section 10
Next Section
notes
bookmark Notes and Bookmarks search Search in title playlist Add to playlist download Download options
font-size Font size

Change the font size

margin-width Margin width

Change margin width

day-mode Day/Sepia/Night Modes

Change background colour

Close icon Search
Country selected
Close icon Your notes and bookmarks

Confirmation

Modal Close icon
claim successful
Order Page Read Now

Buy this book with your credits?

Modal Close icon
Are you sure you want to buy this book with one of your credits?
Close
YES, BUY

Submit Your Feedback

Modal Close icon
Modal Close icon
Modal Close icon