Book Image

Hands-On Data Analysis with Pandas - Second Edition

By : Stefanie Molin
5 (1)
Book Image

Hands-On Data Analysis with Pandas - Second Edition

5 (1)
By: Stefanie Molin

Overview of this book

Extracting valuable business insights is no longer a ‘nice-to-have’, but an essential skill for anyone who handles data in their enterprise. Hands-On Data Analysis with Pandas is here to help beginners and those who are migrating their skills into data science get up to speed in no time. This book will show you how to analyze your data, get started with machine learning, and work effectively with the Python libraries often used for data science, such as pandas, NumPy, matplotlib, seaborn, and scikit-learn. Using real-world datasets, you will learn how to use the pandas library to perform data wrangling to reshape, clean, and aggregate your data. Then, you will learn how to conduct exploratory data analysis by calculating summary statistics and visualizing the data to find patterns. In the concluding chapters, you will explore some applications of anomaly detection, regression, clustering, and classification using scikit-learn to make predictions based on past data. This updated edition will equip you with the skills you need to use pandas 1.x to efficiently perform various data manipulation tasks, reliably reproduce analyses, and visualize your data for effective decision making – valuable knowledge that can be applied across multiple domains.

Related Content you might be interested in

Current Title:

Small book image
Hands-On Data Analysis with Pandas - Second Edition
Stefanie Molin
Apr, 2021 | 788 pages
Small book image
Hands-On Exploratory Data Analysis with Python
Suresh Kumar Mukhiya | Usman Ahmed
Mar, 2020 | 352 pages
Small book image
Learning pandas.
Michael Heydt
Jun, 2017 | 446 pages
Small book image
Data Cleaning and Exploration with Machine Learning
Michael Walker
Aug, 2022 | 542 pages
Table of Contents (21 chapters)
Preface
Preface
Who this book is for
What this book covers
To get the most out of this book
Download the color images
Conventions used
Get in touch
Reviews
1
Section 1: Getting Started with Pandas
Section 1: Getting Started with Pandas
Free Chapter
2
Chapter 1: Introduction to Data Analysis
Chapter 1: Introduction to Data Analysis
Chapter materials
The fundamentals of data analysis
Statistical foundations
Setting up a virtual environment
Summary
Exercises
Further reading
3
Chapter 2: Working with Pandas DataFrames
Chapter 2: Working with Pandas DataFrames
Chapter materials
Pandas data structures
Creating a pandas DataFrame
Inspecting a DataFrame object
Grabbing subsets of the data
Adding and removing data
Summary
Exercises
Further reading
4
Section 2: Using Pandas for Data Analysis
Section 2: Using Pandas for Data Analysis
5
Chapter 3: Data Wrangling with Pandas
Chapter 3: Data Wrangling with Pandas
Chapter materials
Understanding data wrangling
Exploring an API to find and collect temperature data
Cleaning data
Reshaping data
Handling duplicate, missing, or invalid data
Summary
Exercises
Further reading
6
Chapter 4: Aggregating Pandas DataFrames
Chapter 4: Aggregating Pandas DataFrames
Chapter materials
Performing database-style operations on DataFrames
Using DataFrame operations to enrich data
Aggregating data
Working with time series data
Summary
Exercises
Further reading
7
Chapter 5: Visualizing Data with Pandas and Matplotlib
Chapter 5: Visualizing Data with Pandas and Matplotlib
Chapter materials
An introduction to matplotlib
Plotting with pandas
The pandas.plotting module
Summary
Exercises
Further reading
8
Chapter 6: Plotting with Seaborn and Customization Techniques
Chapter 6: Plotting with Seaborn and Customization Techniques
Chapter materials
Utilizing seaborn for advanced plotting
Formatting plots with matplotlib
Customizing visualizations
Summary
Exercises
Further reading
9
Section 3: Applications – Real-World Analyses Using Pandas
Section 3: Applications – Real-World Analyses Using Pandas
10
Chapter 7: Financial Analysis – Bitcoin and the Stock Market
Chapter 7: Financial Analysis – Bitcoin and the Stock Market
Chapter materials
Building a Python package
Collecting financial data
Exploratory data analysis
Technical analysis of financial instruments
Modeling performance using historical data
Summary
Exercises
Further reading
11
Chapter 8: Rule-Based Anomaly Detection
Chapter 8: Rule-Based Anomaly Detection
Chapter materials
Simulating login attempts
Exploratory data analysis
Implementing rule-based anomaly detection
Summary
Exercises
Further reading
12
Section 4: Introduction to Machine Learning with Scikit-Learn
Section 4: Introduction to Machine Learning with Scikit-Learn
13
Chapter 9: Getting Started with Machine Learning in Python
Chapter 9: Getting Started with Machine Learning in Python
Chapter materials
Overview of the machine learning landscape
Exploratory data analysis
Preprocessing data
Clustering
Regression
Classification
Summary
Exercises
Further reading
14
Chapter 10: Making Better Predictions – Optimizing Models
Chapter 10: Making Better Predictions – Optimizing Models
Chapter materials
Hyperparameter tuning with grid search
Feature engineering
Ensemble methods
Inspecting classification prediction confidence
Addressing class imbalance
Regularization
Summary
Exercises
Further reading
15
Chapter 11: Machine Learning Anomaly Detection
Chapter 11: Machine Learning Anomaly Detection
Chapter materials
Exploring the simulated login attempts data
Utilizing unsupervised methods of anomaly detection
Implementing supervised anomaly detection
Incorporating a feedback loop with online learning
Summary
Exercises
Further reading
16
Section 5: Additional Resources
Section 5: Additional Resources
17
Chapter 12: The Road Ahead
Chapter 12: The Road Ahead
Data resources
Practicing working with data
Python practice
Summary
Exercises
Further reading
18
Solutions
Solutions
19
Other Books You May Enjoy
Other Books You May Enjoy
Packt is searching for authors like you
Leave a review - let other readers know what you think
Appendix
Data analysis workflow
Choosing the appropriate visualization
Machine learning workflow
Why subscribe?

Using DataFrame operations to enrich data

Now that we've discussed how to query and merge DataFrame objects, let's learn how to perform complex operations on them to create and modify columns and rows. For this section, we will be working in the 2-dataframe_operations.ipynb notebook using the weather data, along with Facebook stock's volume traded and opening, high, low, and closing prices daily for 2018. Let's import what we will need and read in the data:

>>> import numpy as np
>>> import pandas as pd
>>> weather = pd.read_csv(
...     'data/nyc_weather_2018.csv', parse_dates=['date']
... )
>>> fb = pd.read_csv(
...     'data/fb_2018.csv', index_col='date', parse_dates=True
... )

We will begin by reviewing operations that summarize entire rows and columns before moving on to binning, applying functions across rows and columns, and...

Previous Section
End of Section 4
Next Section