Book Image

Data Wrangling with Python

By : Dr. Tirthajyoti Sarkar, Shubhadeep Roychowdhury
Book Image

Data Wrangling with Python

By: Dr. Tirthajyoti Sarkar, Shubhadeep Roychowdhury

Overview of this book

For data to be useful and meaningful, it must be curated and refined. Data Wrangling with Python teaches you the core ideas behind these processes and equips you with knowledge of the most popular tools and techniques in the domain. The book starts with the absolute basics of Python, focusing mainly on data structures. It then delves into the fundamental tools of data wrangling like NumPy and Pandas libraries. You'll explore useful insights into why you should stay away from traditional ways of data cleaning, as done in other languages, and take advantage of the specialized pre-built routines in Python. This combination of Python tips and tricks will also demonstrate how to use the same Python backend and extract/transform data from an array of sources including the Internet, large database vaults, and Excel financial tables. To help you prepare for more challenging scenarios, you'll cover how to handle missing or wrong data, and reformat it based on the requirements from the downstream analytics tool. The book will further help you grasp concepts through real-world examples and datasets. By the end of this book, you will be confident in using a diverse array of sources to extract, clean, transform, and format your data efficiently.

Related Content you might be interested in

Current Title:

Small book image
Data Wrangling with Python
Dr. Tirthajyoti Sarkar | Shubhadeep Roychowdhury
Feb, 2019 | 452 pages
Small book image
The Python Workshop
Dr Lau Cher Han | Graham Lee | Mario Corchero | Andrew Bird | Corey Wade
Nov, 2022 | 600 pages
Small book image
The Python Workshop
Dr Lau Cher Han | Graham Lee | Mario Corchero | Andrew Bird | Corey Wade
Nov, 2019 | 608 pages
Small book image
The Python Workshop
Bert Gollnick | Dr Lau Cher Han | Tianxiang Liu | Graham Lee | Roderick Person | Corey Wade | Mario Corchero | Andrew Bird
Aug, 2019 | 618 pages
Table of Contents (12 chapters)
Data Wrangling with Python
Data Wrangling with Python
Preface
Preface
Free Chapter
1
Introduction to Data Wrangling with Python
Introduction to Data Wrangling with Python
Introduction
Python for Data Wrangling
Lists, Sets, Strings, Tuples, and Dictionaries
Summary
2
Advanced Data Structures and File Handling
Advanced Data Structures and File Handling
Introduction
Advanced Data Structures
Basic File Operations in Python
Summary
3
Introduction to NumPy, Pandas, and Matplotlib
Introduction to NumPy, Pandas, and Matplotlib
Introduction
NumPy Arrays
Pandas DataFrames
Statistics and Visualization with NumPy and Pandas
Summary
4
A Deep Dive into Data Wrangling with Python
A Deep Dive into Data Wrangling with Python
Introduction
Subsetting, Filtering, and Grouping
Detecting Outliers and Handling Missing Values
Concatenating, Merging, and Joining
Useful Methods of Pandas
Summary
5
Getting Comfortable with Different Kinds of Data Sources
Getting Comfortable with Different Kinds of Data Sources
Introduction
Reading Data from Different Text-Based (and Non-Text-Based) Sources
Introduction to Beautiful Soup 4 and Web Page Parsing
Summary
6
Learning the Hidden Secrets of Data Wrangling
Learning the Hidden Secrets of Data Wrangling
Introduction
Advanced List Comprehension and the zip Function
Data Formatting
Identify and Clean Outliers
Activity 8: Handling Outliers and Missing Data
Summary
7
Advanced Web Scraping and Data Gathering
Advanced Web Scraping and Data Gathering
Introduction
The Basics of Web Scraping and the Beautiful Soup Library
Reading Data from XML
Reading Data from an API
Fundamentals of Regular Expressions (RegEx)
Summary
8
RDBMS and SQL
RDBMS and SQL
Introduction
Refresher of RDBMS and SQL
Using an RDBMS (MySQL/PostgreSQL/SQLite)
Summary
9
Application of Data Wrangling in Real Life
Application of Data Wrangling in Real Life
Introduction
Applying Your Knowledge to a Real-life Data Wrangling Task
Activity 12: Data Wrangling Task – Fixing UN Data
Activity 13: Data Wrangling Task – Cleaning GDP Data
Activity 14: Data Wrangling Task – Merging UN Data and GDP Data
Activity 15: Data Wrangling Task – Connecting the New Data to the Database
An Extension to Data Wrangling
Summary
Appendix
Appendix

Summary

In this topic, we looked at the structure of an HTML document. HTML documents are the cornerstone of the World Wide Web and, given the amount of data that's contained on it, we can easily infer the importance of HTML as a data source.

We learned about bs4 (BeautifulSoup4), a Python library that gives us Pythonic ways to read and query HTML documents. We used bs4 to load an HTML document and also explored several different ways to navigate the loaded document. We also got necessary information about the difference between all of these methods.

We looked at how we can create a pandas DataFrame from an HTML document (which contains a table). Although there are some built-in ways to do this job in pandas, they fail as soon as the target table is encoded inside a complex hierarchy of elements. So, the knowledge we gathered in this topic by transforming an HTML table into a pandas DataFrame in a step-by-step manner is invaluable.

Finally, we looked at how we can create a stack in our code...

Previous Section
End of Chapter 5
Next Chapter