Book Image

Data Wrangling with Python

By : Dr. Tirthajyoti Sarkar, Shubhadeep Roychowdhury
Book Image

Data Wrangling with Python

By: Dr. Tirthajyoti Sarkar, Shubhadeep Roychowdhury

Overview of this book

For data to be useful and meaningful, it must be curated and refined. Data Wrangling with Python teaches you the core ideas behind these processes and equips you with knowledge of the most popular tools and techniques in the domain. The book starts with the absolute basics of Python, focusing mainly on data structures. It then delves into the fundamental tools of data wrangling like NumPy and Pandas libraries. You'll explore useful insights into why you should stay away from traditional ways of data cleaning, as done in other languages, and take advantage of the specialized pre-built routines in Python. This combination of Python tips and tricks will also demonstrate how to use the same Python backend and extract/transform data from an array of sources including the Internet, large database vaults, and Excel financial tables. To help you prepare for more challenging scenarios, you'll cover how to handle missing or wrong data, and reformat it based on the requirements from the downstream analytics tool. The book will further help you grasp concepts through real-world examples and datasets. By the end of this book, you will be confident in using a diverse array of sources to extract, clean, transform, and format your data efficiently.

Related Content you might be interested in

Current Title:

Small book image
Data Wrangling with Python
Dr. Tirthajyoti Sarkar | Shubhadeep Roychowdhury
Feb, 2019 | 452 pages
Small book image
The Python Workshop
Dr Lau Cher Han | Graham Lee | Mario Corchero | Andrew Bird | Corey Wade
Nov, 2022 | 600 pages
Small book image
The Python Workshop
Dr Lau Cher Han | Graham Lee | Mario Corchero | Andrew Bird | Corey Wade
Nov, 2019 | 608 pages
Small book image
The Python Workshop
Bert Gollnick | Dr Lau Cher Han | Tianxiang Liu | Graham Lee | Roderick Person | Corey Wade | Mario Corchero | Andrew Bird
Aug, 2019 | 618 pages
Table of Contents (12 chapters)
Data Wrangling with Python
Data Wrangling with Python
Preface
Preface
Free Chapter
1
Introduction to Data Wrangling with Python
Introduction to Data Wrangling with Python
Introduction
Python for Data Wrangling
Lists, Sets, Strings, Tuples, and Dictionaries
Summary
2
Advanced Data Structures and File Handling
Advanced Data Structures and File Handling
Introduction
Advanced Data Structures
Basic File Operations in Python
Summary
3
Introduction to NumPy, Pandas, and Matplotlib
Introduction to NumPy, Pandas, and Matplotlib
Introduction
NumPy Arrays
Pandas DataFrames
Statistics and Visualization with NumPy and Pandas
Summary
4
A Deep Dive into Data Wrangling with Python
A Deep Dive into Data Wrangling with Python
Introduction
Subsetting, Filtering, and Grouping
Detecting Outliers and Handling Missing Values
Concatenating, Merging, and Joining
Useful Methods of Pandas
Summary
5
Getting Comfortable with Different Kinds of Data Sources
Getting Comfortable with Different Kinds of Data Sources
Introduction
Reading Data from Different Text-Based (and Non-Text-Based) Sources
Introduction to Beautiful Soup 4 and Web Page Parsing
Summary
6
Learning the Hidden Secrets of Data Wrangling
Learning the Hidden Secrets of Data Wrangling
Introduction
Advanced List Comprehension and the zip Function
Data Formatting
Identify and Clean Outliers
Activity 8: Handling Outliers and Missing Data
Summary
7
Advanced Web Scraping and Data Gathering
Advanced Web Scraping and Data Gathering
Introduction
The Basics of Web Scraping and the Beautiful Soup Library
Reading Data from XML
Reading Data from an API
Fundamentals of Regular Expressions (RegEx)
Summary
8
RDBMS and SQL
RDBMS and SQL
Introduction
Refresher of RDBMS and SQL
Using an RDBMS (MySQL/PostgreSQL/SQLite)
Summary
9
Application of Data Wrangling in Real Life
Application of Data Wrangling in Real Life
Introduction
Applying Your Knowledge to a Real-life Data Wrangling Task
Activity 12: Data Wrangling Task – Fixing UN Data
Activity 13: Data Wrangling Task – Cleaning GDP Data
Activity 14: Data Wrangling Task – Merging UN Data and GDP Data
Activity 15: Data Wrangling Task – Connecting the New Data to the Database
An Extension to Data Wrangling
Summary
Appendix
Appendix

The Basics of Web Scraping and the Beautiful Soup Library

In today's connected world, one of the most valued and widely used skill for a data wrangling professional is the ability to extract and read data from web pages and databases hosted on the web. Most organizations host data on the cloud (public or private), and the majority of web microservices these days provide some kind of API for the external users to access data:

Figure 7.1: Data wrangling HTTP request and an XML/JSON reply

It is necessary that, as a data wrangling engineer, you know about the structure of web pages and Python libraries so that you are able to extract data from a web page. The World Wide Web is an ever-growing, ever-changing universe, in which different data exchange protocols and formats are used. A few of these are widely used and have become standard.

Libraries in Python

Python comes equipped with built-in modules, such as urllib 3, which that can place HTTP requests over the internet and receive data from the...

Previous Section
End of Section 3
Next Section