Book Image

Data Wrangling with Python

By : Dr. Tirthajyoti Sarkar, Shubhadeep Roychowdhury
Book Image

Data Wrangling with Python

By: Dr. Tirthajyoti Sarkar, Shubhadeep Roychowdhury

Overview of this book

For data to be useful and meaningful, it must be curated and refined. Data Wrangling with Python teaches you the core ideas behind these processes and equips you with knowledge of the most popular tools and techniques in the domain. The book starts with the absolute basics of Python, focusing mainly on data structures. It then delves into the fundamental tools of data wrangling like NumPy and Pandas libraries. You'll explore useful insights into why you should stay away from traditional ways of data cleaning, as done in other languages, and take advantage of the specialized pre-built routines in Python. This combination of Python tips and tricks will also demonstrate how to use the same Python backend and extract/transform data from an array of sources including the Internet, large database vaults, and Excel financial tables. To help you prepare for more challenging scenarios, you'll cover how to handle missing or wrong data, and reformat it based on the requirements from the downstream analytics tool. The book will further help you grasp concepts through real-world examples and datasets. By the end of this book, you will be confident in using a diverse array of sources to extract, clean, transform, and format your data efficiently.

Related Content you might be interested in

Current Title:

Small book image
Data Wrangling with Python
Dr. Tirthajyoti Sarkar | Shubhadeep Roychowdhury
Feb, 2019 | 452 pages
Small book image
The Python Workshop
Dr Lau Cher Han | Graham Lee | Mario Corchero | Andrew Bird | Corey Wade
Nov, 2022 | 600 pages
Small book image
The Python Workshop
Dr Lau Cher Han | Graham Lee | Mario Corchero | Andrew Bird | Corey Wade
Nov, 2019 | 608 pages
Small book image
The Python Workshop
Bert Gollnick | Dr Lau Cher Han | Tianxiang Liu | Graham Lee | Roderick Person | Corey Wade | Mario Corchero | Andrew Bird
Aug, 2019 | 618 pages
Table of Contents (12 chapters)
Data Wrangling with Python
Data Wrangling with Python
Preface
Preface
Free Chapter
1
Introduction to Data Wrangling with Python
Introduction to Data Wrangling with Python
Introduction
Python for Data Wrangling
Lists, Sets, Strings, Tuples, and Dictionaries
Summary
2
Advanced Data Structures and File Handling
Advanced Data Structures and File Handling
Introduction
Advanced Data Structures
Basic File Operations in Python
Summary
3
Introduction to NumPy, Pandas, and Matplotlib
Introduction to NumPy, Pandas, and Matplotlib
Introduction
NumPy Arrays
Pandas DataFrames
Statistics and Visualization with NumPy and Pandas
Summary
4
A Deep Dive into Data Wrangling with Python
A Deep Dive into Data Wrangling with Python
Introduction
Subsetting, Filtering, and Grouping
Detecting Outliers and Handling Missing Values
Concatenating, Merging, and Joining
Useful Methods of Pandas
Summary
5
Getting Comfortable with Different Kinds of Data Sources
Getting Comfortable with Different Kinds of Data Sources
Introduction
Reading Data from Different Text-Based (and Non-Text-Based) Sources
Introduction to Beautiful Soup 4 and Web Page Parsing
Summary
6
Learning the Hidden Secrets of Data Wrangling
Learning the Hidden Secrets of Data Wrangling
Introduction
Advanced List Comprehension and the zip Function
Data Formatting
Identify and Clean Outliers
Activity 8: Handling Outliers and Missing Data
Summary
7
Advanced Web Scraping and Data Gathering
Advanced Web Scraping and Data Gathering
Introduction
The Basics of Web Scraping and the Beautiful Soup Library
Reading Data from XML
Reading Data from an API
Fundamentals of Regular Expressions (RegEx)
Summary
8
RDBMS and SQL
RDBMS and SQL
Introduction
Refresher of RDBMS and SQL
Using an RDBMS (MySQL/PostgreSQL/SQLite)
Summary
9
Application of Data Wrangling in Real Life
Application of Data Wrangling in Real Life
Introduction
Applying Your Knowledge to a Real-life Data Wrangling Task
Activity 12: Data Wrangling Task – Fixing UN Data
Activity 13: Data Wrangling Task – Cleaning GDP Data
Activity 14: Data Wrangling Task – Merging UN Data and GDP Data
Activity 15: Data Wrangling Task – Connecting the New Data to the Database
An Extension to Data Wrangling
Summary
Appendix
Appendix

Introduction

This chapter of our data journey is focused on RDBMS (Relational Database Management Systems) and SQL (Structured Query Language). In the previous chapter, we stored and read data from a file. In this chapter, we will read structured data, design access to the data, and create query interfaces for databases.

Data has been stored in RDBMS format for years. The reasons behind it are as follows:

  • RDBMS is one of the safest ways to store, manage, and retrieve data.

  • They are backed by a solid mathematical foundation (relational algebra and calculus) and they expose an efficient and intuitive declarative language – SQL – for easy interaction.

  • Almost every language has a rich set of libraries to interact with different RDBMS and the tricks and methods of using them are well tested and well understood.

  • Scaling an RDBMS is a pretty well-understood task and there are a bunch of well trained, experienced professionals to do this job (DBA or database administrator).

As we can see in the following...

Previous Section
End of Section 2
Next Section