Book Image

The Data Wrangling Workshop - Second Edition

By : Brian Lipp, Shubhadeep Roychowdhury, Dr. Tirthajyoti Sarkar
Book Image

The Data Wrangling Workshop - Second Edition

By: Brian Lipp, Shubhadeep Roychowdhury, Dr. Tirthajyoti Sarkar

Overview of this book

While a huge amount of data is readily available to us, it is not useful in its raw form. For data to be meaningful, it must be curated and refined. If you’re a beginner, then The Data Wrangling Workshop will help to break down the process for you. You’ll start with the basics and build your knowledge, progressing from the core aspects behind data wrangling, to using the most popular tools and techniques. This book starts by showing you how to work with data structures using Python. Through examples and activities, you’ll understand why you should stay away from traditional methods of data cleaning used in other languages and take advantage of the specialized pre-built routines in Python. Later, you’ll learn how to use the same Python backend to extract and transform data from an array of sources, including the internet, large database vaults, and Excel financial tables. To help you prepare for more challenging scenarios, the book teaches you how to handle missing or incorrect data, and reformat it based on the requirements from your downstream analytics tool. By the end of this book, you will have developed a solid understanding of how to perform data wrangling with Python, and learned several techniques and best practices to extract, clean, transform, and format your data efficiently, from a diverse array of sources.

Related Content you might be interested in

Current Title:

Small book image
The Data Wrangling Workshop - Second Edition
Brian Lipp | Shubhadeep Roychowdhury | Dr. Tirthajyoti Sarkar
Jul, 2020 | 576 pages
Small book image
Mastering Exploratory Analysis with pandas
Harish Garg
Sep, 2018 | 140 pages
Table of Contents (11 chapters)
Preface
About the Book
1
1. Introduction to Data Wrangling with Python
1. Introduction to Data Wrangling with Python
Introduction
Importance of Data Wrangling
Python for Data Wrangling
Lists, Sets, Strings, Tuples, and Dictionaries
List Functions
Summary
Free Chapter
2
2. Advanced Operations on Built-In Data Structures
2. Advanced Operations on Built-In Data Structures
Introduction
Advanced Data Structures
Basic File Operations in Python
Summary
3
3. Introduction to NumPy, Pandas, and Matplotlib
3. Introduction to NumPy, Pandas, and Matplotlib
Introduction
NumPy Arrays
Advanced Mathematical Operations
Statistics and Visualization with NumPy and Pandas
The Definition of Statistical Measures – Central Tendency and Spread
Data Wrangling in Statistics and Visualization
Summary
4
4. A Deep Dive into Data Wrangling with Python
4. A Deep Dive into Data Wrangling with Python
Introduction
Subsetting, Filtering, and Grouping
Detecting Outliers and Handling Missing Values
Concatenating, Merging, and Joining
Useful Methods of Pandas
Summary
5
5. Getting Comfortable with Different Kinds of Data Sources
5. Getting Comfortable with Different Kinds of Data Sources
Introduction
Reading Data from Different Sources
Introduction to Beautiful Soup 4 and Web Page Parsing
Summary
6
6. Learning the Hidden Secrets of Data Wrangling
6. Learning the Hidden Secrets of Data Wrangling
Introduction
Advanced List Comprehension and the zip Function
Data Formatting
Identifying and Cleaning Outliers
Levenshtein Distance
Summary
7
7. Advanced Web Scraping and Data Gathering
7. Advanced Web Scraping and Data Gathering
Introduction
The Requests and BeautifulSoup Libraries
Reading Data from XML
Reading Data from an API
Fundamentals of Regular Expressions (RegEx)
Summary
8
8. RDBMS and SQL
8. RDBMS and SQL
Introduction
Refresher of RDBMS and SQL
Relation Mapping in Databases
Joins
Retrieving Specific Columns from a JOIN Query
Summary
9
9. Applications in Business Use Cases and Conclusion of the Course
9. Applications in Business Use Cases and Conclusion of the Course
Introduction
Applying Your Knowledge to a Data Wrangling Task
An Extension to Data Wrangling
Summary
Appendix
1. Introduction to Data Wrangling with Python
2. Advanced Operations on Built-In Data Structures
3. Introduction to NumPy, Pandas, and Matplotlib
4. A Deep Dive into Data Wrangling with Python
5. Getting Comfortable with Different Kinds of Data Sources
6. Learning the Hidden Secrets of Data Wrangling
7. Advanced Web Scraping and Data Gathering
8. RDBMS and SQL
9. Applications in Business Use Cases and Conclusion of the Course

2. Advanced Operations on Built-In Data Structures

Activity 2.01: Permutation, Iterator, Lambda, and List

Solution:

These are the detailed steps to solve this activity:

  1. Look up the definition of permutations and dropwhile from itertools. There is a way to look up the definition of a function inside Jupyter itself. Just type the function name, followed by ?, and press Shift + Enter:
    from itertools import permutations, dropwhile
permutations?
dropwhile?

    You will see a long list of definitions after each ?. We will skip it here.

  2. Write an expression to generate all the possible three-digit numbers using 1, 2, and 3:
    permutations(range(3))

    The output (which will vary in your case) is as follows:

    <itertools.permutations at 0x7f6c6c077af0>
  3. Loop over the iterator expression you generated before. Use the print method to print each element returned by the iterator. Use assert and isinstance to make sure that the elements are tuples:
    for number_tuple in permutations(range...
Previous Section
End of Section
Next Section