Book Image

Hands-On Exploratory Data Analysis with Python

By : Suresh Kumar Mukhiya, Usman Ahmed
Book Image

Hands-On Exploratory Data Analysis with Python

By: Suresh Kumar Mukhiya, Usman Ahmed

Overview of this book

Exploratory Data Analysis (EDA) is an approach to data analysis that involves the application of diverse techniques to gain insights into a dataset. This book will help you gain practical knowledge of the main pillars of EDA - data cleaning, data preparation, data exploration, and data visualization. You’ll start by performing EDA using open source datasets and perform simple to advanced analyses to turn data into meaningful insights. You’ll then learn various descriptive statistical techniques to describe the basic characteristics of data and progress to performing EDA on time-series data. As you advance, you’ll learn how to implement EDA techniques for model development and evaluation and build predictive models to visualize results. Using Python for data analysis, you’ll work with real-world datasets, understand data, summarize its characteristics, and visualize it for business intelligence. By the end of this EDA book, you’ll have developed the skills required to carry out a preliminary investigation on any dataset, yield insights into data, present your results with visual aids, and build a model that correctly predicts future outcomes.
Table of Contents (17 chapters)
1
Section 1: The Fundamentals of EDA
6
Section 2: Descriptive Statistics
11
Section 3: Model Development and Evaluation

Using pandas vectorized string functions

For string formatting, it would be better to use a dataset that's a little messier. We will use the dataset that I collected during my Ph.D. research study when writing a review paper. It can be found here: https://raw.githubusercontent.com/sureshHARDIYA/phd-resources/master/Data/Review%20Paper/preprocessed.csv.

  1. Let's load this text article and then display the first eight entries. Let's start by loading the data and checking its structure and a few of the comments, as follows:
import numpy as np
import pandas as pd
import os

  1. Next, let's read the text file and display the last 10 items, as follows:
text = pd.read_csv("https://raw.githubusercontent.com/sureshHARDIYA/phd-resources/master/Data/Review%20Paper/preprocessed.csv")
text = text["TITLE"]
print (text.shape)
print( text.tail(10))
  1. The output...