Browse Library
Sign In Start Free Trial
Close icon Search icon
Account
Browse Library Advanced Search Sign In Start Free Trial

Add to playlist

Create a Playlist

Modal Close icon
You need to login to use this feature.
  • Book Overview & Buying Python Data Analysis
  • Table Of Contents Toc
  • Feedback & Rating feedback
Python Data Analysis

Python Data Analysis

By : Idris
4.5 (13)
Buy this Book
close
close
Python Data Analysis

Python Data Analysis

4.5 (13)
By: Idris
Buy this Book

Overview of this book

This book is for programmers, scientists, and engineers who have knowledge of the Python language and know the basics of data science. It is for those who wish to learn different data analysis methods using Python and its libraries. This book contains all the basic ingredients you need to become an expert data analyst.
Table of Contents (20 chapters)
close
close
close
Preface
Icon
Preface
Icon
Who this book is for
Icon
What this book covers
Icon
To get the most out of this book
Icon
Get in touch
Lock Free Chapter
1
Section 1: Foundation for Data Analysis
Icon
Section 1: Foundation for Data Analysis
2
Getting Started with Python Libraries
Icon
Getting Started with Python Libraries
Icon
Understanding data analysis
Icon
The standard process of data analysis
Icon
The KDD process
Icon
SEMMA
Icon
CRISP-DM
Icon
Comparing data analysis and data science
Icon
The skillsets of data analysts and data scientists
Icon
Installing Python 3
Icon
Software used in this book
Icon
Using IPython as a shell
Icon
Using JupyterLab
Icon
Using Jupyter Notebooks
Icon
Advanced features of Jupyter Notebooks
Icon
Summary
3
NumPy and pandas
Icon
NumPy and pandas
Icon
Technical requirements
Icon
Understanding NumPy arrays
Icon
NumPy array numerical data types
Icon
Manipulating array shapes
Icon
The stacking of NumPy arrays
Icon
Partitioning NumPy arrays
Icon
Changing the data type of NumPy arrays
Icon
Creating NumPy views and copies
Icon
Slicing NumPy arrays
Icon
Boolean and fancy indexing
Icon
Broadcasting arrays
Icon
Creating pandas DataFrames
Icon
Understanding pandas Series
Icon
Reading and querying the Quandl data
Icon
Describing pandas DataFrames
Icon
Grouping and joining pandas DataFrame
Icon
Working with missing values
Icon
Creating pivot tables
Icon
Dealing with dates
Icon
Summary
Icon
References
4
Statistics
Icon
Statistics
Icon
Technical requirements
Icon
Understanding attributes and their types
Icon
Measuring central tendency
Icon
Measuring dispersion
Icon
Skewness and kurtosis
Icon
Understanding relationships using covariance and correlation coefficients
Icon
Central limit theorem
Icon
Collecting samples
Icon
Performing parametric tests
Icon
Performing non-parametric tests
Icon
Summary
5
Linear Algebra
Icon
Linear Algebra
Icon
Technical requirements
Icon
Fitting to polynomials with NumPy
Icon
Determinant
Icon
Finding the rank of a matrix
Icon
Matrix inverse using NumPy
Icon
Solving linear equations using NumPy
Icon
Decomposing a matrix using SVD
Icon
Eigenvectors and Eigenvalues using NumPy
Icon
Generating random numbers
Icon
Binomial distribution
Icon
Normal distribution
Icon
Testing normality of data using SciPy
Icon
Creating a masked array using the numpy.ma subpackage
Icon
Summary
6
Section 2: Exploratory Data Analysis and Data Cleaning
Icon
Section 2: Exploratory Data Analysis and Data Cleaning
7
Data Visualization
Icon
Data Visualization
Icon
Technical requirements
Icon
Visualization using Matplotlib
Icon
Advanced visualization using the Seaborn package
Icon
Interactive visualization with Bokeh
Icon
Summary
8
Retrieving, Processing, and Storing Data
Icon
Retrieving, Processing, and Storing Data
Icon
Technical requirements
Icon
Reading and writing CSV files with NumPy
Icon
Reading and writing CSV files with pandas
Icon
Reading and writing data from Excel
Icon
Reading and writing data from JSON
Icon
Reading and writing data from HDF5
Icon
Reading and writing data from HTML tables
Icon
Reading and writing data from Parquet
Icon
Reading and writing data from a pickle pandas object
Icon
Lightweight access with sqllite3
Icon
Reading and writing data from MySQL
Icon
Reading and writing data from MongoDB
Icon
Reading and writing data from Cassandra
Icon
Reading and writing data from Redis
Icon
PonyORM
Icon
Summary
9
Cleaning Messy Data
chevron up
Icon
Cleaning Messy Data
Icon
Technical requirements
Icon
Exploring data
Icon
Filtering data to weed out the noise
Icon
Handling missing values
Icon
Handling outliers
Icon
Feature encoding techniques
Icon
Feature scaling
Icon
Feature transformation
Icon
Feature splitting
Icon
Summary
10
Signal Processing and Time Series
Icon
Signal Processing and Time Series
Icon
Technical requirements
Icon
The statsmodels modules
Icon
Moving averages
Icon
Window functions
Icon
Defining cointegration
Icon
STL decomposition
Icon
Autocorrelation
Icon
Autoregressive models
Icon
ARMA models
Icon
Generating periodic signals
Icon
Fourier analysis
Icon
Spectral analysis filtering
Icon
Summary
11
Section 3: Deep Dive into Machine Learning
Icon
Section 3: Deep Dive into Machine Learning
12
Supervised Learning - Regression Analysis
Icon
Supervised Learning - Regression Analysis
Icon
Technical requirements
Icon
Linear regression
Icon
Understanding multicollinearity
Icon
Dummy variables
Icon
Developing a linear regression model
Icon
Evaluating regression model performance
Icon
Fitting polynomial regression
Icon
Regression models for classification
Icon
Logistic regression
Icon
Implementing logistic regression using scikit-learn
Icon
Summary
13
Supervised Learning - Classification Techniques
Icon
Supervised Learning - Classification Techniques
Icon
Technical requirements
Icon
Classification
Icon
Naive Bayes classification
Icon
Decision tree classification
Icon
KNN classification
Icon
SVM classification
Icon
Splitting training and testing sets
Icon
Evaluating the classification model performance
Icon
ROC curve and AUC
Icon
Summary
14
Unsupervised Learning - PCA and Clustering
Icon
Unsupervised Learning - PCA and Clustering
Icon
Technical requirements
Icon
Unsupervised learning
Icon
Reducing the dimensionality of data
Icon
Clustering
Icon
Partitioning data using k-means clustering
Icon
Hierarchical clustering
Icon
DBSCAN clustering
Icon
Spectral clustering
Icon
Evaluating clustering performance
Icon
Summary
15
Section 4: NLP, Image Analytics, and Parallel Computing
Icon
Section 4: NLP, Image Analytics, and Parallel Computing
16
Analyzing Textual Data
Icon
Analyzing Textual Data
Icon
Technical requirements
Icon
Installing NLTK and SpaCy
Icon
Text normalization
Icon
Tokenization
Icon
Removing stopwords
Icon
Stemming and lemmatization
Icon
POS tagging
Icon
Recognizing entities
Icon
Dependency parsing
Icon
Creating a word cloud
Icon
Bag of Words
Icon
TF-IDF
Icon
Sentiment analysis using text classification
Icon
Text similarity
Icon
Summary
17
Analyzing Image Data
Icon
Analyzing Image Data
Icon
Technical requirements
Icon
Installing OpenCV
Icon
Understanding image data
Icon
Color models
Icon
Drawing on images
Icon
Writing on images
Icon
Resizing images
Icon
Flipping images
Icon
Changing the brightness
Icon
Blurring an image
Icon
Face detection
Icon
Summary
18
Parallel Computing Using Dask
Icon
Parallel Computing Using Dask
Icon
Parallel computing using Dask
Icon
Dask data types
Icon
Dask Delayed
Icon
Preprocessing data at scale
Icon
Machine learning at scale
Icon
Summary
19
Other Books You May Enjoy
Icon
Other Books You May Enjoy
Icon
Leave a review - let other readers know what you think

Analyzing word frequencies

The NLTK FreqDist class encapsulates a dictionary of words and counts for a given list of words. Load the Gutenberg text of Julius Caesar by William Shakespeare. Let's filter out stopwords and punctuation:

punctuation = set(string.punctuation)
filtered = [w.lower() for w in words if w.lower() not in sw and w.lower() not in punctuation]

Create a FreqDist object and print associated keys and values with highest frequency:

fd = nltk.FreqDist(filtered)
print "Words", fd.keys()[:5]
print "Counts", fd.values()[:5]

The keys and values are printed as follows:

Words ['d', 'caesar', 'brutus', 'bru', 'haue']
Counts [215, 190, 161, 153, 148]

The first word in this list is of course not an English word, so we may need to add the heuristic that words have a minimum of two characters. The NLTK FreqDist class allows dictionary-like access, but it also has convenience methods. Get the word with the most frequent word and the related count:

print "Max", fd.max()
print "Count", fd['d...
Visually different images
CONTINUE READING
83
Tech Concepts
36
Programming languages
73
Tech Tools
Icon Unlimited access to the largest independent learning library in tech of over 8,000 expert-authored tech books and videos.
Icon Innovative learning tools, including AI book assistants, code context explainers, and text-to-speech.
Icon 50+ new titles added per month and exclusive early access to books as they are being written.
Python Data Analysis
End of Section 9
Next Section
notes
bookmark Notes and Bookmarks search Search in title playlist Add to playlist font-size Font size

Change the font size

margin-width Margin width

Change margin width

day-mode Day/Sepia/Night Modes

Change background colour

Close icon Search
Country selected
Close icon Your notes and bookmarks

Confirmation

Modal Close icon
claim successful
Order Page Read Now

Buy this book with your credits?

Modal Close icon
Are you sure you want to buy this book with one of your credits?
Close
YES, BUY

Submit Your Feedback

Modal Close icon
Modal Close icon
Modal Close icon