Book Image

Python Data Analysis

By : Ivan Idris
Book Image

Python Data Analysis

By: Ivan Idris

Overview of this book

Related Content you might be interested in

Current Title:

Small book image
Python Data Analysis
Ivan Idris
Oct, 2014 | 348 pages
No titles found
Table of Contents (22 chapters)
Python Data Analysis
Python Data Analysis
Credits
Credits
About the Author
About the Author
About the Reviewers
About the Reviewers
www.PacktPub.com
www.PacktPub.com
Preface
Preface
Free Chapter
1
Getting Started with Python Libraries
Getting Started with Python Libraries
Software used in this book
Building NumPy, SciPy, matplotlib, and IPython from source
Installing with setuptools
NumPy arrays
A simple application
Using IPython as a shell
Reading manual pages
IPython notebooks
Where to find help and references
Summary
2
NumPy Arrays
NumPy Arrays
The NumPy array object
Creating a multidimensional array
Selecting NumPy array elements
NumPy numerical types
One-dimensional slicing and indexing
Manipulating array shapes
Creating array views and copies
Fancy indexing
Indexing with a list of locations
Indexing NumPy arrays with Booleans
Broadcasting NumPy arrays
Summary
3
Statistics and Linear Algebra
Statistics and Linear Algebra
NumPy and SciPy modules
Basic descriptive statistics with NumPy
Linear algebra with NumPy
Finding eigenvalues and eigenvectors with NumPy
NumPy random numbers
Creating a NumPy-masked array
Summary
4
pandas Primer
pandas Primer
Installing and exploring pandas
pandas DataFrames
pandas Series
Querying data in pandas
Statistics with pandas DataFrames
Data aggregation with pandas DataFrames
Concatenating and appending DataFrames
Joining DataFrames
Handling missing values
Dealing with dates
Pivot tables
Remote data access
Summary
5
Retrieving, Processing, and Storing Data
Retrieving, Processing, and Storing Data
Writing CSV files with NumPy and pandas
Comparing the NumPy .npy binary format and pickling pandas DataFrames
Storing data with PyTables
Reading and writing pandas DataFrames to HDF5 stores
Reading and writing to Excel with pandas
Using REST web services and JSON
Reading and writing JSON with pandas
Parsing RSS and Atom feeds
Parsing HTML with Beautiful Soup
Summary
6
Data Visualization
Data Visualization
matplotlib subpackages
Basic matplotlib plots
Logarithmic plots
Scatter plots
Legends and annotations
Three-dimensional plots
Plotting in pandas
Lag plots
Autocorrelation plots
Plot.ly
Summary
7
Signal Processing and Time Series
Signal Processing and Time Series
statsmodels subpackages
Moving averages
Window functions
Defining cointegration
Autocorrelation
Autoregressive models
ARMA models
Generating periodic signals
Fourier analysis
Spectral analysis
Filtering
Summary
8
Working with Databases
Working with Databases
Lightweight access with sqlite3
Accessing databases from pandas
SQLAlchemy
Pony ORM
Dataset – databases for lazy people
PyMongo and MongoDB
Storing data in Redis
Apache Cassandra
Summary
9
Analyzing Textual Data and Social Media
Analyzing Textual Data and Social Media
Installing NLTK
Filtering out stopwords, names, and numbers
The bag-of-words model
Analyzing word frequencies
Naive Bayes classification
Sentiment analysis
Creating word clouds
Social network analysis
Summary
10
Predictive Analytics and Machine Learning
Predictive Analytics and Machine Learning
A tour of scikit-learn
Preprocessing
Classification with logistic regression
Classification with support vector machines
Regression with ElasticNetCV
Support vector regression
Clustering with affinity propagation
Mean Shift
Genetic algorithms
Neural networks
Decision trees
Summary
11
Environments Outside the Python Ecosystem and Cloud Computing
Environments Outside the Python Ecosystem and Cloud Computing
Exchanging information with MATLAB/Octave
Installing rpy2
Interfacing with R
Sending NumPy arrays to Java
Integrating SWIG and NumPy
Integrating Boost and Python
Using Fortran code through f2py
Setting up Google App Engine
Running programs on PythonAnywhere
Working with Wakari
Summary
12
Performance Tuning, Profiling, and Concurrency
Performance Tuning, Profiling, and Concurrency
Profiling the code
Installing Cython
Calling C code
Creating a process pool with multiprocessing
Speeding up embarrassingly parallel for loops with Joblib
Comparing Bottleneck to NumPy functions
Performing MapReduce with Jug
Installing MPI for Python
IPython Parallel
Summary
Key Concepts
Key Concepts
Useful Functions
Useful Functions
matplotlib
NumPy
pandas
Scikit-learn
SciPy
Online Resources
Online Resources
Index
Index

Chapter 5. Retrieving, Processing, and Storing Data

Data can be found everywhere in all shapes and forms. We can get it from the Web, by e-mail and FTP, or create it ourselves in a lab experiment or marketing poll. An exhaustive overview of how to acquire data in various formats will require many more pages than what we have available. Sometimes, we need to store data before we can analyze it or after we are done with our analysis. We will also discuss storing data in this chapter. Chapter 8, Working with Databases, gives information about various databases (relational and NoSQL) and related APIs. The following is a list of the topics that we are going to cover in this chapter:

  • Writing CSV files with NumPy and pandas

  • The binary .npy and pickle formats

  • Reading and writing to Excel with pandas

  • JSON

  • REST web services

  • Parsing RSS feeds

  • Scraping the Web

  • Parsing HTML

  • Storing data with PyTables

  • HDF5 pandas I/O

Previous Section
End of Section 1
Next Section