Book Image

Getting Started with Python Data Analysis

Book Image

Getting Started with Python Data Analysis

Overview of this book

Data analysis is the process of applying logical and analytical reasoning to study each component of data. Python is a multi-domain, high-level, programming language. It’s often used as a scripting language because of its forgiving syntax and operability with a wide variety of different eco-systems. Python has powerful standard libraries or toolkits such as Pylearn2 and Hebel, which offers a fast, reliable, cross-platform environment for data analysis. With this book, we will get you started with Python data analysis and show you what its advantages are. The book starts by introducing the principles of data analysis and supported libraries, along with NumPy basics for statistic and data processing. Next it provides an overview of the Pandas package and uses its powerful features to solve data processing problems. Moving on, the book takes you through a brief overview of the Matplotlib API and some common plotting functions for DataFrame such as plot. Next, it will teach you to manipulate the time and data structure, and load and store data in a file or database using Python packages. The book will also teach you how to apply powerful packages in Python to process raw data into pure and helpful data using examples. Finally, the book gives you a brief overview of machine learning algorithms, that is, applying data analysis results to make decisions or build helpful products, such as recommendations and predictions using scikit-learn.

Related Content you might be interested in

Current Title:

Small book image
Getting Started with Python Data Analysis
Nov, 2015 | 188 pages
No titles found
Table of Contents (15 chapters)
Getting Started with Python Data Analysis
Getting Started with Python Data Analysis
Credits
Credits
About the Authors
About the Authors
About the Reviewers
About the Reviewers
www.PacktPub.com
www.PacktPub.com
Preface
Preface
Free Chapter
1
Introducing Data Analysis and Libraries
Introducing Data Analysis and Libraries
Data analysis and processing
An overview of the libraries in data analysis
Python libraries in data analysis
Summary
2
NumPy Arrays and Vectorized Computation
NumPy Arrays and Vectorized Computation
NumPy arrays
Array functions
Data processing using arrays
Linear algebra with NumPy
NumPy random numbers
Summary
3
Data Analysis with Pandas
Data Analysis with Pandas
An overview of the Pandas package
The Pandas data structure
The essential basic functionality
Indexing and selecting data
Computational tools
Working with missing data
Advanced uses of Pandas for data analysis
Summary
4
Data Visualization
Data Visualization
The matplotlib API primer
Exploring plot types
Legends and annotations
Plotting functions with Pandas
Additional Python data visualization tools
Summary
5
Time Series
Time Series
Time series primer
Working with date and time objects
Resampling time series
Downsampling time series data
Upsampling time series data
Time zone handling
Timedeltas
Time series plotting
Summary
6
Interacting with Databases
Interacting with Databases
Interacting with data in text format
Interacting with data in binary format
Interacting with data in MongoDB
Interacting with data in Redis
Summary
7
Data Analysis Application Examples
Data Analysis Application Examples
Data munging
Data aggregation
Grouping data
Summary
8
Machine Learning Models with scikit-learn
Machine Learning Models with scikit-learn
An overview of machine learning models
The scikit-learn modules for different models
Data representation in scikit-learn
Supervised learning – classification and regression
Unsupervised learning – clustering and dimensionality reduction
Measuring prediction performance
Summary
Index
Index

Data aggregation

As a final topic, we will look at ways to get a condensed view of data with aggregations. Pandas comes with a lot of aggregation functions built-in. We already saw the describe function in Chapter 3, Data Analysis with Pandas. This works on parts of the data as well. We start with some artificial data again, containing measurements about the number of sunshine hours per city and date:

>>> df.head()
   country     city        date  hours
0  Germany  Hamburg  2015-06-01      8
1  Germany  Hamburg  2015-06-02     10
2  Germany  Hamburg  2015-06-03      9
3  Germany  Hamburg  2015-06-04      7
4  Germany  Hamburg  2015-06-05      3

To view a summary per city, we use the describe function on the grouped data set:

>>> df.groupby("city").describe()
                      hours
city
Berlin     count  10.000000
           mean    6.000000
           std     3.741657
           min     0.000000
           25%     4.000000
           50%     6.000000
           75...
Previous Section
End of Section 3
Next Section