Book Image

Practical Data Analysis - Second Edition

By : Hector Cuesta, Dr. Sampath Kumar
Book Image

Practical Data Analysis - Second Edition

By: Hector Cuesta, Dr. Sampath Kumar

Overview of this book

Beyond buzzwords like Big Data or Data Science, there are a great opportunities to innovate in many businesses using data analysis to get data-driven products. Data analysis involves asking many questions about data in order to discover insights and generate value for a product or a service. This book explains the basic data algorithms without the theoretical jargon, and you’ll get hands-on turning data into insights using machine learning techniques. We will perform data-driven innovation processing for several types of data such as text, Images, social network graphs, documents, and time series, showing you how to implement large data processing with MongoDB and Apache Spark.

Related Content you might be interested in

Current Title:

Small book image
Practical Data Analysis - Second Edition
Hector Cuesta | Dr. Sampath Kumar
Sep, 2016 | 338 pages
No titles found
Table of Contents (21 chapters)
Practical Data Analysis - Second Edition
Practical Data Analysis - Second Edition
Credits
Credits
About the Authors
About the Authors
About the Reviewers
About the Reviewers
www.PacktPub.com
www.PacktPub.com
Preface
Preface
Free Chapter
1
Getting Started
Getting Started
Computer science
Artificial intelligence
Machine learning
Statistics
Mathematics
Knowledge domain
Data, information, and knowledge
The data analysis process
Quantitative versus qualitative data analysis
Importance of data visualization
What about big data?
Quantified self
Tools and toys for this book
Summary
2
Preprocessing Data
Preprocessing Data
Data sources
Data scrubbing
Data formats
Data reduction methods
Getting started with OpenRefine
Summary
3
Getting to Grips with Visualization
Getting to Grips with Visualization
What is visualization?
Working with web-based visualization
Exploring scientific visualization
Visualization in art
The visualization life cycle
Visualizing different types of data
Getting started with D3.js
Interaction and animation
Data from social networks
An overview of visual analytics
Summary
4
Text Classification
Text Classification
Learning and classification
Bayesian classification
E-mail subject line tester
The data
The algorithm
Classifier accuracy
Summary
5
Similarity-Based Image Retrieval
Similarity-Based Image Retrieval
Image similarity search
Dynamic time warping
Processing the image dataset
Implementing DTW
Analyzing the results
Summary
6
Simulation of Stock Prices
Simulation of Stock Prices
Financial time series
Random Walk simulation
Monte Carlo methods
Generating random numbers
Implementation in D3js
Quantitative analyst
Summary
7
Predicting Gold Prices
Predicting Gold Prices
Working with time series data
Smoothing time series
Lineal regression
The data - historical gold prices
Nonlinear regressions
Summary
8
Working with Support Vector Machines
Working with Support Vector Machines
Understanding the multivariate dataset
Dimensionality reduction
Getting started with SVM
Summary
9
Modeling Infectious Diseases with Cellular Automata
Modeling Infectious Diseases with Cellular Automata
Introduction to epidemiology
The epidemic models
Modeling with Cellular Automaton
Simulation of the SIRS model in CA with D3.js
Summary
10
Working with Social Graphs
Working with Social Graphs
Structure of a graph
Social networks analysis
Acquiring the Facebook graph
Working with graphs using Gephi
Statistical analysis
Degree distribution
Transforming GDF to JSON
Graph visualization with D3.js
Summary
11
Working with Twitter Data
Working with Twitter Data
The anatomy of Twitter data
Using OAuth to access Twitter API
Getting started with Twython
Summary
12
Data Processing and Aggregation with MongoDB
Data Processing and Aggregation with MongoDB
Getting started with MongoDB
Data preparation
Group
Aggregation framework
Summary
13
Working with MapReduce
Working with MapReduce
An overview of MapReduce
Programming model
Using MapReduce with MongoDB
Filtering the input collection
Grouping and aggregation
Counting the most common words in tweets
Summary
14
Online Data Analysis with Jupyter and Wakari
Online Data Analysis with Jupyter and Wakari
Getting started with Wakari
Getting started with IPython notebook
Introduction to image processing with PIL
Getting started with pandas
Sharing your Notebook
Summary
15
Understanding Data Processing using Apache Spark
Understanding Data Processing using Apache Spark
Platform for data processing
An introduction to the distributed file system
An introduction to Apache Spark
Summary

Chapter 15.  Understanding Data Processing using Apache Spark

In this chapter, we will present the main features of data processing architecture and the Cloudera platform distribution. Then, we will explore how to use a distributed filesystem and how to managing files from terminal and using a web interface. Finally, we will describe the use of Apache Spark, which is an open source, big data processing framework built with the goal of being fast and easy to use. Apache Spark provides us with a unified framework to manage big data processing requirements, such as data streaming, machine learning, and analytics.

In this chapter, we will cover these topics:

  • Understanding data processing

  • Platform for data processing

  • An introduction to the distributed file system

  • An introduction to Apache Spark

  • Understanding data processing

Since the first edition of this book in 2013, there has been big changes in the data-driven scene. With the emerge of buzzwords such as big data, data science, and deep learning...

Previous Section
End of Section 1
Next Section