Book Image

Practical Data Analysis - Second Edition

By : Hector Cuesta, Dr. Sampath Kumar
Book Image

Practical Data Analysis - Second Edition

By: Hector Cuesta, Dr. Sampath Kumar

Overview of this book

Beyond buzzwords like Big Data or Data Science, there are a great opportunities to innovate in many businesses using data analysis to get data-driven products. Data analysis involves asking many questions about data in order to discover insights and generate value for a product or a service. This book explains the basic data algorithms without the theoretical jargon, and you’ll get hands-on turning data into insights using machine learning techniques. We will perform data-driven innovation processing for several types of data such as text, Images, social network graphs, documents, and time series, showing you how to implement large data processing with MongoDB and Apache Spark.

Related Content you might be interested in

Current Title:

Small book image
Practical Data Analysis - Second Edition
Hector Cuesta | Dr. Sampath Kumar
Sep, 2016 | 338 pages
No titles found
Table of Contents (21 chapters)
Practical Data Analysis - Second Edition
Practical Data Analysis - Second Edition
Credits
Credits
About the Authors
About the Authors
About the Reviewers
About the Reviewers
www.PacktPub.com
www.PacktPub.com
Preface
Preface
Free Chapter
1
Getting Started
Getting Started
Computer science
Artificial intelligence
Machine learning
Statistics
Mathematics
Knowledge domain
Data, information, and knowledge
The data analysis process
Quantitative versus qualitative data analysis
Importance of data visualization
What about big data?
Quantified self
Tools and toys for this book
Summary
2
Preprocessing Data
Preprocessing Data
Data sources
Data scrubbing
Data formats
Data reduction methods
Getting started with OpenRefine
Summary
3
Getting to Grips with Visualization
Getting to Grips with Visualization
What is visualization?
Working with web-based visualization
Exploring scientific visualization
Visualization in art
The visualization life cycle
Visualizing different types of data
Getting started with D3.js
Interaction and animation
Data from social networks
An overview of visual analytics
Summary
4
Text Classification
Text Classification
Learning and classification
Bayesian classification
E-mail subject line tester
The data
The algorithm
Classifier accuracy
Summary
5
Similarity-Based Image Retrieval
Similarity-Based Image Retrieval
Image similarity search
Dynamic time warping
Processing the image dataset
Implementing DTW
Analyzing the results
Summary
6
Simulation of Stock Prices
Simulation of Stock Prices
Financial time series
Random Walk simulation
Monte Carlo methods
Generating random numbers
Implementation in D3js
Quantitative analyst
Summary
7
Predicting Gold Prices
Predicting Gold Prices
Working with time series data
Smoothing time series
Lineal regression
The data - historical gold prices
Nonlinear regressions
Summary
8
Working with Support Vector Machines
Working with Support Vector Machines
Understanding the multivariate dataset
Dimensionality reduction
Getting started with SVM
Summary
9
Modeling Infectious Diseases with Cellular Automata
Modeling Infectious Diseases with Cellular Automata
Introduction to epidemiology
The epidemic models
Modeling with Cellular Automaton
Simulation of the SIRS model in CA with D3.js
Summary
10
Working with Social Graphs
Working with Social Graphs
Structure of a graph
Social networks analysis
Acquiring the Facebook graph
Working with graphs using Gephi
Statistical analysis
Degree distribution
Transforming GDF to JSON
Graph visualization with D3.js
Summary
11
Working with Twitter Data
Working with Twitter Data
The anatomy of Twitter data
Using OAuth to access Twitter API
Getting started with Twython
Summary
12
Data Processing and Aggregation with MongoDB
Data Processing and Aggregation with MongoDB
Getting started with MongoDB
Data preparation
Group
Aggregation framework
Summary
13
Working with MapReduce
Working with MapReduce
An overview of MapReduce
Programming model
Using MapReduce with MongoDB
Filtering the input collection
Grouping and aggregation
Counting the most common words in tweets
Summary
14
Online Data Analysis with Jupyter and Wakari
Online Data Analysis with Jupyter and Wakari
Getting started with Wakari
Getting started with IPython notebook
Introduction to image processing with PIL
Getting started with pandas
Sharing your Notebook
Summary
15
Understanding Data Processing using Apache Spark
Understanding Data Processing using Apache Spark
Platform for data processing
An introduction to the distributed file system
An introduction to Apache Spark
Summary

An introduction to the distributed file system

A distributed file system is practically the same as any file system due to its basic actions such as storing, reading, deleting files, and assigning security levels are support. The main difference is focused on the number of servers that can be used at same time without dealing with complexity of synchronization. In this case, we can store large files in different server nodes without caring about redundancy or parallel operations.

There are a lot of frameworks for distributed file systems, such as Red Hat Cluster FS, Ceph File system, Hadoop Distributed File System (HDFS), and Tachyon File System.

In this chapter, we will use HDFS, which is an open source implementation of Google File System, built to handle large files into a cluster of commodity hardware. The HDFS cluster implements a NameNode that manages operations through the file system, and a series of DataNodes that manage the storage of the files in the cluster nodes individually,...

Previous Section
End of Section 3
Next Section