Book Image

Spark for Data Science

By : Srinivas Duvvuri, Bikramaditya Singhal
Book Image

Spark for Data Science

By: Srinivas Duvvuri, Bikramaditya Singhal

Overview of this book

This is the era of Big Data. The words ‘Big Data’ implies big innovation and enables a competitive advantage for businesses. Apache Spark was designed to perform Big Data analytics at scale, and so Spark is equipped with the necessary algorithms and supports multiple programming languages. Whether you are a technologist, a data scientist, or a beginner to Big Data analytics, this book will provide you with all the skills necessary to perform statistical data analysis, data visualization, predictive modeling, and build scalable data products or solutions using Python, Scala, and R. With ample case studies and real-world examples, Spark for Data Science will help you ensure the successful execution of your data science projects.

Related Content you might be interested in

Current Title:

Small book image
Spark for Data Science
Srinivas Duvvuri | Bikramaditya Singhal
Sep, 2016 | 344 pages
No titles found
Table of Contents (18 chapters)
Spark for Data Science
Spark for Data Science
Credits
Credits
Foreword
Foreword
About the Authors
About the Authors
About the Reviewers
About the Reviewers
www.PacktPub.com
www.PacktPub.com
Preface
Preface
Free Chapter
1
Big Data and Data Science – An Introduction
Big Data and Data Science – An Introduction
Big data overview
Challenges with big data analytics
Evolution of big data analytics
Spark for data analytics
The Spark stack
Summary
References
2
The Spark Programming Model
The Spark Programming Model
The programming paradigm
The Spark engine
The RDD API
RDD operations
Summary
References
3
Introduction to DataFrames
Introduction to DataFrames
Why DataFrames?
Spark SQL
The DataFrame API
Creating DataFrames
DataFrame operations
Summary
References
4
Unified Data Access
Unified Data Access
Data abstractions in Apache Spark
Datasets
Spark SQL
Structured Streaming
Continuous applications
Summary
References
5
Data Analysis on Spark
Data Analysis on Spark
Data analytics life cycle
Data acquisition
Data preparation
Basics of statistics
Descriptive statistics
Inferential statistics
Summary
References
6
Machine Learning
Machine Learning
Introduction
MLlib and the Pipeline API
Introduction to machine learning
Regression methods
Classification methods
Linear Support Vector Machines (SVM)
Training an SVM
Decision trees
Ensembles
Multilayer perceptron classifier
Clustering techniques
Summary
References
7
Extending Spark with SparkR
Extending Spark with SparkR
SparkR basics
Advantages and limitations
Programming with SparkR
SparkR DataFrames
Machine learning
Summary
References
8
Analyzing Unstructured Data
Analyzing Unstructured Data
Sources of unstructured data
Processing unstructured data
Text classification
Text clustering
Dimensionality reduction
Singular Value Decomposition
Summary
References:
9
Visualizing Big Data
Visualizing Big Data
Why visualize data?
Data visualization tools
Data visualization techniques
Summary
References
10
Putting It All Together
Putting It All Together
A quick recap
Introducing a case study
The business problem
Data acquisition and data cleansing
Developing the hypothesis
Data exploration
Data preparation
Model building
Data visualization
Communicating the results to business users
Summary
References
11
Building Data Science Applications
Building Data Science Applications
Scope of development
The Scala advantage
Spark development status
The big data trends
Summary
References

Chapter 8.  Analyzing Unstructured Data

In this Big Data era, the proliferation of unstructured data is overwhelming. Numerous methods such as data mining, Natural Language Processing (NLP), information retrieval, and so on, exist for analyzing unstructured data. Due to the rapid growth of unstructured data in all kinds of businesses, scalable solutions have become the need of the hour. Apache Spark is equipped with out of the box algorithms for text analytics, and it also supports custom development of algorithms that are not available by default.

In the previous chapter we have shown how SparkR, an R API to Spark for R programmers can harness the power of Spark, without learning a new language .  In this chapter, we are going to step into a whole new dimension and explore algorithms and techniques to extract information out of unstructured data by leveraging Spark.

As a prerequisite for this chapter, a basic understanding of programming in Python or Scala and an overall understanding...

Previous Section
End of Section 1
Next Section