Book Image

Spark for Data Science

By : Srinivas Duvvuri, Bikramaditya Singhal
Book Image

Spark for Data Science

By: Srinivas Duvvuri, Bikramaditya Singhal

Overview of this book

This is the era of Big Data. The words ‘Big Data’ implies big innovation and enables a competitive advantage for businesses. Apache Spark was designed to perform Big Data analytics at scale, and so Spark is equipped with the necessary algorithms and supports multiple programming languages. Whether you are a technologist, a data scientist, or a beginner to Big Data analytics, this book will provide you with all the skills necessary to perform statistical data analysis, data visualization, predictive modeling, and build scalable data products or solutions using Python, Scala, and R. With ample case studies and real-world examples, Spark for Data Science will help you ensure the successful execution of your data science projects.

Related Content you might be interested in

Current Title:

Small book image
Spark for Data Science
Srinivas Duvvuri | Bikramaditya Singhal
Sep, 2016 | 344 pages
No titles found
Table of Contents (18 chapters)
Spark for Data Science
Spark for Data Science
Credits
Credits
Foreword
Foreword
About the Authors
About the Authors
About the Reviewers
About the Reviewers
www.PacktPub.com
www.PacktPub.com
Preface
Preface
Free Chapter
1
Big Data and Data Science – An Introduction
Big Data and Data Science – An Introduction
Big data overview
Challenges with big data analytics
Evolution of big data analytics
Spark for data analytics
The Spark stack
Summary
References
2
The Spark Programming Model
The Spark Programming Model
The programming paradigm
The Spark engine
The RDD API
RDD operations
Summary
References
3
Introduction to DataFrames
Introduction to DataFrames
Why DataFrames?
Spark SQL
The DataFrame API
Creating DataFrames
DataFrame operations
Summary
References
4
Unified Data Access
Unified Data Access
Data abstractions in Apache Spark
Datasets
Spark SQL
Structured Streaming
Continuous applications
Summary
References
5
Data Analysis on Spark
Data Analysis on Spark
Data analytics life cycle
Data acquisition
Data preparation
Basics of statistics
Descriptive statistics
Inferential statistics
Summary
References
6
Machine Learning
Machine Learning
Introduction
MLlib and the Pipeline API
Introduction to machine learning
Regression methods
Classification methods
Linear Support Vector Machines (SVM)
Training an SVM
Decision trees
Ensembles
Multilayer perceptron classifier
Clustering techniques
Summary
References
7
Extending Spark with SparkR
Extending Spark with SparkR
SparkR basics
Advantages and limitations
Programming with SparkR
SparkR DataFrames
Machine learning
Summary
References
8
Analyzing Unstructured Data
Analyzing Unstructured Data
Sources of unstructured data
Processing unstructured data
Text classification
Text clustering
Dimensionality reduction
Singular Value Decomposition
Summary
References:
9
Visualizing Big Data
Visualizing Big Data
Why visualize data?
Data visualization tools
Data visualization techniques
Summary
References
10
Putting It All Together
Putting It All Together
A quick recap
Introducing a case study
The business problem
Data acquisition and data cleansing
Developing the hypothesis
Data exploration
Data preparation
Model building
Data visualization
Communicating the results to business users
Summary
References
11
Building Data Science Applications
Building Data Science Applications
Scope of development
The Scala advantage
Spark development status
The big data trends
Summary
References

Chapter 4.  Unified Data Access

Data integration from disparate data sources had always been a daunting feat. The three V's of big data and ever-shrinking processing time frames have made the task even more challenging. Delivering a clear view of well-curated data in near real time is extremely important for business. However, real-time curated data along with the ability to perform different operations such as ETL, ad hoc querying, and machine learning in a unified fashion is what is emerging as a key business differentiator.

Apache Spark was created to offer a single general-purpose engine that can process data from a variety of data sources and support large-scale data processing for various different operations. Spark enables developers to combine SQL, Streaming, graphs, and machine learning algorithms in a single workflow!

In the previous chapters, we discussed Resilient Distributed Datasets (RDDs) as well as DataFrames. In Chapter 3, Introduction to DataFrames, we introduced Spark...

Previous Section
End of Section 1
Next Section