Book Image

Spark for Data Science

By : Srinivas Duvvuri, Bikramaditya Singhal
Book Image

Spark for Data Science

By: Srinivas Duvvuri, Bikramaditya Singhal

Overview of this book

This is the era of Big Data. The words ‘Big Data’ implies big innovation and enables a competitive advantage for businesses. Apache Spark was designed to perform Big Data analytics at scale, and so Spark is equipped with the necessary algorithms and supports multiple programming languages. Whether you are a technologist, a data scientist, or a beginner to Big Data analytics, this book will provide you with all the skills necessary to perform statistical data analysis, data visualization, predictive modeling, and build scalable data products or solutions using Python, Scala, and R. With ample case studies and real-world examples, Spark for Data Science will help you ensure the successful execution of your data science projects.

Related Content you might be interested in

Current Title:

Small book image
Spark for Data Science
Srinivas Duvvuri | Bikramaditya Singhal
Sep, 2016 | 344 pages
No titles found
Table of Contents (18 chapters)
Spark for Data Science
Spark for Data Science
Credits
Credits
Foreword
Foreword
About the Authors
About the Authors
About the Reviewers
About the Reviewers
www.PacktPub.com
www.PacktPub.com
Preface
Preface
Free Chapter
1
Big Data and Data Science – An Introduction
Big Data and Data Science – An Introduction
Big data overview
Challenges with big data analytics
Evolution of big data analytics
Spark for data analytics
The Spark stack
Summary
References
2
The Spark Programming Model
The Spark Programming Model
The programming paradigm
The Spark engine
The RDD API
RDD operations
Summary
References
3
Introduction to DataFrames
Introduction to DataFrames
Why DataFrames?
Spark SQL
The DataFrame API
Creating DataFrames
DataFrame operations
Summary
References
4
Unified Data Access
Unified Data Access
Data abstractions in Apache Spark
Datasets
Spark SQL
Structured Streaming
Continuous applications
Summary
References
5
Data Analysis on Spark
Data Analysis on Spark
Data analytics life cycle
Data acquisition
Data preparation
Basics of statistics
Descriptive statistics
Inferential statistics
Summary
References
6
Machine Learning
Machine Learning
Introduction
MLlib and the Pipeline API
Introduction to machine learning
Regression methods
Classification methods
Linear Support Vector Machines (SVM)
Training an SVM
Decision trees
Ensembles
Multilayer perceptron classifier
Clustering techniques
Summary
References
7
Extending Spark with SparkR
Extending Spark with SparkR
SparkR basics
Advantages and limitations
Programming with SparkR
SparkR DataFrames
Machine learning
Summary
References
8
Analyzing Unstructured Data
Analyzing Unstructured Data
Sources of unstructured data
Processing unstructured data
Text classification
Text clustering
Dimensionality reduction
Singular Value Decomposition
Summary
References:
9
Visualizing Big Data
Visualizing Big Data
Why visualize data?
Data visualization tools
Data visualization techniques
Summary
References
10
Putting It All Together
Putting It All Together
A quick recap
Introducing a case study
The business problem
Data acquisition and data cleansing
Developing the hypothesis
Data exploration
Data preparation
Model building
Data visualization
Communicating the results to business users
Summary
References
11
Building Data Science Applications
Building Data Science Applications
Scope of development
The Scala advantage
Spark development status
The big data trends
Summary
References

Advantages and limitations

The R language has long been the lingua franca of data scientists. Its simple-to-understand DataFrame abstraction, expressive APIs, and vibrant package ecosystem are exactly what the analysts needed. The main challenge was with the scalability. SparkR bridges that gap by providing distributed in-memory DataFrames without leaving the R eco-system. Such a symbiotic relationship allows users to gain the following benefits:

  • There is no need for the analyst to learn a new language

  • The SparkR APIs are similar to R APIs

  • You can access SparkR from R studio, along with the autocomplete feature

  • Performing interactive, exploratory analysis of a very large dataset is no longer hindered by memory limitations or long turnaround times

  • Accessing data from different types of data sources becomes a lot easier. Most of the tasks which were imperative before have become declarative. Check Chapter 4, Unified Data Access, to learn more

  • You can freely mix dplyr such as Spark functions, SQL...

Previous Section
End of Section 3
Next Section