Sign In Start Free Trial

Book Overview & Buying
Table Of Contents

Spark for Data Science

By : Duvvuri, Singhal

Spark for Data Science

By: Duvvuri, Singhal

Overview of this book

This is the era of Big Data. The words ‘Big Data’ implies big innovation and enables a competitive advantage for businesses. Apache Spark was designed to perform Big Data analytics at scale, and so Spark is equipped with the necessary algorithms and supports multiple programming languages. Whether you are a technologist, a data scientist, or a beginner to Big Data analytics, this book will provide you with all the skills necessary to perform statistical data analysis, data visualization, predictive modeling, and build scalable data products or solutions using Python, Scala, and R. With ample case studies and real-world examples, Spark for Data Science will help you ensure the successful execution of your data science projects.

Preface

Preface

What this book covers

What you need for this book

Who this book is for

Conventions

Reader feedback

Customer support

Free Chapter

1. Big Data and Data Science – An Introduction

1. Big Data and Data Science – An Introduction

Big data overview

Challenges with big data analytics

Evolution of big data analytics

Spark for data analytics

The Spark stack

Summary

References

2. The Spark Programming Model

2. The Spark Programming Model

The programming paradigm

The Spark engine

The RDD API

RDD operations

Summary

References

3. Introduction to DataFrames

3. Introduction to DataFrames

Why DataFrames?

Spark SQL

The DataFrame API

Creating DataFrames

DataFrame operations

Summary

References

4. Unified Data Access

4. Unified Data Access

Data abstractions in Apache Spark

Datasets

Spark SQL

Structured Streaming

Continuous applications

Summary

References

5. Data Analysis on Spark

5. Data Analysis on Spark

Data analytics life cycle

Data acquisition

Data preparation

Basics of statistics

Descriptive statistics

Inferential statistics

Summary

References

6. Machine Learning

6. Machine Learning

Introduction

MLlib and the Pipeline API

Introduction to machine learning

Regression methods

Classification methods

Linear Support Vector Machines (SVM)

Training an SVM

Decision trees

Ensembles

Multilayer perceptron classifier

Clustering techniques

Summary

References

7. Extending Spark with SparkR

7. Extending Spark with SparkR

SparkR basics

Advantages and limitations

Programming with SparkR

SparkR DataFrames

Machine learning

Summary

References

8. Analyzing Unstructured Data

8. Analyzing Unstructured Data

Sources of unstructured data

Processing unstructured data

Text classification

Text clustering

Dimensionality reduction

Singular Value Decomposition

Summary

References:

9. Visualizing Big Data

9. Visualizing Big Data

Why visualize data?

Data visualization tools

Data visualization techniques

Summary

References

10. Putting It All Together

10. Putting It All Together

A quick recap

Introducing a case study

The business problem

Data acquisition and data cleansing

Developing the hypothesis

Data exploration

Data preparation

Model building

Data visualization

Communicating the results to business users

Summary

References

11. Building Data Science Applications

11. Building Data Science Applications

Scope of development

The Scala advantage

Spark development status

The big data trends

Summary

References

Chapter 2. The Spark Programming Model

Large-scale data processing using thousands of nodes with built-in fault tolerance has become widespread due to the availability of open source frameworks, with Hadoop being a popular choice. These frameworks are quite successful in executing specific tasks such as Extract, Transform, and Load (ETL) and storage applications that deal with web-scale data. However, developers were left with a myriad of tools to work with, along with the well-established Hadoop ecosystem. There was a need for a single, general-purpose development platform that caters to batch, streaming, interactive, and iterative requirements. This was the motivation behind Spark.

The previous chapter outlined the big data analytics challenges and how Spark addressed most of them at a very high level. In this chapter, we will examine the design goals and choices involved in the making of Spark to get a clearer understanding of its suitability as a data science platform for big...

CONTINUE READING

83

Tech Concepts

36

Programming languages

73

Tech Tools

Unlimited access to the largest independent learning library in tech of over 8,000 expert-authored tech books and videos.

Innovative learning tools, including AI book assistants, code context explainers, and text-to-speech.

50+ new titles added per month and exclusive early access to books as they are being written.

Spark for Data Science

Search

Your notes and bookmarks