Book Image

Machine Learning with Spark - Second Edition

By : Rajdeep Dua, Manpreet Singh Ghotra
Book Image

Machine Learning with Spark - Second Edition

By: Rajdeep Dua, Manpreet Singh Ghotra

Overview of this book

This book will teach you about popular machine learning algorithms and their implementation. You will learn how various machine learning concepts are implemented in the context of Spark ML. You will start by installing Spark in a single and multinode cluster. Next you'll see how to execute Scala and Python based programs for Spark ML. Then we will take a few datasets and go deeper into clustering, classification, and regression. Toward the end, we will also cover text processing using Spark ML. Once you have learned the concepts, they can be applied to implement algorithms in either green-field implementations or to migrate existing systems to this new platform. You can migrate from Mahout or Scikit to use Spark ML. By the end of this book, you will acquire the skills to leverage Spark's features to create your own scalable machine learning applications and power a modern data-driven business.

Related Content you might be interested in

Current Title:

Small book image
Machine Learning with Spark - Second Edition
Rajdeep Dua | Manpreet Singh Ghotra
Apr, 2017 | 532 pages
Small book image
Kali Linux Intrusion and Exploitation Cookbook
Dhruv Shah | Ishan Girdhar
Apr, 2017 | 512 pages
Small book image
Apache Spark 2.x Machine Learning Cookbook
Shuen Mei | Siamak Amirghodsi | Meenakshi Rajendran | Broderick Hall
Sep, 2017 | 666 pages
Small book image
Learning Spark SQL
Aurobindo Sarkar
Sep, 2017 | 452 pages
Table of Contents (13 chapters)
Preface
Preface
What this book covers
What you need for this book
Who this book is for
Conventions
Reader feedback
Customer support
Free Chapter
1
Getting Up and Running with Spark
Getting Up and Running with Spark
Installing and setting up Spark locally
Spark clusters
The Spark programming model
SchemaRDD
Spark data frame
The first step to a Spark program in Scala
The first step to a Spark program in Java
The first step to a Spark program in Python
The first step to a Spark program in R
Getting Spark running on Amazon EC2
Configuring and running Spark on Amazon Elastic Map Reduce
UI in Spark
Supported machine learning algorithms by Spark
Benefits of using Spark ML as compared to existing libraries
Spark Cluster on Google Compute Engine - DataProc
Summary
2
Math for Machine Learning
Math for Machine Learning
Linear algebra
Gradient descent
Prior, likelihood, and posterior
Calculus
Plotting
Summary
3
Designing a Machine Learning System
Designing a Machine Learning System
What is Machine Learning?
Introducing MovieStream
Business use cases for a machine learning system
Types of machine learning models
The components of a data-driven machine learning system
An architecture for a machine learning system
Spark MLlib
Performance improvements in Spark ML over Spark MLlib
Comparing algorithms supported by MLlib
MLlib supported methods and developer APIs
MLlib vision
MLlib versions compared
Summary
4
Obtaining, Processing, and Preparing Data with Spark
Obtaining, Processing, and Preparing Data with Spark
Accessing publicly available datasets
Exploring and visualizing your data
Processing and transforming your data
Extracting useful features from your data
Summary
5
Building a Recommendation Engine with Spark
Building a Recommendation Engine with Spark
Types of recommendation models
Extracting the right features from your data
Training the recommendation model
Using the recommendation model
Evaluating the performance of recommendation models
FP-Growth algorithm
Summary
6
Building a Classification Model with Spark
Building a Classification Model with Spark
Types of classification models
Extracting the right features from your data
Training classification models
Using classification models
Improving model performance and tuning parameters
Additional features
Summary
7
Building a Regression Model with Spark
Building a Regression Model with Spark
Types of regression models
Evaluating the performance of regression models
Extracting the right features from your data
Training and using regression models
Improving model performance and tuning parameters
Summary
8
Building a Clustering Model with Spark
Building a Clustering Model with Spark
Types of clustering models
Extracting the right features from your data
K-means - training a clustering model
K-means - evaluating the performance of clustering models
Effect of iterations on WSSSE
Bisecting KMeans
Bisecting K-means - training a clustering model
Gaussian Mixture Model
Summary
9
Dimensionality Reduction with Spark
Dimensionality Reduction with Spark
Types of dimensionality reduction
Extracting the right features from your data
Training a dimensionality reduction model
Using a dimensionality reduction model
Evaluating dimensionality reduction models
Summary
10
Advanced Text Processing with Spark
Advanced Text Processing with Spark
What's so special about text data?
Extracting the right features from your data
Using a tf-idf model
Evaluating the impact of text processing
Text classification with Spark 2.0
Word2Vec models
Word2Vec with Spark ML on the 20 Newsgroups dataset
Summary
11
Real-Time Machine Learning with Spark Streaming
Real-Time Machine Learning with Spark Streaming
Online learning
Stream processing
Online learning with Spark Streaming
Online model evaluation
Structured Streaming
Summary
12
Pipeline APIs for Spark ML
Pipeline APIs for Spark ML
Introduction to pipelines
How pipelines work
Machine learning pipeline with an example
Summary

Customer support

Now that you are the proud owner of a Packt book, we have a number of things to help you to get the most from your purchase.

Downloading the example code

You can download the example code files for this book from your account at http://www.packtpub.com. If you purchased this book elsewhere, you can visit http://www.packtpub.com/support and register to have the files e-mailed directly to you.

You can download the code files by following these steps:

  1. Log in or register to our website using your e-mail address and password.
  2. Hover the mouse pointer on the SUPPORT tab at the top.
  3. Click on Code Downloads & Errata.
  4. Enter the name of the book in the Search box.
  5. Select the book for which you're looking to download the code files.
  6. Choose from the drop-down menu where you purchased this book from.
  7. Click on Code Download.

Once the file is downloaded, please make sure that you unzip or extract the folder using the latest version of:

  • WinRAR / 7-Zip for Windows
  • Zipeg / iZip / UnRarX for Mac
  • 7-Zip / PeaZip for Linux

The code bundle for the book is also hosted on GitHub at https://github.com/PacktPublishing/Machine-Learning-with-Spark-Second-Edition. We also have other code bundles from our rich catalog of books and videos available at https://github.com/PacktPublishing/. Check them out!

Downloading the color images of this book

Errata

Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you find a mistake in one of our books-maybe a mistake in the text or the code-we would be grateful if you could report this to us. By doing so, you can save other readers from frustration and help us improve subsequent versions of this book. If you find any errata, please report them by visiting http://www.packtpub.com/submit-errata, selecting your book, clicking on the Errata Submission Form link, and entering the details of your errata. Once your errata are verified, your submission will be accepted and the errata will be uploaded to our website or added to any list of existing errata under the Errata section of that title.

To view the previously submitted errata, go to https://www.packtpub.com/books/content/support and enter the name of the book in the search field. The required information will appear under the Errata section.

Piracy

Piracy of copyrighted material on the Internet is an ongoing problem across all media. At Packt, we take the protection of our copyright and licenses very seriously. If you come across any illegal copies of our works in any form on the Internet, please provide us with the location address or website name immediately so that we can pursue a remedy.

Please contact us at [email protected] with a link to the suspected pirated material.

We appreciate your help in protecting our authors and our ability to bring you valuable content.

Questions

If you have a problem with any aspect of this book, you can contact us at [email protected], and we will do our best to address the problem.

Previous Section
Next Chapter