Book Image

Apache Spark for Data Science Cookbook

By : Padma Priya Chitturi
Book Image

Apache Spark for Data Science Cookbook

By: Padma Priya Chitturi

Overview of this book

Spark has emerged as the most promising big data analytics engine for data science professionals. The true power and value of Apache Spark lies in its ability to execute data science tasks with speed and accuracy. Spark’s selling point is that it combines ETL, batch analytics, real-time stream analysis, machine learning, graph processing, and visualizations. It lets you tackle the complexities that come with raw unstructured data sets with ease. This guide will get you comfortable and confident performing data science tasks with Spark. You will learn about implementations including distributed deep learning, numerical computing, and scalable machine learning. You will be shown effective solutions to problematic concepts in data science using Spark’s data science libraries such as MLLib, Pandas, NumPy, SciPy, and more. These simple and efficient recipes will show you how to implement algorithms and optimize your work.
Table of Contents (17 chapters)
Apache Spark for Data Science Cookbook
Credits
About the Author
About the Reviewer
www.PacktPub.com
Customer Feedback
Preface

Building a recommendation system


Recommendation engines are one of the types of machine learning algorithms. Often, people might have experienced them using the popular websites such as Amazon, Netflix, YouTube, Twitter, LinkedIn and Facebook. The idea behind recommendation engines is to predict what people might like and to uncover relationships between the items to aid in the discovery process.

Recommender systems are widely studied and there are many approaches such as - content-based filtering and collaborative filtering. Other approaches, such as ranking models, have also gained popularity. Since Spark's recommendation models only include an implementation of matrix factorization, this recipe shows how to run matrix factorization on rating datasets from the MovieLens website.

The algorithm is available in the Spark MLLib package. The code is written in Scala.

Getting ready

To step through this recipe, you will need a running Spark cluster in any one of the modes, that is, local, standalone...