Book Image

Mastering Machine Learning on AWS

By : Dr. Saket S.R. Mengle, Maximo Gurmendez
Book Image

Mastering Machine Learning on AWS

By: Dr. Saket S.R. Mengle, Maximo Gurmendez

Overview of this book

Amazon Web Services (AWS) is constantly driving new innovations that empower data scientists to explore a variety of machine learning (ML) cloud services. This book is your comprehensive reference for learning and implementing advanced ML algorithms in AWS cloud. As you go through the chapters, you’ll gain insights into how these algorithms can be trained, tuned, and deployed in AWS using Apache Spark on Elastic Map Reduce (EMR), SageMaker, and TensorFlow. While you focus on algorithms such as XGBoost, linear models, factorization machines, and deep nets, the book will also provide you with an overview of AWS as well as detailed practical applications that will help you solve real-world problems. Every application includes a series of companion notebooks with all the necessary code to run on AWS. In the next few chapters, you will learn to use SageMaker and EMR Notebooks to perform a range of tasks, right from smart analytics and predictive modeling through to sentiment analysis. By the end of this book, you will be equipped with the skills you need to effectively handle machine learning projects and implement and evaluate algorithms on AWS.
Table of Contents (24 chapters)
Free Chapter
1
Section 1: Machine Learning on AWS
3
Section 2: Implementing Machine Learning Algorithms at Scale on AWS
9
Section 3: Deep Learning
13
Section 4: Integrating Ready-Made AWS Machine Learning Services
17
Section 5: Optimizing and Deploying Models through AWS
Appendix: Getting Started with AWS

Implementing linear regression through Apache Spark

You are likely interested in training regression models that can take huge datasets as input, beyond what you can do in scikit-learn. Apache Spark is a good candidate for this scenario. As we mentioned in the previous chapter, Apache Spark can easily run training algorithms on a cluster of machines using Elastic MapReduce (EMR) on AWS. We will explain how to set up EMR clusters in the next chapter. In this section, we'll explain how you can use the Spark ML library to train linear regression algorithms:

  1. The first step is to create a dataframe from our training data:
housing_df = sql.read.csv(SRC_PATH + 'train.csv', header=True, inferSchema=True)

The following screenshot shows the first few rows of the dataset:

  1. Typically, Apache Spark requires the input dataset to have a single column with a vector of numbers...