Book Image

Learn Amazon SageMaker

By : Julien Simon
Book Image

Learn Amazon SageMaker

By: Julien Simon

Overview of this book

Amazon SageMaker enables you to quickly build, train, and deploy machine learning (ML) models at scale, without managing any infrastructure. It helps you focus on the ML problem at hand and deploy high-quality models by removing the heavy lifting typically involved in each step of the ML process. This book is a comprehensive guide for data scientists and ML developers who want to learn the ins and outs of Amazon SageMaker. You’ll understand how to use various modules of SageMaker as a single toolset to solve the challenges faced in ML. As you progress, you’ll cover features such as AutoML, built-in algorithms and frameworks, and the option for writing your own code and algorithms to build ML models. Later, the book will show you how to integrate Amazon SageMaker with popular deep learning libraries such as TensorFlow and PyTorch to increase the capabilities of existing models. You’ll also learn to get the models to production faster with minimum effort and at a lower cost. Finally, you’ll explore how to use Amazon SageMaker Debugger to analyze, detect, and highlight problems to understand the current model state and improve model accuracy. By the end of this Amazon book, you’ll be able to use Amazon SageMaker on the full spectrum of ML workflows, from experimentation, training, and monitoring to scaling, deployment, and automation.
Table of Contents (19 chapters)
Section 1: Introduction to Amazon SageMaker
Section 2: Building and Training Models
Section 3: Diving Deeper on Training
Section 4: Managing Models in Production

Setting up Amazon SageMaker on your local machine

A common misconception is that you can't use SageMaker outside of the AWS cloud. Obviously, it is a cloud-based service, and its most appealing capabilities require cloud infrastructure to run. However, many developers like to set up their development environment their own way, and SageMaker lets them do that: in this section, you will learn how to install the SageMaker SDK on your local machine or on a local server. In later chapters, you'll learn how to train and deploy models locally.

It's good practice to isolate Python environments in order to avoid dependency hell. Let's see how we can achieve this using two popular projects: virtualenv ( and Anaconda (

Installing the SageMaker SDK with virtualenv

If you've never worked with virtualenv before, please read this tutorial before proceeding –

  1. First, let's create a new environment named sagemaker, and activate it:
    $ mkdir workdir $ cd workdir $ python3 -m venv sagemaker $ source sagemaker/bin/activate
  2. Now, let's install boto3, the SageMaker SDK, and the pandas library (, which is also required:
    $ pip install boto3 sagemaker pandas
  3. Now, let's quickly check that we can import these SDKs in Python:
    $ python3
    Python 3.7.4 (default, Aug 13 2019, 15:17:50)>>> import boto3 >>> import sagemaker >>> print(boto3.__version__)1.12.39 >>> print(sagemaker.__version__)1.55.3 >>> exit

The installation looks fine. Your own versions will certainly be newer and that's fine. Now, let's run a quick test with a local Jupyter server ( If Jupyter isn't installed on your machine, you can find instructions at

  1. First, let's create a Jupyter kernel based on our virtual environment:
    $ pip install ipykernel $ python3 -m ipykernel install --user --name=sagemaker
  2. Then, we can launch Jupyter:
    $ jupyter notebook
  3. Creating a new notebook, we can see that the sagemaker kernel is available, so let's select it in the New menu, as seen in the following screenshot:
    Figure 1.2 Creating a new notebook

    Figure 1.2 Creating a new notebook

  4. Finally, we can check that the SDKs are available, by importing them and printing their version, as shown in the following screenshot:
Figure 1.3 Checking the SDK version

Figure 1.3 Checking the SDK version

This completes the installation with virtualenv. Don't forget to terminate Jupyter, and to deactivate your virtualenv:

$ deactivate

Installing the SageMaker SDK with Anaconda

Anaconda includes a package manager named conda that lets you create and manage isolated environments. If you've never worked with conda before, you should do the following first:

  • Install Anaconda:
  • Read this tutorial:

Perform the following steps to install the Sagemaker SDK with conda:

  1. Let's create and activate a new conda environment named conda-sagemaker:
    $ conda create -y -n conda-sagemaker $ conda activate conda-sagemaker
  2. Then, we install pandas, boto3, and the SageMaker SDK. The latter has to be installed with pip, as it's not available as a conda package:
    $ conda install -y boto3 pandas $ pip install sagemaker
  3. Now, let's add Jupyter and its dependencies to the environment, and create a new kernel:
    $ conda install -y jupyter ipykernel $ python3 -m ipykernel install --user --name conda-sagemaker
  4. Then, we can launch Jupyter, and check that the conda-sagemaker kernel is present in the New menu, as is visible in the next screenshot:
    $ jupyter notebook
    Figure 1.4 Creating a new conda environment

    Figure 1.4 Creating a new conda environment

  5. As shown in the following screenshot, we can create a notebook using this kernel, and check that the SDKs are imported correctly:
Figure 1.5 Checking the SDK version

Figure 1.5 Checking the SDK version

This completes the installation with conda. Whether you'd rather use it over virtualenv is largely a matter of personal preference. You can definitely run all notebooks in this book and build your own projects with one or the other.

A word about AWS permissions

Amazon Identity and Access Management (IAM) enables you to manage access to AWS services and resources securely ( Of course, this applies to Amazon SageMaker as well, and you need to make sure that your AWS user has sufficient permissions to invoke the SageMaker API.


If you're not familiar with IAM at all, please read the following documentation:

You can run a quick test by using the AWS CLI on one of the SageMaker APIs, for example, list-endpoints. I'm using the eu-west-1 region here, but feel free to use the region that is nearest to you:

$ aws sagemaker list-endpoints --region eu-west-1 {    "Endpoints": []}

If you get an error message complaining about insufficient permissions, you need to update the IAM role attached to your AWS user.

If you own the AWS account in question, you can easily do this yourself in the IAM console, by adding the AmazonSageMakerFullAccess managed policy to your role. Note that this policy is extremely permissive: this is fine for a development account, but certainly not for a production account.

If you work with an account where you don't have administrative rights (such as a company-provided account), please contact your IT administrator to add SageMaker permissions to your AWS account.

For more information on SageMaker permissions, please refer to the documentation: