Book Image

Building Serverless Microservices in Python

By : Richard Takashi Freeman
Book Image

Building Serverless Microservices in Python

By: Richard Takashi Freeman

Overview of this book

Over the last few years, there has been a massive shift from monolithic architecture to microservices, thanks to their small and independent deployments that allow increased flexibility and agile delivery. Traditionally, virtual machines and containers were the principal mediums for deploying microservices, but they involved a lot of operational effort, configuration, and maintenance. More recently, serverless computing has gained popularity due to its built-in autoscaling abilities, reduced operational costs, and increased productivity. Building Serverless Microservices in Python begins by introducing you to serverless microservice structures. You will then learn how to create your first serverless data API and test your microservice. Moving on, you'll delve into data management and work with serverless patterns. Finally, the book introduces you to the importance of securing microservices. By the end of the book, you will have gained the skills you need to combine microservices with serverless computing, making their deployment much easier thanks to the cloud provider managing the servers and capacity planning.
Table of Contents (13 chapters)
Title Page
Dedication

Setting up your serverless environment

If you already have an AWS account and configured it locally you can skip this section, but for security reasons, I recommend you enable Multi-Factor Authentication (MFA) for console access and do not use the root user account keys for the course.

There are three ways to access resources in AWS:

  • AWS Management Console is a web-based interface to manage your services and billing.
  • AWS Command Line Interface is a unified tool to manage and automate all your AWS services.
  • The software-development kit in Python, JavaScript, Java, .NET, and GO, which allows you to programmatically interact with AWS.

Setting up your AWS account

It's very simple to set up an account; all you need is about five minutes, a smartphone, and a credit card:

  1. Create an account. AWS accounts include 12 months of Free Tier access: https://aws.amazon.com/free/.
  2. Enter your name and address.
  3. Provide a payment method.
  4. Verify your phone number.

This will create a root account, I recommend you only use it for billing and not development

Setting up MFA

I recommend you use MFA as it adds an extra layer of protection on top of your username and password. It's free using your mobile phone as a Virtual MFA Device (https://aws.amazon.com/iam/details/mfa/). Perform the following steps to set it up:

  1. Sign into the AWS Management Console: https://console.aws.amazon.com.
  2. Choose Dashboard on the left menu.
  3. Under Security Status, expand Activate MFA on your root account.
  4. Choose Activate MFA or Manage MFA.
  1. In the wizard, choose Virtual MFA device, and then choose Continue.
  2. Install an MFA app such as Authy (https://authy.com/).
  3. Choose Show QR code then scan the QR code with you smartphone. Click on the account and generate an Amazon six-digit token.
  4. Type the six-digit token in the MFA code 1 box.
  5. Wait for your phone to generate a new token, which is generated every 30 seconds.
  6. Type the six-digit token into the MFA code 2 box.
  7. Choose Assign MFA:

Setting up a new user with keys

For security reasons, I recommend you use the root account only for billing! So, the first thing is to create another user with fewer privileges:

Create a user with the following steps:

  1. Sign into the AWS Management console (https://console.aws.amazon.com/).
  2. Choose Security, Identity, & Compliance > IAM or search for IAM under Find services.
  3. In the IAM page, choose Add User.
  4. For User name, type new user on the set user details pane.
  1. For Select AWS access Type, select the check boxes next to Programmatic access, AWS Console access. Optionally select Autogenerated password and Require password rest.
  2. Choose Next: Permissions:

Follow these steps to set the permission for the new user:

  1. Choose Create group.
  2. In the Create group dialog box, type Administrator for new group name.
  3. In policy list, select the checkbox next to AdministratorAccess (note that, for non-proof of concept or non-development AWS environments, I recommend using more restricted access policies).
  4. Select Create group.
  1. Choose refresh and ensure the checkbox next to Administrator is selected.
  2. Choose Next: Tags.
  3. Choose Next: Review.
  4. Choose Create user.
  5. Choose Download .csv and take a note of the keys and password. You will need these to access the account programmatically and log on as this user.
  6. Choose Close.

As with the root account, I recommend you enable MFA:

  1. In the Management Console, choose IAM | User and choose the newuser.
  2. Choose the Security Credentials tab, then choose Manage next to Assigned MFA device Not assigned.
  3. Choose a virtual MFA device and choose Continue.
  4. Install an MFA application such as Authy (https://authy.com/).
  5. Choose Show QR code then scan the QR code with you smartphone. Click on the Account and generate an Amazon six-digit token.
  6. Type the six-digit token in the MFA code 1 box.
  7. Wait for your phone to generate a new token, which is generated every 30 seconds.
  8. Type the six-digit token into the MFA code 2 box.
  9. Choose Assign MFA.

Managing your infrastructure with code

A lot can be done with the web interface in the AWS Management Console. It's a good place to start and help you to understand what you are building, but most often it is not recommended for production deployments as it is time-consuming and prone to human error. Best practice is to deploy and manage your infrastructure using code and configuration only. We will be using the AWS Command-line Interface (CLI), bash shell scripts, and Python 3 throughout this book, so let's set these up now.

Installing bash on Windows 10

Please skip this step if you are not using Windows.

Using bash (Unix shell) makes your life much easier when deploying and managing your serverless stack. I think all analysts, data scientists, architects, administrators, database administrators, developers, DevOps, and technical people should know some basic bash and be able to run shell scripts, which are typically used on Linux and Unix (including the macOS Terminal).

Alternatively, you can adapt the scripts to use MS-DOS or PowerShell, but it's not something I recommended, given that bash can now run natively on Windows 10 as an application, and there are many more examples online in bash.

Note that I have stripped off the \r or carriage returns, as they are illegal in shell scripts. You can use something such as Notepad++ (https://notepad-plus-plus.org/) on Windows if you want to view the carriage returns in your files properly. If you use traditional Windows Notepad, the new lines may not be rendered at all, so use Notepad++, Sublime (https://www.sublimetext.com/), Atom (https://atom.io/), or another editor.

A detailed guide on how to install Linux Bash shell on Windows 10 can be found at https://www.howtogeek.com/249966/how-to-install-and-use-the-linux-bash-shell-on-windows-10/. The main steps are as follows:

  1. Navigate to Control Panel | Programs | Turn Windows Features On Or Off.
  2. Choose the check box next to the Windows Subsystem for Linux option in the list, and then Choose OK.
  3. Navigate to Microsoft Store | Run Linux on Windows and select Ubuntu.
  4. Launch Ubuntu and set up a root account with a username and password the Windows C:\ and other drives are already mounted, and you can access them with the following command in the Terminal:
$ cd /mnt/c/

Well done, you now have full access to Linux on Windows!

Updating Ubuntu, installing Git and Python 3

Git will be used later on in this book:

$ sudo apt-get update
$ sudo apt-get -y upgrade
$ apt-get install git-core

The Lambda code is written in Python 3.6. pip is a tool for installing and managing Python packages. Other popular Python package and dependency managers are available, such as Conda (https://conda.io/docs/index.html) or Pipenv (https://pipenv.readthedocs.io/en/latest/), but we will be using pip as it is the recommended tool for installing packages from the Python Package Index PyPI (https://pypi.org/) and is the most widely supported:

$ sudo apt -y install python3.6
$ sudo apt -y install python3-pip

Check the Python version:

$ python --version

You should get Python version 3.6+.

The dependent packages required for running, testing, and deploying the severless microservices are listed in requirements.txt under each project folder, and can be installed using pip:

$ sudo pip install -r /path/to/requirements.txt

This will install the dependent libraries for local development, such as Boto3, which is the Python AWS Software Development Kit (SDK).

In some projects, there is a file called lambda-requirements.txt, which contains the third-party packages that are required by the Lambda when it is deployed. We have created this other requirements file as the Boto3 package is already included when the Lambda is deployed to AWS, and the deployed Lambda does not need testing-related libraries, such as nose or locust, which increase the package size.

Installing and setting up the AWS CLI

The AWS CLI is used to package and deploy your Lambda functions, as well as to set up the infrastructure and security in a repeatable way:

$ sudo pip install awscli --upgrade

You created a user called newuser earlier and have a crednetials.csv file with the AWS keys. Enter them by running aws configure:

$ aws configure
AWS Access Key ID: <the Access key ID from the csv>
AWS Secret Access Key: <the Secret access key from the csv>
Default region name: <your AWS region such as eu-west-1>
Default output format: <optional>

More details on setting up the AWS CLI are available in the AWS docs (https://docs.aws.amazon.com/lambda/latest/dg/welcome.html).

To choose your AWS Region, refer to AWS Regions and Endpoints (https://docs.aws.amazon.com/general/latest/gr/rande.html). Generally, those in the USA use us-east-1 and those in Europe use eu-west-1.