Book Image

Serverless Programming Cookbook

By : Heartin Kanikathottu
Book Image

Serverless Programming Cookbook

By: Heartin Kanikathottu

Overview of this book

Managing physical servers will be a thing of the past once you’re able to harness the power of serverless computing. If you’re already prepped with the basics of serverless computing, Serverless Programming Cookbook will help you take the next step ahead. This recipe-based guide provides solutions to problems you might face while building serverless applications. You'll begin by setting up Amazon Web Services (AWS), the primary cloud provider used for most recipes. The next set of recipes will cover various components to build a Serverless application including REST APIs, database, user management, authentication, web hosting, domain registration, DNS management, CDN, messaging, notifications and monitoring. The book also introduces you to the latest technology trends such as Data Streams, Machine Learning and NLP. You will also see patterns and practices for using various services in a real world application. Finally, to broaden your understanding of Serverless computing, you'll also cover getting started guides for other cloud providers such as Azure, Google Cloud Platform and IBM cloud. By the end of this book, you’ll have acquired the skills you need to build serverless applications efficiently using various cloud offerings.
Table of Contents (12 chapters)

Your first Lambda with AWS CLI

The AWS Command Line Interface (CLI) is a command line tool provided by AWS to manage AWS services. You can save your credentials and config into profiles, and then specify a profile while executing a command. The more you get familiar with the CLI commands, the faster you can work with AWS services, making you more productive.

In this recipe, we will deploy an AWS Lambda using AWS CLI. We will use an updated hello world. In the last recipe, we had sent and received back simple text. In this recipe, we will demonstrate the use of POJOs for sending to and retrieving JSON data from the Lambda handler.

In most of the later recipes within this book, I will be including AWS CLI commands along with either Management Console or CloudFormation steps to provide an overview of various API usages in a programming language-independent way. You can follow these API usages along with any particular programming language SDK documentation to implement it in that language. The CLI commands also help us better understand the CloudFormation templates.

Getting ready

Following are the prerequisites for this recipe:

  1. Install and configure JDK, Maven and the parent project, serverless-cookbook-parent-aws-java, and read the section as outlined in
  2. Follow the Getting ready section of the recipe Your first AWS Lambda to install and configure JDK, Maven and the parent project, serverless-cookbook-parent-aws-java, and follow the notes given in that section for code usage guidelines
  3. Configure AWS CLI as given later in this section
  4. Create an S3 bucket

Configuring AWS CLI

We can use pip or pip3 to install AWS CLI.

In a Windows machine, you can also install AWS CLI using the MSI installer following the steps at https://docs.aws.amazon.com/cli/latest/userguide/awscli-install-windows.html#install-msi-on-windows.

You use pip or pip3 to install AWS CLI as:

pip install awscli --upgrade --user

Pip is a Python package management tool that can be installed along with Python. You may replace pip with pip3 if you have installed pip3. The --upgrade option upgrades any installed requirements. The --user option installs the program to a sub-directory of your user directory to avoid modifying libraries used by operating system.

The ids or keys shown within the examples in this book should be replaced with your own ids wherever applicable. Simply copy pasting the commands will not work in such cases.

We can configure our AWS credentials in our local machine by running aws configure. This will setup a default AWS profile. You can have more named profiles if you want.

It is recommended that you create the default profile with credentials of a user with basic permissions. You can then create additional profiles for other use cases. We will be creating a user profile called admin later within this section for a user with admin permissions.

Run the below command to configure AWS CLI for the default profile. If aws command is not recognized, you will need to add it to the path.

aws configure

Provide your AWS Access Key ID, AWS Secret Access Key, Default region name, and Default output format:

AWS Access Key ID and AWS Secret Access Key is generated by AWS when you create a user with programmatic access. We had created an user and generated these credentials in the recipe Getting started with the AWS platform. You can also regenerate them later if you forget or miss them following the below steps:

  1. Log in to AWS.
  2. Go to IAM service.
  3. Click on Users from the sidebar. This will show you the user summary page.
  4. From within the user summary page, click on Security Credentials tab.
  5. Click on Create access key to create a new key. You may make the old key inactive or delete it.

The AWS Access Key ID and AWS Secret Access Key entered is stored in a file, ~/.aws/credentials, and the region name and output format is stored in a file, ~/.aws/config.

If you are using a Windows machine please refer to the sub heading Note for Windows users at the end of this section.

Verify the configuration as given as follows:

cat ~/.aws/credentials 

And next, run cat ~/.aws/config:

AWS documentation recommends creating a named profile for your admin user (for instance, a user with administrator access policy) and then using it with AWS CLI. You can add an additional profile in ~/.aws/credentials, as shown here:

You can add an additional profile by editing the file ~/.aws/config, as shown here:

Creating S3 bucket

We will be using Amazon Simple Storage Service (S3) to upload our JAR files. Therefore it would be good to do some reading on basic S3 concepts, such as S3 buckets and S3 keys.

You can create a bucket using the below command:

aws s3 mb s3://<bucket name> --profile admin

Replace the <bucket name> with your bucket's name. Remember that the S3 bucket name has to be unique across AWS.

Note for Windows users

If you are using a Windows machine the .aws folder should be present inside your user profile folder and may be found as dir %UserProfile%\.aws. You may also use the notepad command to edit files in a notepad instead of the cat command. Remember to save the notepad file if you are editing:

CLI commands that feature in this book should work on the terminals of a UNIX-style operating system, such as Linux or Mac, without any or many changes. Minor modifications may be needed to execute them in other platforms. For example, specifying multi-line commands using \ has to be replaced with ^ for the Windows OS command prompt, and ` for PowerShell.

How to do it...

We will create our Lambda, similar to in the Your First AWS Lambda recipe, but using POJOs for input and output. We will not go deep into concepts discussed previously. If in doubt, please refer to the Your First AWS Lambda recipe.

  1. Create the Maven project with only the core dependency, aws-lambda-java-core:
<groupId>tech.heartin.books.serverless-cookbook</groupId>
<artifactId>lambda-handler-with-pojos</artifactId>
<version>0.0.1-SNAPSHOT</version>

<parent>
<groupId>tech.heartin.books.serverlesscookbook</groupId>
<artifactId>serverless-cookbook-parent-aws-java</artifactId>
<version>0.0.1-SNAPSHOT</version>
</parent>

<dependencies>
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-lambda-java-core</artifactId>
<version>${aws.lambda.java.core.version}</version>
</dependency>
</dependencies>
  1. Create POJO for input:
import lombok.Data;

@Data
public class HandlerRequest {
private String name;
}
  1. Create POJO for output:
import lombok.AllArgsConstructor;
import lombok.Data;

@Data
@AllArgsConstructor
public class HandlerResponse {
private String message;
}
I have used project lombok within the POJOs to autogenerate setters, getters, and all-arguments constructor. The lombok dependencies are defined in the parent project, simple-starter-parent-java.
  1. Create a Lambda handler with input and output POJOs:
public final class MyLambdaHandler implements RequestHandler<HandlerRequest, HandlerResponse> {
public HandlerResponse handleRequest(final HandlerRequest request,
final Context context) {
context.getLogger().log("Hello " + request.getName());
return new HandlerResponse("Hello " + request.getName());
}
}
  1. Package the JAR.

We can generate JARs by running mvn clean package. Two JARs are created: one with only class files (starting with original-) and an Uber JAR with dependencies (starting with serverless-). In this recipe, we will use the original JAR.

  1. Upload the JAR file to your S3 bucket using AWS CLI:
aws s3 cp target/original-serverless-cookbook-lambda-handler-with-pojos-0.0.1-SNAPSHOT.jar s3://serverless-cookbook/lambda-handler-with-pojos-0.0.1-SNAPSHOT.jar --profile admin
Replace the bucket name serverless-cookbook with your bucket's name. We saw the steps to create a bucket in the Getting ready section. Also, --profile admin is the profile we created in the Getting ready section.
  1. Create a policy with the aws iam create-policy command:
aws iam create-policy \
--policy-name lambda_iam_policy_test \
--policy-document file://basic-lambda-permissions.txt \
--profile admin

Replace <account_id> with your account id. You can get your account number by going to the My Account page after clicking on your name on the top right of your AWS management console. The policy file is also available in the resources folder of the recipe. If successful, you should get a response with the ARN of the policy created.

You may create a more restricting policy after checking the basic Lambda permissions template at https://docs.aws.amazon.com/lambda/latest/dg/policy-templates.html.
  1. Create a role using the aws iam create-role command:
aws iam create-role \
--role-name lambda_iam_role_test \
--assume-role-policy-document file://iam-role-trust-relationship.txt \
--profile admin

The policy file is available in the resources folder of the recipe. If successful, you should get a response with the arn of the role created.

Trust relationship policies allow the Lambda service to assume this role whereas the standard policy document is attached to a role to allow or deny access to resources.
  1. Attach the policy to the role:
aws iam attach-role-policy \
--role-name lambda_iam_role_test \
--policy-arn arn:aws:iam::<account_id>:policy/lambda_iam_policy_test \
--profile admin

Replace <account_id> with your account number.

  1. Create a Lambda function providing the role and the S3 location:
aws lambda create-function \
--function-name demo-lambda-with-cli \
--runtime java8 \
--role arn:aws:iam::<account_id>:role/lambda_iam_role_test \
--handler tech.heartin.books.serverlesscookbook.MyLambdaHandler::handleRequest \
--code S3Bucket=serverless-cookbook,S3Key=lambda-handler-with-pojos-0.0.1-SNAPSHOT.jar \
--timeout 15 \
--memory-size 512 \
--profile admin

Replace <account_id> with your account number. The code option can accept the shorthand form as used here, or a JSON.

  1. Invoke our Lambda from CLI:
aws lambda invoke \
--invocation-type RequestResponse \
--function-name demo-lambda-with-cli \
--log-type Tail \
--payload '{"name":"Heartin"}' \
--profile admin \
outputfile.txt

In certain platforms, you might have to add escaping for the payload specified in the command line. This is not required as the payload is specified as a file, as here:

--payload file://input.txt \

The output can be viewed in the outputfile.txt file:

  1. Note the following regarding cleanup roles, policy, and Lambda.

To delete Lambda, perform the following:

aws lambda delete-function \
--function-name demo-lambda-with-cli \
--profile admin

To detach policy from the role, perform the following:

aws iam detach-role-policy \
--role-name lambda_iam_role_test \
--policy-arn arn:aws:iam::<account_id>:policy/lambda_iam_policy_test \
--profile admin

Replace <account_id> with your account number.

To delete a role, note the following:

aws iam delete-role \
--role-name lambda_iam_role_test \
--profile admin

To delete policy, perform the following:

aws iam delete-policy \
--policy-arn arn:aws:iam::<account_id>:policy/lambda_iam_policy_test \
--profile admin

Replace <account_id> with your account number.

How it works...

The following are the important details and concepts that were introduced in this recipe:

Creating a role and attaching a policy

You need to create a role with a trust policy that allows our Lambda to assume the role. You also need to attach a policy that has CloudWatch permissions for logging.

Lambda memory-size and timeout

When creating a function from CLI, the default value of timeout is 3 seconds, and default value memory-size is 128 MB, which may not be sufficient for Lambdas with Uber JARs, and you may get a timeout exception or Process exited before completing request. Hence, I have set a higher timeout and memory-size. Other parameters are mostly self-explanatory.

S3 Bucket and Key

Amazon S3 is an object store. Objects (files) are stored as simple key-value pairs within containers called buckets. Bucket names have to be unique across AWS. There is no folder hierarchy within the buckets like traditional file systems. However, we can simulate folder structure with hierarchical key names. For example, consider the folder1/folder2/file.txt key, that simulates a folder-like structure. Read more about simulating folders in S3 at https://docs.aws.amazon.com/AmazonS3/latest/user-guide/using-folders.html.

Cleaning up

You need to do a cleanup in the following order:

  1. Delete Lambda that uses the role
  2. Detach policy from role
  3. Delete role and policy
We cannot delete a role without detaching all policies. We can however delete a role without deleting the Lambda. If you try to invoke the Lambda before attaching another role, it will give you an error such as—The role defined for the function cannot be assumed by Lambda.

There's more...

Once you get familiar with the AWS CLI commands, it is much faster and easier to work with AWS CLI, rather than navigate through the pages of AWS management console. This chapter covers only a very basic use case. Please follow the links in the See also section and try out more examples with AWS CLI and Lambda.

See also