Book Image

Accelerating DevSecOps on AWS

By : Nikit Swaraj
Book Image

Accelerating DevSecOps on AWS

By: Nikit Swaraj

Overview of this book

Continuous integration and continuous delivery (CI/CD) has never been simple, but these days the landscape is more bewildering than ever; its terrain riddled with blind alleys and pitfalls that seem almost designed to trap the less-experienced developer. If you’re determined enough to keep your balance on the cutting edge, this book will help you navigate the landscape with ease. This book will guide you through the most modern ways of building CI/CD pipelines with AWS, taking you step-by-step from the basics right through to the most advanced topics in this domain. The book starts by covering the basics of CI/CD with AWS. Once you’re well-versed with tools such as AWS Codestar, Proton, CodeGuru, App Mesh, SecurityHub, and CloudFormation, you’ll focus on chaos engineering, the latest trend in testing the fault tolerance of your system. Next, you’ll explore the advanced concepts of AIOps and DevSecOps, two highly sought-after skill sets for securing and optimizing your CI/CD systems. All along, you’ll cover the full range of AWS CI/CD features, gaining real-world expertise. By the end of this AWS book, you’ll have the confidence you need to create resilient, secure, and performant CI/CD pipelines using the best techniques and technologies that AWS has to offer.
Table of Contents (15 chapters)
1
Section 1:Basic CI/CD and Policy as Code
5
Section 2:Chaos Engineering and EKS Clusters
9
Section 3:DevSecOps and AIOps

Validating PRs/MRs into the develop branch from the feature branch via CodeBuild and AWS Lambda

In this section, we will basically implement a solution that gives the status of the build of the PR raised in CodeCommit. This helps the maintainer see that the PR raised is at least passing all the builds and tests. Let's have a look at the solution and understand the flow, as follows:

Figure 1.47 – Flow diagram of the solution

Figure 1.47 – Flow diagram of the solution

The following steps explains the flow of diagram:

  1. When a developer finishes their work in the feature branch, they will then raise a PR/MR to the develop branch.
  2. A CloudWatch event that is watching our repository will get triggered, and that will invoke the TriggerCodeBuildStart lambda function by passing some information.
  3. This TriggerCodeBuildStart lambda function will use the CloudWatch information and trigger an AWS CodeBuild Project to our latest commit. After that, it will create a custom message that we want on our PR activity.
  4. Once this CodeBuild event finishes, another CloudWatch event will send those build results and comments to another lambda function (TriggerCodeBuildResult).
  5. This TriggerCodeBuildResult lambda function will comment the build result on the PR in the CodeCommit activity.

To set up the solution, perform the following steps:

  1. Go to the CodeBuild section of the project and click on Create Build Project.
  2. Enter the following information:
    1. Project name: northstar-pr
    2. Description: PR-Build
    3. Repository: northstar; Branch: feature/image
    4. Environment Image: Managed Image; OS: Amazon Linux 2; Runtime: Standard; Image: aws/codebuild/amazonlinux2-x86_64-standard:3.0; Environment type: Linux
    5. Service Role: New service role
    6. Buildspec: Inset build commands; Build Commands: npm install && nohup npm start &
    7. Leave the rest at their default settings and click on Create build project.

The CodeBuild console of the northstar-pr project is shown in the following screenshot:

Figure 1.48 – CodeBuild console of northstar-pr project

Figure 1.48 – CodeBuild console of northstar-pr project

  1. Go to the AWS Lambda console and under Create a function, select Author from scratch and give the name TriggerCodebuildStart and Node.js 12.x under Runtime. In the Permissions section, you need to select a role that has permission to access AWS CloudWatch Logs, CodeBuild, and CodeCommit. You can create a policy using the lambdapermission.json file and attach it to the role.

An overview of the process is shown in the following screenshot:

Figure 1.49 – Creating a lambda function

Figure 1.49 – Creating a lambda function

  1. Go to the Code Source section and modify the index.js file. We will be using the source code present in the Accelerating-DevSecOps-on-AWS/chapter-01 folder that we downloaded for our sample application. There is a folder called TriggerCodeBuildStart that includes index.js and package.json files. Copy and paste both the files into this Lambda function and click on Deploy, as illustrated in the following screenshot:
Figure 1.50 – Lambda function code editor

Figure 1.50 – Lambda function code editor

  1. After that, go to the Configuration section and click on Environment variable to add the environment variables. The lambda function code uses three environment variables shown in the following screenshot. Click on Save once you have entered the three environment variables:
Figure 1.51 – Environment variables for TriggerCodebuildStart lambda function

Figure 1.51 – Environment variables for TriggerCodebuildStart lambda function

  1. Similarly, create another lambda function, TriggerCodebuildResult, with the code available in the TriggerCodeBuildResult folder. Deploy the Lambda function and go to the Configuration section to enter the environment variable, as illustrated in the following screenshot:
Figure 1.52 – Environment variable for TriggerCodebuildResult lambda function

Figure 1.52 – Environment variable for TriggerCodebuildResult lambda function

  1. Once we have created our Lambda function, we need to create CloudWatch event rules. Go to the CloudWatch console, click on Events on the left-hand side, and then click on Rule. After that, click on Create rule.
  2. Once you click on Create rule, you will be redirected to the Event Source section. Click on Edit in Event Pattern Preview, as illustrated in the following screenshot:
Figure 1.53 – CloudWatch rule creation with event pattern

Figure 1.53 – CloudWatch rule creation with event pattern

  1. You will get a box where you need to paste the following event pattern and then click Save:
    {
      "source": [
        "aws.codecommit"
      ],
      "detail-type": [
        "CodeCommit Pull Request State Change"
      ],
      "resources": [
        "arn:aws:codecommit:us-east-1:<Your accountID>:northstar"
      ],
      "detail": {
        "event": [
          "pullRequestCreated",
          "pullRequestSourceBranchUpdated"
        ]
      }
    }
  2. In the Targets section, select Lambda function in the dropdown, and then select the TriggerCodebuildStart function, as illustrated in the following screenshot:

Figure 1.54 – CloudWatch target

Figure 1.54 – CloudWatch target

  1. Click on Configure details to proceed to Step 2, where you need to give the rule a name and a description. Name the rule TriggerValidatePRCodeBuildStart and then save it.
  2. Similarly, create another CloudWatch rule, naming it TriggerValidatePRCodeBuildResult and giving it the following event pattern, with the target being the TriggerCodebuildResult Lambda function:
    {
      "source": [
        "aws.codebuild"
      ],
      "detail-type": [
        "CodeBuild Build State Change"
      ],
      "detail": {
        "project-name": [
          "northstar-pr"
        ],
        "build-status": [
          "FAILED",
          "SUCCEEDED"
        ]
      }
    }
  3. We now have two CloudWatch rules, TriggerValidatePRCodeBuildStart and TriggerValidatePRCodeBuildResult, as we can see in the following screenshot:
Figure 1.55 – CloudWatch rules

Figure 1.55 – CloudWatch rules

  1. We are all set up with the solution. Now, to test this solution, we need to modify the feature/image branch and create a PR to the develop branch. We will modify the northstar/source/templates/default.jade file, save it, and push it, as illustrated in the following screenshot:
Figure 1.56 – Editing feature branch code for PR to develop branch

Figure 1.56 – Editing feature branch code for PR to develop branch

  1. Now, let's create a PR from the CodeCommit console. Choose feature/image under Source and develop under Destination. Enter Raising PR for Codestar-PR for Title and Description and click on Create pull request, as illustrated in the following screenshot:
Figure 1.57 – Raising PR via CodeCommit

Figure 1.57 – Raising PR via CodeCommit

  1. If you go to the Activity section of Pull requests, you can see a comment in Activity history, as illustrated in the following screenshot:
Figure 1.58 – PR status

Figure 1.58 – PR status

  1. Meanwhile, you can see the CodeBuild logs or the CodeBuild project by going to the following screen:
Figure 1.59 – Build status

Figure 1.59 – Build status

  1. Once the build is successful, you can see the build status on the Activity page, as illustrated in the following screenshot:
Figure 1.60 – PR build status

Figure 1.60 – PR build status

  1. Once you see that the build related to the PR commit is successful, you can then merge the code to develop from feature/image by clicking on Merge (Fast forward merge), which will eventually trigger a development pipeline and deploy the new changes into the development environment, as illustrated in the following screenshot:
Figure 1.61 – northstar develop code pipeline

Figure 1.61 – northstar develop code pipeline

  1. After that, you can go to Elastic Beanstalk and open the northstarappdev endpoint, and you can then see the change on the home page, as illustrated in the following screenshot:
Figure 1.62 – Modified web application running in the development environment

Figure 1.62 – Modified web application running in the development environment

So far, we have a feature branch and an associated CodeBuild project when a PR is raised and a develop branch with its own development pipeline and environment. In the next section, we will modify the existing pipeline that came by default during the start of the project. We will rename the environment as staging and create a new production stage and environment.