Book Image

Amazon EC2 Cookbook

Book Image

Amazon EC2 Cookbook

Overview of this book

Discover how to perform a complete forensic investigation of large-scale Hadoop clusters using the same tools and techniques employed by forensic experts. This book begins by taking you through the process of forensic investigation and the pitfalls to avoid. It will walk you through Hadoop’s internals and architecture, and you will discover what types of information Hadoop stores and how to access that data. You will learn to identify Big Data evidence using techniques to survey a live system and interview witnesses. After setting up your own Hadoop system, you will collect evidence using techniques such as forensic imaging and application-based extractions. You will analyze Hadoop evidence using advanced tools and techniques to uncover events and statistical information. Finally, data visualization and evidence presentation techniques are covered to help you properly communicate your findings to any audience.
Table of Contents (15 chapters)
Amazon EC2 Cookbook
Credits
About the Authors
About the Reviewer
www.PacktPub.com
Preface
Index

Architecting for high availability


Application and network errors can render the system unavailable to the user. Multi-availability zone deployments are used for building high-availability applications at the AWS region level. For implementing fault tolerance for region level failures, we have to deploy our application in availability zones spanning across different regions. If we use multiple regions, we have to use Route 53 for failover. If the primary region goes down, Route 53 fails over to the secondary region.

Increasing load on system can also cause system availability issues, but the autoscaling feature can help us solve the problem by autoscaling the number of servers during a spike in load. The number of servers is automatically reduced when the load comes back to normal levels. Detailed explanation on autoscaling is in Chapter 3, Managing AWS Resources Using AWS CloudFormation.

Building loosely coupled applications can also help avoid single points of failure. We can use Simple Queue Service (SQS) to build loosely coupled applications. Using the SQS queue size as a parameter, we can auto-scale our EC2 instances. For RDS high availability, we can configure a multi availability zone-deployment option. This will deploy the primary and secondary database instances in two different availability zones.

How to do it…

Here, we list the commands required for configuring high availability across two different regions using Route 53:

  1. Create an instance in the first region. Before launching the EC2 instance, create the required VPC, subnets, key pairs, and security groups in this region.

    $ aws ec2 run-instances 
    --image-id [ImageId] 
    --count [InstanceCount] 
    --instance-type [InstanceType] 
    --key-name [KeyPairName] 
    --security-group-ids [SecurityGroupIds]
    --subnet-id [SubnetId]
    

    The parameters used in this command are described as follows:

    • [ImageId]: This option gives the ID of the image

    • [InstanceCount]: This parameter provides the number of instances to create

    • [InstanceType]: This parameter provides the type of EC2 instance

    • [KeyPairName]: This gives a key/pair name for authentication

    • [SecurityGroupIds]: This option provides the security group ID

    • [SubnetId]: This parameter provides the ID of subnet where you want to launch your instance

  2. Create an instance in the second region. Before launching the EC2 instance, create the required VPC, subnets, key pairs, and security groups in this region:

    $ aws ec2 run-instances 
    --image-id [ImageId] 
    --count [InstanceCount] 
    --instance-type [InstanceType] 
    --key-name [KeyPairName] 
    --security-group-ids [SecurityGroupIds]
    --subnet-id [SubnetId] 
    

    The parameters used in this command are described as follows:

    • [ImageId]: This parameter provides the ID of the image

    • [InstanceCount]: This option gives the number of instances to create

    • [InstanceType]: This one gives the type of EC2 instance

    • [KeyPairName]: This parameter provides a key/pair name for authentication

    • [SecurityGroupIds]: This option gives a security group ID

    • [SubnetId]: This parameter provides the ID of the subnet where you want to launch your instance

  3. Create an AWS hosted zone in Route 53 service.

    The following command will return the name server records. Record the name server records and the hosted zone ID for the further usage.

    $ aws route53 create-hosted-zone 
    --name [Name] 
    --caller-reference [CallReference]
    

    The parameters used in this command are described as follows:

    • [Name]: This parameter gives the name of the domain

    • [CallReference]: This parameter gives a unique string that identifies the request and that allows failed create-hosted-zone requests to be retried without the risk of executing the operation twice

    Change the name servers records with your domain registrar.

    Note

    Use the following link to understand how to change name servers with GoDaddy:

    https://support.godaddy.com/help/article/664/setting-nameservers-for-your-domain-names

  4. Create health checks for previously created instances in the first region by performing the following steps:

    1. First create a virginiahc.json file with the following JSON. The IP address used is the public IP address of EC2 instance.

      {
      "IPAddress":"54.173.200.169",
      "Port":8080,
      "Type":"HTTP",
      "ResourcePath":"/index.html",
      "RequestInterval":30,
      "FailureThreshold":3
      }
    2. Execute the following command for the first region:

      $ aws route53 create-health-check 
      --caller-reference [CallReference] 
      --health-check-config [HealthCheckConfig]
      

      The parameters used in this command are described as follows:

      • [CallReference]: This is a unique string that identifies the request and that allows failed create-health-check requests to be retried without the risk of executing the operation twice

      • [HealthCheckConfig]: This option gives the health check configuration

        Syntax:

        file://virginiahc.json
    3. Create health check by running the following command. Record the health check ID for further usage.

      $ aws route53 create-health-check 
      --caller-reference 2014-11-29-17:03 
      --health-check-config file://virginiahc.json
      
  5. Create health checks for previously created instances in second region by performing the following steps:

    1. Create a second singaporehc.json file with the following JSON. The IP address used is the public IP address of EC2 instance.

      {
      "IPAddress":"54.169.85.163", 
      "Port":8080, 
      "Type":"HTTP", 
      "ResourcePath":"/index.html", 
      "RequestInterval":30, 
      "FailureThreshold":3 
      }
    2. Execute the following command for the second region:

      $ aws route53 create-health-check 
      --caller-reference [CallReference] 
      --health-check-config [HealthCheckConfig]
      

      The parameters used in this command are described as follows:

      • [CallReference]: A unique string that identifies the request and that allows failed create-health-check requests to be retried without the risk of executing the operation twice

      • [HealthCheckConfig]: This option provides the health check configuration

        Syntax:

        file:// singaporehc.json
    3. Create health check by running the following command. Record the health check ID for further usage.

      $ aws route53 create-health-check 
      --caller-reference 2014-11-29-17:04 
      --health-check-config file://singaporehc.json
      
  6. Add a primary and secondary record set to the Route 53-hosted zone by performing the following steps:

    1. Create a recordset.json file with the following JSON. In primary record set, replace health check ID and IP address with first region health check ID and EC2 public IP address accordingly. In secondary record set, replace health check ID and IP address with second region health check ID and EC2 public IP address accordingly.

      {
          "Comment":"Creating Record Set",
          "Changes":[ 
              {
                  "Action":"CREATE", 
                  "ResourceRecordSet":{ 
                      "Name":"DNS Domain Name",
                      "Type":"A", 
                      "SetIdentifier":"PrimaryRecordSet", 
                      "Failover":"PRIMARY",  
                      "TTL":300, 
                      "ResourceRecords":[
                          {
                              "Value":"54.173.200.169"
                          }
                      ],
                      "HealthCheckId":"<your first region's 
                       health check id>"
                  }
              },
              {
                  "Action":"CREATE", 
                  "ResourceRecordSet":{ 
                      "Name":" DNS Domain Name", 
                      "Type":"A", 
                      "SetIdentifier":"SecondaryRecordSet",  
                      "Failover":"SECONDARY", 
                      "TTL":300, 
                      "ResourceRecords":[
                          {
                              "Value":"54.169.85.163"
                          }
                      ],
                      "HealthCheckId":"<your second region's 
                       health check id>"
                  }
              }
          ]
      }
    2. Execute the following command to add record set:

      $ aws route53 change-resource-record-sets 
      --hosted-zone-id [HostedZoneId] 
      --change-batch [ChangeBatch] 
      

      The parameters used in this command are described as follows:

      • [HostedZoneId]: This option provides the Route 53-hosted zone ID

      • [ChangeBatch]: A complex type that contains an optional comment and the changes element

        Syntax:

        file://recordset.json
    3. Add the record set to the hosted zone by running the following command:

      $ aws route53 change-resource-record-sets 
      --hosted-zone-id Z3DYG8V5Z07JP8 
      --change-batch file://recordset.json
      
  7. Test the failover configuration by stopping the server in the primary region. You can stop your first region EC2 instance by running the aws ec2 stop-instances command.