Book Image

Practical OneOps

By : Nilesh Nimkar
Book Image

Practical OneOps

By: Nilesh Nimkar

Overview of this book

Walmart’s OneOps is an open source DevOps platform that is used for cloud and application lifecycle management. It can manage critical and complex application workload on any multi cloud-based infrastructure and revolutionizes the way administrators, developers, and engineers develop and launch new products. This practical book focuses on real-life cases and hands-on scenarios to develop, launch, and test your applications faster, so you can implement the DevOps process using OneOps. You will be exposed to the fundamental aspects of OneOps starting with installing, deploying, and configuring OneOps in a test environment, which will also come in handy later for development and debugging. You will also learn about design and architecture, and work through steps to perform enterprise level deployment. You will understand the initial setup of OneOps such as creating organization, teams, and access management. Finally, you will be taught how to configure, repair, scale, and extend applications across various cloud platforms.
Table of Contents (18 chapters)
Practical OneOps
Credits
About the Author
About the Reviewer
www.PacktPub.com
Customer Feedback
Preface

Amazon AMI


There are multiple reasons why you may want to run OneOps on AWS rather than your own infrastructure. Running on AWS offloads all your infrastructure management needs to Amazon so you can focus on delivery. This is crucial for you if you want to keep your DevOps team small. If the rest of your infrastructure, services, and application are also on AWS, it makes strategic sense to have your management software on AWS too. Finally, AWS provides easy options for redundancy and scalability. The downside of this option is that you cannot add your own custom cloud that runs on your own premises without considerable configuration, for example, a VPN tunnel.

OneOps provides a basic AMI in the public marketplace on Amazon AWS. This AMI comes with some basic configuration already in place to get you up-and-running. Instantiating OneOps on AWS is the quickest way you can get up and running with OneOps. There are two ways you can do this. They are as follows:

  • Using the AWS command line tool

  • Using the AWS web-based console

Knowing how to spin up the OneOps AMI from the command line comes in handy as you can make it a part of your automation or DevOps delivery chain later.

Note

You can find more details on installing the AWS command line tool here:http://docs.aws.amazon.com/cli/latest/userguide/installing.html.

Once you have the command line tool installed, the first step is to generate a keypair, which you will use to connect to the new instance you will spin up.

  1. Run the command following to generate the keypair.

    $ aws ec2 create-key-pair --key-name OneOpsAuth         --query 'KeyMaterial' --output 'text' > OneOpsAuth.pem
    
  2. Once the keypair is generated, create a security group to control the traffic to the instance.

    $ aws ec2 create-security-group --group-name OneOps-sg         --description "Oneops Security Group"
            {   
              "GroupId": "sg-05e2847d"
            }
    

    Note the GroupId of the group that you just created as we will need it when we open up ports. We will now open up access to ports 22 and 3000, port 22 so you can SSH to it remotely and 3000 because the OneOps frontend is a Ruby on Rails application and is accessible on port 3000.

    $ aws ec2 authorize-security-group-ingress --group-id         sg-903004f8 --protocol tcp --port 22 --cidr 0.0.0.0/0
            $ aws ec2 authorize-security-group-ingress --group-id         sg-903004f8 --protocol tcp --port 3000 --cidr 0.0.0.0/0
    

    Note that we opened up access to the whole world on port 22 and 3000 by giving access to 0.0.0.0/0. If you always login from a fixed set of IP addresses, such as a corporate network, you might want to secure your installation by giving access to that set of IP addresses only.

  3. Finally verify that your security group was created successfully and the permissions were applied correctly.

    $ aws ec2 describe-security-groups --group-id sg-05e2847d
            {
            "SecurityGroups": [
                {
                    "IpPermissionsEgress": [
                        {
                            "IpProtocol": "-1", 
                            "IpRanges": [
                                {
                                    "CidrIp": "0.0.0.0/0"
                                }
                            ], 
                            "UserIdGroupPairs": [], 
                            "PrefixListIds": []
                        }
                    ], 
                    "Description": "OneOps Security Group", 
                    "IpPermissions": [
                        {
                            "PrefixListIds": [], 
                            "FromPort": 22, 
                            "IpRanges": [
                                {
                                    "CidrIp": "0.0.0.0/0"
                                }
                            ], 
                            "ToPort": 22, 
                            "IpProtocol": "tcp", 
                            "UserIdGroupPairs": []
                        }, 
                        {
                            "PrefixListIds": [], 
                            "FromPort": 3000, 
                            "IpRanges": [
                                {
                                    "CidrIp": "0.0.0.0/0"
                                }
                            ], 
                            "ToPort": 3000, 
                            "IpProtocol": "tcp", 
                            "UserIdGroupPairs": []
                        }
                    ], 
                    "GroupName": "OneOps-sg", 
                    "VpcId": "vpc-45e3af21", 
                    "OwnerId": "377640004228", 
                    "GroupId": "sg-05e2847d"
                }
            ]
            }
    

    Finally, we are almost ready to launch the AMI. However, for that we need the instance ID for the correct AMI. OneOps AMI names are structured as OneOps-basic-preconf-v* where * is 1.1, 1.2.

  4. To find the latest version of image available run the command following:

    $ aws ec2 describe-images --filter "Name=name,Values=OneOps*"         --query 'Images[*].{Name:Name,ID:ImageId}'
            [
                {
                    "Name": "OneOps-basic-preconf-v1.3", 
                    "ID": "ami-076c416d"
                }, 
            {
                    "Name": "OneOps-basic-preconf-v1.2", 
                    "ID": "ami-fc1b3996"
            }
            ]
    

    Once you have the AMI ID, you are now ready to launch the OneOps instance.

  5. Run the following command to launch the AMI instance.

    $ aws ec2 run-instances --image-id ami-076c416d         --count 1 --instance-type m4.large  
            --key-name OneOpsAuth --security-group-ids sg-05e2847d
            {
            "OwnerId": "377640004228",
            "ReservationId": "r-e8a1e943",
            "Instances": [
                {
                    "Hypervisor": "xen",
                    "PublicDnsName": "",
                    "VpcId": "vpc-45e3af21",
                    "RootDeviceName": "/dev/sda1",
                    "StateReason": {
                        "Message": "pending",
                        "Code": "pending"
                    },
                    "ProductCodes": [],
                    "ClientToken": "",
                    "Architecture": "x86_64",
                    "SubnetId": "subnet-2311d009",
                    "State": {
                        "Name": "pending",
                        "Code": 0
                    },
                    "PrivateIpAddress": "172.31.56.32",
                    "InstanceId": "i-4af1d7c9",
                    "SecurityGroups": [
                        {
                            "GroupId": "sg-05e2847d",
                            "GroupName": "OneOps-sg"
                        }
                    ],
                    "EbsOptimized": false,
                    "VirtualizationType": "hvm",
                    "KeyName": "OneOpsAuth",
                    "Placement": {
                        "AvailabilityZone": "us-east-1b",
                        "Tenancy": "default",
                        "GroupName": ""
                    },
                    "StateTransitionReason": "",
                    "ImageId": "ami-076c416d",
                    "Monitoring": {
                        "State": "disabled"
                    },
                    "SourceDestCheck": true,
                    "InstanceType": "m4.large",
                    "BlockDeviceMappings": [],
                    "RootDeviceType": "ebs",
                    "PrivateDnsName":                     "ip-172-31-56-32.ec2.internal",
                    "NetworkInterfaces": [
                        {
                            "PrivateIpAddress": "172.31.56.32",
                            "PrivateIpAddresses": [
                                {
                                    "PrivateIpAddress": "172.31.56.32",
                                    "Primary": true,
                                    "PrivateDnsName":                                     "ip-172-31-56- 32.ec2.internal"
                                }
                             ],
                            "MacAddress": "12:a5:58:7e:f5:f1",
                            "VpcId": "vpc-45e3af21",
                            "Groups": [
                                {
                                    "GroupId": "sg-05e2847d",
                                    "GroupName": "OneOps-sg"
                                }
                            ],
                            "OwnerId": "377649884228",
                            "SourceDestCheck": true,
                            "Attachment": {
                                "DeleteOnTermination": true,
                                "AttachTime": "2016-03-24T19:53:13.000Z",
                                "Status": "attaching",
                                "DeviceIndex": 0,
                                "AttachmentId": "eni-attach-9a96416b"
                            },
                            "PrivateDnsName":                             "ip-172-31-56-32.ec2.internal",
                            "Status": "in-use",
                            "Description": "",
                            "SubnetId": "subnet-2311d009",
                            "NetworkInterfaceId": "eni-2aeed90f"
                        }
                    ],
                    "LaunchTime": "2016-03-24T19:53:13.000Z",
                    "AmiLaunchIndex": 0
                }
            ],
            "Groups": []
    }
    

    image-id is the ID you obtained from the previous query for the latest image version (in this case v1.3). The key name is the new key you created and the security group is the group you created. Replace the values in the preceding command as needed. Make sure the InstanceType is m4.large as OneOps needs at least 8 GB of memory to run. The image will also attach two block devices to the instance for storage. Give the instance a few minutes to start. Lastly tag the instance with a tag called OneOps so you can easily find it. For this you will need the InstanceId. For example, in the previous output, you can see the InstanceId for the instance created above is i-4af1d7c9.

That's it! Now look up the public IP of your instance and connect to it at port 3000. You should see the OneOps login screen.

$ aws ec2 describe-instances --filter "Name=tag:Name,Values=OneOps"
{
"Reservations": [
    {
        "Groups": [],
        "OwnerId": "377649884228",
        "ReservationId": "r-e8a1e943",
        "Instances": [
            {
                 "PublicIpAddress": "54.86.6.251",
                "State": {
                    "Code": 16,
                    "Name": "running"
                },
                "Architecture": "x86_64",
                "LaunchTime": "2016-03-24T19:53:13.000Z",
                "SubnetId": "subnet-2311d009",
                "ProductCodes": [],
                "PrivateIpAddress": "172.31.56.32",
                "Monitoring": {
                    "State": "disabled"
                },
                "PrivateDnsName": "ip-172-31-56-32.ec2.internal",
                "NetworkInterfaces": [
                    {
                        "Groups": [
                            {
                                "GroupId": "sg-05e2847d",
                                "GroupName": "OneOps-sg"
                            }
                        ],
                        "OwnerId": "377649884228",
                        "NetworkInterfaceId": "eni-2aeed90f",
                        "SourceDestCheck": true,
                        "SubnetId": "subnet-2311d009",
                        "PrivateIpAddress": "172.31.56.32",
                        "Status": "in-use",
                        "MacAddress": "12:a5:58:7e:f5:f1",
                            "VpcId": "vpc-45e3af21",
                        "Description": "",
                        "PrivateDnsName":                             "ip-172-31-56-32.ec2.internal",
                        "PrivateIpAddresses": [
                            {
                                "Association": {
                                    "PublicIp": "54.86.6.251",
                                    "IpOwnerId": "amazon",
                                    "PublicDnsName":                                         "ec2-54-86-6-251.compute-                                        1.amazonaws.com"
                                },
                                "PrivateDnsName":                                     "ip-172-31-56-32.ec2.internal",
                                "PrivateIpAddress": "172.31.56.32",
                                "Primary": true
                            }
                        ],
                        "Attachment": {
                            "DeviceIndex": 0,
                            "AttachTime": "2016-03-24T19:53:13.000Z",
                            "AttachmentId": "eni-attach-9a96416b",
                            "DeleteOnTermination": true,
                                "Status": "attached"
                        },
                        "Association": {
                            "PublicIp": "54.86.6.251",
                            "IpOwnerId": "amazon",
                            "PublicDnsName":                                 "ec2-54-86-6-251.compute-                                 1.amazonaws.com"
                        }
                     }
                ],
                "ImageId": "ami-076c416d",
                "RootDeviceType": "ebs",
                "KeyName": "OneOpsAuth",
                "Placement": {
                    "AvailabilityZone": "us-east-1b",
                    "Tenancy": "default",
                    "GroupName": ""
                },
                "Hypervisor": "xen",
                "RootDeviceName": "/dev/sda1",
                "Tags": [
                    {
                        "Key": "Name",
                        "Value": "OneOps"
                    }
                ],
                "VirtualizationType": "hvm",
                "SourceDestCheck": true,
                "VpcId": "vpc-45e3af21",
                "StateTransitionReason": "",
                "ClientToken": "",
                "AmiLaunchIndex": 0,
                "InstanceType": "m4.large",
                "EbsOptimized": false,
                "InstanceId": "i-4af1d7c9",
                "BlockDeviceMappings": [
                    {
                        "DeviceName": "/dev/sda1",
                        "Ebs": {
                            "AttachTime": "2016-03-24T19:53:14.000Z",
                            "DeleteOnTermination": false,
                            "VolumeId": "vol-a0c6837e",
                            "Status": "attached"
                        }
                    },
                    {
                        "DeviceName": "/dev/sdb",
                        "Ebs": {
                            "AttachTime": "2016-03-24T19:53:14.000Z",
                            "DeleteOnTermination": false,
                            "VolumeId": "vol-23c782fd",
                            "Status": "attached"
                        }
                    }
                ],
                "SecurityGroups": [
                    {
                        "GroupId": "sg-05e2847d",
                        "GroupName": "OneOps-sg"
                    }
                ],
                "PublicDnsName":                     "ec2-54-86-6-251.compute-1.amazonaws.com"
            }
        ]
    }
]
}

Although it's nice to know how to create a OneOps instance from the command line and it does come in handy if you want to make it part of your automation, the same results can be obtained in a much quicker way through the Amazon AWS console.

On the AWS console:

  1. Select EC2.

  2. Then select Instances from the left navigation bar.

  3. Click the big blue Launch Instance button on the top.

  4. On the left navigation bar click on Community AMIs.

  5. Search for OneOps.

You may see several results for OneOps AMIs as shown in the following screenshot:

Select the AMI with the latest revision number; in the screenshot preceding it is v1.3. Click Select. On the next screen you will be asked what kind of instance you want to launch. If you have the one-year free membership from Amazon, it is very tempting to launch the t2.micro instance, which is eligible for the free tier. However OneOps will give very poor performance on micro instances and may even fail to launch fully due to a lack of resources. To perform well OneOps needs at least 8 GB of memory. It is recommended you use at least the m4.large instance, which comes with 2 vCPUs and 8 GB of memory. Select m4.large and click on Next which will allow you to configure and review instance details.

On this screen a few options are of interest. Keep the number of instances to 1. Don't request a spot instance as they are best used for batch jobs and processes that are fault-tolerant. Select the defaults for the rest. Make sure you assign a public IP if you want your OneOps instance to be accessible from the outside world. You can also add protection against accidental termination and CloudWatch monitoring from here. Amazon charges extra for CloudWatch monitoring; however, if you will be using OneOps for a long period and ultimately carrying it over into production, you should select CloudWatch. Lastly the Tenancy of the instance will also affect its performance. For most purposes a shared instance should be fine.

Click on Next to review the storage. The AMI already comes with two volumes attached to it: a root volume of 10 GB and an additional volume of 40 GB. You can increase the size of the additional volume as most of OneOps is stored in /opt/oneops. You can also select the Delete on Termination options if you are just trying out OneOps since the storage volumes will not be terminated on instance termination and Amazon will charge you for volume storage. After making your changes, click Next to tag your instance. Here you can create tags as name/value pairs. Create a tag with the name Name and the value as OneOps to easily identify your instance. Click Next to configure the security group. Create a new group called OneOps-sg. Make sure port 22 is enabled so you can SSH to your instance. Also enable access to port 3000; OneOps is a Ruby on Rails application and binds to port 3000.

It is not really prudent to open access to all IP addresses and Amazon will warn you against that. If you know the range of IP addresses you will be logging in from, restrict access to that range.

You are now ready to review and launch your instance. You may get a couple of warnings. If you are on the free tier, you will get a warning that this instance is not eligible for the free tier since you selected m4.large. You will also get a warning that access to your instance is open to the world. Review the warnings and click Launch. Once you click Launch, you will be asked to either create a new keypair or use an existing keypair to attach to the instance. If you have an existing keypair that you have access to, select it here or else generate a new keypair and download the key. Be warned if you lose this key, you will be unable to SSH to the instance to keep this key in a safe place. Click on Launch instance. The instance will take some time to initialize. OneOps will also take some time to initialize. Once the initialization is done, you can connect to port 3000 via a browser and you should see a login screen as shown in the image above. You can also use the key you generated to login to the instance to have a look around. Make sure to use centos as the username. You can SSH to your instance directly from a compatible browser as shown in the following screenshot.