Book Image

Hands-On Cloud Administration in Azure

By : Mustafa Toroman
Book Image

Hands-On Cloud Administration in Azure

By: Mustafa Toroman

Overview of this book

Azure continues to dominate the public cloud market and grow rapidly thanks to a number of recent innovations. Azure's wide range of services and support has led to a large number of customers switching to Azure cloud. Hands-On Cloud Administration in Azure starts with the basics of Azure cloud fundamentals and key concepts of the cloud computing ecosystem and services. Then, you will gradually get acquainted with core services provided by Azure, including Azure VNet, types and assignments of IP addresses, and network security groups. You will also work on creating and administering Azure Virtual Machines, types of virtual machines (VMs), and design VM solutions based on computing workloads. As you make your way through the chapters, you will explore Azure App Service, discover how to host your web apps in Azure, and monitor and troubleshoot them. In the concluding chapters, you will learn more complex and abstract services, such as Azure Storage, Azure Backup, and Azure Site Recovery. You will also get to grips with Azure SQL Databases and the SQL on Azure VM concept. By the end of this book, you will have obtained practical experience of working with Azure services and Azure administration, along with maintaining, monitoring, and securing your Azure resources.
Table of Contents (13 chapters)

Understanding the Azure subscription model

Most of the cloud service providers have similar subscription models but have some unique features. We are going to concentrate on Microsoft Azure as this is the cloud service we are going to cover in this book. From now on, all features we are going to discuss are going to be Azure-specific.

For Microsoft Azure subscription, the highest level of administration is that of a tenant. Azure is a public cloud with data centers all over the world that are available to everyone. There are a few exceptions such as the US government's data center that is available only to US government institutions, the Chinese government's data center for Chinese official institutions, or the German data center only available to companies registered in Germany.

As a public cloud provider, Microsoft has to keep data separate for each user. Azure fabric is used to separate resources in the data center and tie them to a specific customer. So, even if you are sharing physical resources such as network, servers, and storage, your services can be accessed and managed only by you.

As the highest level of Azure, a tenant is created by default when you create your first Azure subscription. Many people don't realize that they already have an Azure tenant if they use Office 365. Office 365 requires Azure Active Directory and creates your first Azure tenant. I have seen many people making the mistake of creating a new Azure tenant even when they had Office 365 in use. The issue is that the tenant is tied to Azure Active Directory and creating a new tenant creates a new copy of Azure Active Directory. This makes Azure Active Directory hard to manage as you have two copies and differences appearing over time.

Creating your first Azure subscription creates a new Azure tenant and a new Azure Active Directory. There are multiple options for managing Azure Active Directory but we'll discuss that in Chapter 8, Azure Active Directory – Identity in the Cloud. Creating an additional Azure Active Directory creates a new Azure tenant as well.

The next level under tenant is Azure subscription. You can have multiple Azure subscriptions under a single tenant. Creating a new tenant will result in an empty tenant with only an Azure Active Directory without subscriptions. As Azure Active Directory has multiple tiers, you will not be able to change Azure Active Directory from Basic (that is free) to another tier without a valid subscription. A subscription is needed in order to collect usage information, generate a billing report, and finally issue an invoice for service usage.

An Azure subscription can be used to separate Azure environments by financial and administration logic. This can be done in many ways and you can design it to fit your needs. One example would be to have a single tenant at the company level and an Azure subscription for each department. This way you can assign a different administrator to each subscription/department and keep track of how much each department is spending. Another example for subscription separation would be to have different stage environments. I've seen many companies dividing their subscriptions into development, testing, and production environments and having different Azure subscriptions for each of these environments. This approach gives you the ability to administer and manage each environment separately but provides insight into how much you are spending on each environment as well.

The third part of separating resources would be using resource groups. Resource groups were introduced with the ARM model and bring many benefits. As with subscriptions, you can use resource groups to separate resources in terms of logical or billing level. An example would be to have a different resource group for each department or development/test/production environment. You can then assign different administrators to each resource group and track billing for each resource group. Note that for billing, you will still receive a single invoice at the end of the month and need to manage and track spending manually. Billing separation is much easier on a subscription level. If you need to separate invoices per department/environment, you should go with subscription separation.

Every resource in Azure can be tracked using hierarchy. Resource belongs to a resource group, resource group belongs to subscription, and subscription belongs to tenant. Logging in to the Azure portal will connect you to the default tenant. You can manage which tenant is going to be the default one as a single account can be in multiple tenants. For example, my corporate account is by default in my corporate tenant but I'm a guest user in multiple client tenants. By default, I connect to my company tenant but can select client tenants from a drop-down list. I can change my default tenant as well and select in which tenant I want to be logged in by default when I sign in to my Microsoft Azure account.

To look at this from a Microsoft Azure perspective, when you log in to Microsoft Azure with your account, Azure fabric determines to which tenant you have access, signs you in to your default tenant, and you have access to subscriptions that are in that tenant. From there you can manage all subscriptions, resource groups, and resources that belong to that tenant. By switching tenants, you have access to different subscriptions, resource groups, and resources that belong in that tenant. All this is handled by Azure fabric in order to separate client environments.

This approach is much better since the ARM model was introduced as things were much different in ASM. In ASM, after login you would have access to all subscriptions that were under that single account. Azure Active Directory wasn't tied to a specific tenant and you could have multiple Azure Active Directories in a single tenant. It was difficult to keep track of resources as there were no resource groups and the only thing separating them was the subscription level.

A similar hierarchy can be applied to resources administration too. You can assign a user to have certain access to your resources as well. The access level can have different kinds of permissions such as owner, contributor, reader, and so on. You can build custom permissions to achieve your own rules and policies. User roles can be assigned on the level of tenant, subscription, resource group, or single resource. Managing user access at the resource level can be hard and time-consuming and I wouldn't recommend this approach.

On the other hand, access to the tenant level is something you will often want to avoid because in most cases you don't want users to have same access to all resources. A few administrators can be exceptions, but this approach is something you want to avoid in general. The best and most common option is to assign users access at the subscription or resource group level. Subscription-level access can be used if you have different departments or environments for each subscription and you can assign an administrator for that department or environment as a subscription administrator. Access to the resource group level can be applied if you have a single application or environment in the resource group and assign an application/environment administrator for corresponding resource groups. These aren't the only options or models you can use but you can adjust and create whatever option best fits your needs. For example, I have seen where organizations have placed similar resources in a single resource group and assigned an administrator based on their on-premises role. All network resources would be in a single resource group and network engineers assigned to a network resource group. All databases would be placed in another resource group and a database administrator assigned as administrator for that resource group and so on.

Azure subscription types

To create your first Azure subscription, you need a few things. The first thing is to provide an email address that needs to be either a Microsoft Live account or an Office 365 account. You need to provide a phone number. Finally, you need to provide credit or debit card information along with a billing address. Credit card information is needed even for free subscriptions because Microsoft uses it to verify your identity.

When talking about Azure subscriptions, we can divide them into three different types:

  • Sponsored subscriptions
  • Pay as you go
  • Enterprise subscriptions

There are a few different sponsored subscriptions in Azure: trial, Azure pass, MSDN subscription, Azure sponsorship, and so on. What all of them have in common is that they have a certain amount of resources available to you free of charge. Another thing they have in common is that not all services are available in all regions. For example, you may be able to create an A2 standard virtual machine only when selecting the North Europe region, but you will not be able to create this virtual machine in any other region.

The Azure trial offers you $200 of service for 30 days. Subscription will expire whatever comes first: either you spend $200 or it expires at the end of the month. You need to provide credit/debit card information for this type of subscription. You can convert subscription from a trial to a pay-as-you-go model at any time using that card or by providing details of a new one. Credit card information must be provided—it's used only for identity verification, and you will not be charged any amount of money unless you specify that you want to remove the spending limit and start billing after the trial is over. What Microsoft is trying to achieve is to prevent you using Azure for anything illegal. Without credit card information, anyone could set up a trial subscription and use it to host illegal things for 30 days. At the end of a trial, a person would need to set up a new trial and continue to use Azure services for illegal content. In this case, Microsoft wouldn't be able to provide information on who was conducting illegal activities using Azure and they would be held accountable by the respective authorities.

Azure pass is offered as another type of trial subscription and offers a limited amount of credit for 30 days. This type of subscription is tied to Microsoft's official courseware and the amount of credit is determined by the course, each type having a different amount of credit. This type of subscription doesn't require a credit card as you need to register for a course and information from that registration can be used to verify your identity if needed. As with a trial, you are limited in the type of resources you can create, the amount of resources available, and the region resources can be created in.

MSDN Azure subscription is tied to a user's MSDN subscription and has a different credit amount based on the MSDN subscription level (a different amount per MSDN level, such as professional and enterprise). The amount of credit given is on a per-month basis and you get a certain amount at the beginning of each billing period (the billing period depends on the date of activation, the date of activation will be the beginning of your billing period, and the end of the billing period will be in 30 days).

The Azure subscription will be active as long as the MSDN subscription is active as well. Credit card information isn't needed as another way of identifying verification can be used (MSDN payment information). If you reach your spending limit in a single month, your resources will be deactivated and stopped until the end of the billing period.

To use these services again, you need to wait for the beginning of a new billing period or provide credit card information that will be used to charge any use over the sponsored amount. Removing the limit can be specified to apply to single months or for subscriptions. Single-month removal will remove the spending limit only for that month while subscription removal will the remove spending limit permanently and start billing every time the spending limit is reached. In the case of a single-month limit removal, the limit will be removed only for a specified month and if the issue happens again in the following month, it will disable your service. If the limit is removed from the subscription, once you reach the spending limit, it will automatically start to charge your usage.

Note that in every case, first the sponsored amount will be spent and only then will the credit card be activated. An Azure subscription for MSDN is limited for development and testing; it should never be used for commercial or production purposes. You also have a limit on the amount of resources and regions available. MSDN subscription also applies different pricing for resources. You are not charged for software licensing as this is a dev/test environment and prices of resources are much cheaper as a result.

Azure sponsorship is very similar to an MSDN subscription. It should not be used for commercial or production purposes. Azure sponsorship also has spending limits but in this case, it is not per-month but per-year. The billing period is one out of two differences between Azure sponsorship and Azure MSDN subscription, where sponsorship is billed per-year and MSDN per-month. The limit can be removed; there is a limit for resources and regions. The second difference is in that normal prices apply and you will be charged for software licenses.

Pay as you go is the most simple and most common type of Azure subscription. You sign up for an Azure subscription, provide credit card information, and this credit card is used for billing at the end of each month. The name tells you almost everything in this case: there are no limitations and you are billed for only what you use. If you don't have a single resource in your subscription, Microsoft will not charge you for only having a subscription. If you have resources in your subscription, you will be charged only for those resources. If you add some resources, you will be charged additionally. If you delete some of them, you will be charged only for those still active. There is no minimum or maximum limit on your subscription; you can spend nothing or millions per month.

Enterprise subscription requires a contract that determines a minimum amount you will spend on Azure resources. You receive a certain discount for resource prices as you commit that you will be spending a certain amount of money at a yearly level. You are charged on a monthly basis, based on the amount in the contract. Any amount that is over the minimum amount determined in the contract is billed separately at end of the year. With an enterprise subscription, there is also an option to bring your own licenses to Azure, enabling you to reuse existing licenses you have for on-premises resources.

Additionally, there is a reserved instances discount. It can be applied to both pay-as-you-go and enterprise subscriptions. You determine the number and type of virtual machines that you are going to use in the next period. The period can be 1 or 2 years. One year gives you a discount on these virtual machines and 2 years gives you an additional discount (for a longer time, and a bigger discount as you are obliged to use the service for longer). You can edit the reserved instances agreement at any time by adding or removing virtual machines. An increase in number can provide an additional discount and a decrease will result in penalties.

Deciding between IaaS or PaaS

Once a subscription is in place, you can start creating resources in order to use them and deploy your application. Choosing what to use and when can be overwhelming in terms of the broad choice Microsoft Azure has to offer. There are different approaches and different architectures we need to consider before even starting.

We have already talked about IaaS, PaaS, and SaaS. An example of Microsoft SaaS is Office 365 and, as a cloud software, it is available under a subscription model. Office 365 even runs in Azure data centers (it was the initial purpose of these data centers along with identity management—we now call this Azure Active Directory), but we will not discuss this product further as it isn't directly connected to Azure subscriptions. Our goal will be to distinguish Microsoft Azure's offerings when it comes to IaaS and PaaS.

IaaS is the first step in migration to the cloud. It's natural for traditional IT professionals to accept this as a first step in the cloud journey. Creating an Azure virtual machine is simple and from a VM level there isn't much difference between a local VM and a cloud VM. You don't have access to hardware or host components, which makes maintenance easier and cheaper. But administering and managing VM in Azure isn't much different form on-premises versions, no matter what host we used locally—Hyper-V, VMWare or something else (Microsoft Azure uses a modified version of Hyper-V hosts that are different than the version used on-premises).

You select an image for the operating system, select the size of the VM, and some other parameters. From there forward, you connect to your VM and install features and software as you see fit. You can control access, frameworks, and data for all software installed on your VM; you'll need to pay for it as part of a subscription or provide a valid license of you own. If your create a VM with Windows Server 2016 and SQL Server 2016, you will be charged extra for both licenses.

Creating a PaaS resource is even simpler than IaaS. It's easier to administer and manage as well. But, on the other hand, control is no longer completely in your hands. You can edit some key features that are predefined to have different values or to be turned on and off. But, some things are default and you are no longer able to edit them. All licenses are included in the price of resources by default.

Let's consider a simple scenario where you have a web application running on IIS in the frontend and a database on SQL Server in the backend.

For IaaS, you'll need to create two virtual machines, a web server, and a database server. In order to host your application, you'll need to set up IIS on a web server that will be running Windows Server 2016. A database server can be a VM with SQL Server 2016 running on Windows Server 2016. By now, added to our computing prices, we have two licenses for Windows Server and one license for SQL Server (price also varies in the version of SQL Server if we choose web, standard, or enterprise edition). Once we have installed and configured IIS, we need to create firewall and network security group rules that will allow us to access our application over the internet. We need to set up communication between the web server and database server and create similar rules in order to allow communication between our application and database. We already have our hands full enabling a simple scenario with IaaS.

For IaaS, we will set up the app service plan in order to host our web application and Azure SQL database for the backend. All licenses are already configured and we don't have to do anything else in terms of configuration. The process is much simpler and easier.

Note that pricing is cheaper for PaaS in most cases.

But on the other hand, PaaS doesn't always allow us to have everything we need to run our applications. If we need to use an older version of some framework, PaaS will not work. PaaS already has a preconfigured set of frameworks that you can use but you can't install anything additionally. In the case that you have some features for the database that are not supported in Azure SQL or compatibility issues, you need to use SQL Server in VM.

Overall, PaaS is usually cheaper and needs less attention than IaaS, but IaaS gives better control and better legacy support.

Service offerings are growing by the day and every few weeks we have new services and new features in Microsoft Azure. A couple of services that we mentioned are only examples for a simple scenario. IaaS does give us control as to what is going to be on our VM and offers better combinations compared to a single resource, but the PaaS list doesn't stop there. For Azure PaaS, we can create an app service, content delivery network, Azure SQL database, traffic manager, service bus, Azure functions, Azure CosmosDB, Azure storage, and Redis cache just to name a few.

Azure data platform has over 50 different services that are PaaS. The same goes for other platforms such as web, media, compute, and so on. Choosing the right service can be beneficial both when we look at the solution from a financial perspective and a performance perspective. We need to consider if some service has some limitation that will make us use another service just to cover that limit. And there could be another service that will cover both aspects and there will be no need to use an additional service. Limitations can cause performance issues if we don't look at all aspects and try to anticipate all possible scenarios. Luckily, with Azure, we are not stuck with a single solution, even if we see that we have made a mistake and the service we chose doesn't really cover our needs—we can always scale out or switch to another service completely.

Understanding the pricing of Azure resources

There are few things you need to consider regarding pricing in Microsoft Azure. There are resources that have fixed pricing and resources that have consumption pricing. Also, fixed-price resources sometimes have service level limits included in the basic price. Once you reach that limit, you will be charged extra based on consumption. This potentially turns fixed-priced resources into consumption-priced ones. There are also exceptions that have different uses applied.

Resources that have fixed pricing will be added to your bill as soon as you create them. Fixed-pricing resources are billed on a monthly basis, have a fixed price, and are added to your bill as soon as you create them. Examples of these services would be OMS or Azure Active Directory on higher tiers. A reserved public IP address is also one of the fixed-cost resources. After you delete this type of resource, it will be added to your bill for the current billing period.

Azure storage is based on consumption and you pay for the amount of data you have in your storage account. But the amount of data varies every month and there could be less data at the beginning or end of the month, but more data in the middle of the month. In this case, average consumption is calculated and you are billed on the average consumption of Azure Storage.

An example of a service that has performance-level restrictions would be bandwidth. Inbound data transfer is free of charge and you don't pay anything as long as data is going towards an Azure data center. Outbound data is free for the first 5 GB per month and you are charged extra for everything over those 5 GB. Another example of a service limit would be Azure container registry, where you have 100 minutes of CPU included but pay extra when exceeding that limit. Azure function is also an example of this type of resource where you get the first 1,000,000 executions for free and you are charged for additional executions.

Resources based on computing are calculated on a per-minute basis. If you have a virtual machine or app service running, billing will stop as soon as you delete these resources. This also applies to resources being stopped in some cases. Azure SQL database can't be stopped but pricing will be calculated on a per-minute basis. If you delete Azure SQL database, you stop paying for it the same minute it's deleted. For virtual machines and app services, you can stop these resources and you will not pay for compute hours any longer. Note that there are a few pricing details related to a virtual machine. Virtual machine price information, that you can find in Azure portal when creating virtual machine or shown in Azure calculator, is only for compute hours. A virtual machine also uses Azure storage that you pay for, separately from computing, and you will pay for storage no matter if the virtual machine is running or stopped. Some network components can create additional charges for virtual machines. For example, you can reserve a public IP address that is charged separately.

It is very important to keep track of your resources in Azure and to know what you are using, if resources are utilized, and what the limits are for current resources. If you are not using something correctly or not using it at all, you are still paying for it. This is different from a traditional IT environment where you have resources prepaid and it doesn't really matter if a single virtual machine is being used or not as long as you don't reach the resource limit at the host level. In Azure, you are paying for everything you created (or have running in some cases) so you need to keep track of active resources and what you actually need. Otherwise, the cloud can become a very expensive solution and bills will start to pile up. Financial benefits can be great but if you are not careful, it can go the other way as well. I've seen many companies using a development and test environment without control and many resources where no one knows who created them and why, let alone if there is someone actually using them. This is not as often a problem with production environments, but can happen in some cases.

Be smart and Azure will help you, but if you don't keep track of usage and billing, it can go the other way. It's your choice: you can get either promoted or fired!

ARM revolution

In April 2014, Microsoft announced a new approach to Azure with a new portal and ARM model. We already discussed how ARM and RBAC changed the administration of Azure resources and made life in the cloud much easier.

But another option ARM brought to the table was ARM templates. ARM templates are files in JSON format that contain information on all resources in a single resource group. We can use ARM templates to deploy new resources, update resources, or remove resources from a resource group. Every resource group in Azure portal has an automation blade that allows us to save an ARM template for that resource group, redeploy resources, or download a template locally. This allows us to replicate resources from a resource group quickly and simply.

This is why placing resources for a single application is common practice in Azure. If we have a complicated environment that takes some time to deploy, it can be very useful. For example, let's say that we have a SharePoint farm with Windows Server running a domain controller role, two servers for SharePoint farm, and two additional servers running SQL Server. This requires us to create a virtual network, create five servers, and add them to the virtual network. Doing this manually in the Azure portal requires some time to create. But with ARM templates we can do this in a matter of seconds.

Deploying infrastructure with ARM templates is only one option for infrastructure as a code. We can achieve similar things with PowerShell and Azure CLI as well. Using infrastructure as a code allows us to deploy infrastructure needed to run our application in a fast and reliable way. It's a more consistent option as well because we are excluding manual tasks from deployment, creating an automated process, and eliminating human error in deployment.

Another great thing about ARM templates is that they can be added to our application project and stored in a repository, and we can keep track of versions of our environment. Different versions of an application may require changes in environment and with ARM templates and code repositories we can keep track of these changes and make sure that the correct environment is deployed for the version of the application we are deploying.

To sum everything up, ARM templates are a fast, reliable, and consistent way to deploy Azure resources that allow us to keep track of environment versioning and automate deployment processes. This is especially interesting and useful if we are trying to implement DevOps in our software delivery. In combination with configuration as a code (we'll talk about this in later chapters), we can build or replicate any environment with all resources required and apply the configuration needed for everything to run correctly.

A sample of an empty ARM template is shown here:

{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
},
"variables": {
},
"resources": [
],
"outputs": {
}
}

An ARM template contains information on parameters, variables, resources, and outputs. Using this information we define what resources need to be deployed, using which parameters, and which of these parameters can be variables that can be changed. Finally, we define the output as a replay, but this part is optional.