Businesses are rapidly transforming to the digital era. Companies are reinventing processes and cultures to deliver enhanced experience to their customers using digital technologies. This drives the need to build new capabilities and modernize existing applications using the latest technology more quickly. Enterprises are trying to stay ahead of the competition. Being late to market can mean missed opportunities, lost revenue, or, even worse, going out of business. Companies who have been agile and successful are leveraging cloud at the heart of this digital transformation. Furthermore, they are taking a hybrid multi-cloud strategy and approach consisting of on-premises, private, and public clouds to drive better efficiency, performance, and cost optimization. For a business rapidly transforming into a digital enterprise that relies on a hybrid multi-cloud environment to do so, the security threats and attack surface become greater. It is critical to stay ahead of threats, protect valuable data and resources, and achieve regulatory compliance. This chapter discusses digitization trends, the hybrid cloud strategy adopted by enterprises, and the related security challenges.

In this chapter, we’re going to cover the following topics:

• The evolution of the cloud
• The digitization trends that drive opportunities and challenges for hybrid cloud solutions
• Security in the digital hybrid multi-cloud era

Driven by trends in the consumer internet, cloud computing has become the preferred way to consume and deliver IT solutions and services. Before we dive deeper into cloud security, it is important to understand some basic aspects of the cloud, the emerging trends in cloud solutions, culture, technologies, and modern development and delivery models.

Defining cloud computing

Let’s start by understanding and defining the term cloud computing in detail. It comprises two words – cloud and computing. So, simply put, it is computing that you can offer on the cloud. What exactly is the cloud referred to here? IT architects used the cloud symbol to represent the internet or the network in their drawings. The term cloud has evolved as a metaphor for the internet. Computing could be any goal-oriented activity requiring or benefiting from the usage of IT, which includes hardware and software systems used for a wide range of purposes – collecting, storing, processing, and analyzing various kinds of information. Cloud computing has evolved over time from utility computing to what it is today, enabled by virtualization, automation, and service orientation.

The following diagram defines the key elements of cloud computing:

Figure 1.1 – Cloud computing

There are several definitions that you can find on the web for cloud computing. National Institute of Standards and Technology (NIST) has promoted the effective and secure use of cloud computing technology within government and industry by providing technical guidance and promoting standards. According to NIST, cloud computing is a pay-per-use model of enabling available, convenient, and on-demand network access to a shared pool of configurable computing resources (for example, networks, servers, storage, applications, and services) that can be rapidly provisioned and released with minimal management effort or service provider interaction. In general, most of the companies have agreed on certain general characteristics or essentials that NIST has pioneered that qualify any internet-based computing to be referred to as a cloud. They are the following:

• On-demand self-service: Cloud computing provides a catalog through which a consumer can request to provision any kind of service – computing involving a server, network, and storage or a middleware service such as a database or a software service such as email. This catalog provides self-service without requiring manual intervention on the part of the service provider.
• Ubiquitous network access: The key premise of cloud computing is that all the services and capabilities provided are accessible through the network. This can be the internet in the case of a public cloud or the intranet in the case of a private cloud. The resources on a cloud can be accessed through a variety of devices such as computers, mobile phones, and IoT devices over the network through multiple protocols.
• Location-independent resource pooling: A cloud’s business value comes from the economy of scale that is achieved by resource pooling. The provider pools the available computing resources and makes them dynamically available to clients based on demand. Physical resources including compute, network, and storage are pooled and leveraging virtualization assigned to clients in a multi-tenant model. In certain cases, consumers may not even know the exact location of the provided resources.
• Rapid elasticity: The cloud provides a means to rapidly scale up or scale down based on the demand. For the consumer, this is a very valuable business advantage of cloud solutions, as it requires them to only invest in resources when they need to. For instance, cloud consumers can start small with addressing requirements for one region or country and then scale their operations across the globe. Modern cloud technologies offer running applications and managing data without having to worry about infrastructure. Technologies such as serverless computing provide rapid elasticity and scale at a lower cost.
• Pay per use: Each cloud service is monitored, metered, and facilitates chargeback. This allows providers to promote their subscription plans and consumers to choose a billing model that is optimal for their resource usage. One example is a time-based pricing model – a per hour, per minute, or per second basis for resources such as servers. A tiered pricing model provides consumers to choose a plan from a set of price points that map to their volume or period of consumption – such as for storage, network bandwidth, or data used. Certain other services such as authentication or validation services can be consumed from the cloud with a plan that is based on active user accounts per month. The chargeback to specific departments inside the organization is now also possible with an accounting model supported by the providers and the ability to tag cloud resources to specific departments.

Cloud personas

There are several actors typically involved in building and operating a cloud solution. Their roles and responsibilities and their relationships with other actors vary based on the industry:

• Business owners: This actor’s responsibilities are to make appropriate cloud investment decisions. This section is more focused on the innovation and agility that the cloud can provide for their business. Once an organization has started with cloud solutions, then there are some typical actors that are involved in the day-to-day operational consumption and provision of cloud services.

Cloud personas and their roles are shown in the following diagram and described in the section that follows:

Figure 1.2 – Cloud personas

• Cloud service consumer: The enterprise or end user who subscribes and uses the cloud-based application or service.
• Cloud service provider: The organization that defines, hosts, and delivers cloud computing services to its consumers.
• Cloud service creator or developer: The organization or developer who creates and publishes the cloud service on a catalog for consumption.

Out of all the roles across all these organizations, the key roles from an implementation and operation perspective are the following:

• Cloud administrator who can perform the following tasks:
  • Setting up the cloud account(s) for the organization
  • Defining the users, teams, and their associated roles
  • Allocating or defining the quota for projects and users with the associated charges
  • Approving or denying requests for provisioning or de-provisioning cloud resources
  • Monitoring consumption by project
• Cloud user: Accesses or uses the cloud deployed applications, services, or provisioned resources (for example, the application, storage, or servers available to them).

There are variations within these two roles depending on the cloud provider and consumer organization design. There is more rationalization of these traditional roles in the modern context. These roles include the following:

• Cloud solution architect: The person with the knowledge and skills on how to design applications that can effectively leverage cloud capabilities. They understand specific cloud environments, such as AWS, Azure, IBM, and Google, and leverage their services and technologies to build highly scalable, performant, and available applications.
• Cloud DevOps engineer: A cloud user who is primarily responsible for developing the application component or service. The Dev-Ops engineer is also responsible for building the pipeline to deploy, monitor, and operate the service. DevOps speeds up software development and delivery, bringing close collaboration with engineering and operations teams.
• Service Reliability Engineer (SRE): Primarily responsible for improving the reliability of services through collaboration with development, proactive monitoring, and optimization of redundancies in operations. SRE is an integral part of modern cloud development teams who are involved in proactive testing, observability, service reliability, and speed.
• Security and compliance focal: Core members of the cloud teams who ensure the services are designed, developed, and deployed securely on the cloud. Ensuring services meet regulatory and security compliance requirements is the responsibility of the security and compliance focal. These resources define security policies and procedures, execute audit checks and governance related to backup, and restore automation for security and compliance tasks.

Cloud deployment models

Driven by trends in the consumer internet, cloud computing has become the preferred way to consume and deliver IT services. The cloud supports multiple deployment models based on the given requirements. The capabilities delivered by cloud are accessible via a cloud catalog and categorized based on the IT service delivered. These integrated services or layers of IT-as-a-Service are often referred to as cloud deployment models. The details of each of the cloud deployment models are shown in the following diagram:

Figure 1.3 – Cloud deployment models

The different deployment models are as follows:

• Infrastructure as a Service (IaaS): In this service delivery model, IT infrastructure is delivered over the network to consumers. This includes the compute (servers), network, storage, and any other data center resources. IaaS provides the ability to rapidly scale up or scale down infrastructure resources. IaaS consumers can concentrate on deploying and running their software, services, or applications without having to worry about managing or controlling the underlying resources.
• Platform as a Service (PaaS): Provides a platform for consumers to develop and deploy their applications. While IaaS provides the infrastructure resources, PaaS provides the programming languages, tools, and platforms to develop and deploy applications. Consumers have the ability the to control deployed applications and operating systems and environments.
• Software as a Service (SaaS): The cloud deployment model where application and services are made available to clients. In this scenario, customers can use a service without having to worry about the development, deployment, or management of these applications. In the SaaS model, the provider takes care of making the applications available to multiple clients. End users need not install or manage any software on their side and can access the applications through their devices of choice. Popular services or applications provided in the SaaS model are e-mail, ERP, and CRM.
• Business Process as a Service (BPaaS): An emerging model on top of SaaS where customers can consume business processes such as accounting and payroll, or HR processes such as travel and expense management as a service. These business services are accessed via the internet and support multiple subscription plans as advertised by the provider. The consumer can choose from these plans and subscribe to the services based on their requirements.

Cloud delivery models

The support for different delivery models is the critical success factor of the cloud for business. The flexible cloud delivery models or cloud types are shown in the following diagram:

Figure 1.4 – Cloud types (delivery models)

We shall see the various types in detail:

• Private cloud: Refers to resource pooling and sharing IT capabilities within the enterprise or behind a firewall. These are often managed privately and run by the enterprise itself and made available to the users on their intranet. A private cloud provides more flexibility to the enterprise in terms of the customization of services. At the same time, a private cloud also drives internal efficiency, standardization, and best practices. Since the resources and management are mostly local or dedicated, private cloud provides tighter control and visibility.
• Public cloud: Refers to a standardized set of business, application, or IT services provided as a service over the internet. In this model, the service provider owns and manages the service and access is by subscription. Multitenancy is a key characteristic of public cloud services that enable economies of scale. The flexible price per use basis is applicable and greater discounts apply to a committed higher usage.
• Hybrid cloud: Combines the characteristics and delivery models of both public and private clouds. The hybrid cloud as a solution combines the best of all worlds – on-premises, private clouds, and multiple public cloud services. In a hybrid cloud model, a solution can have components running on-prem on a private cloud or enterprise infrastructure connecting to services running on a public cloud. A hybrid cloud strategy is preferred by businesses, as it provides greater flexibility and resiliency for scaling workloads based on demand at reduced cost.
• Multi-cloud: Refers to leveraging services provided by more than one cloud – refers to the use of private and public services and their integration. A business may have multiple services across IaaS, PaaS, and SaaS provided by multiple vendors. A multi-cloud approach consists of a mix of major public cloud providers or hyperscalars, namely Amazon Web Services (AWS), Google Cloud Platform (GCP), Microsoft (Azure), and IBM.
• Hybrid multi-cloud: If the multi-cloud deployment includes a private cloud or an on-premise data center as well, then the cloud deployment can actually be considered a hybrid multi-cloud. We also see several variations of adoption of these cloud delivery and consumption models.

From cloud to hybrid multi-cloud

More cloud service types are emerging and guiding the development of the IT industry. These multiple delivery models can co-exist and integrate with traditional IT systems. The cloud type and delivery model selection depend on the workload and the intended benefits.

The key intended benefits from cloud are as follows:

• Speed: Capability to provision on demand and elastically scale computing resources (infrastructure, platforms, software, and business services). This is enabled through automated secure and managed provisioning process. Most cloud computing services are provided through self-service catalogs on demand. A big set of computing resources and environments can be automatically provisioned in minutes instead of having to wait for hours and days. The delivery of services more rapidly is enabled with automation and less human intervention. With proper automation, this ensures fewer errors and fulfillment of requested qualities of service or Service Level Agreements (SLAs).
• Cost: Enterprises don’t have to invest in buying hardware and software for their data centers, as well as incurring the cost of managing these resources. Depending on the delivery and consumption model, the cost and security of the cloud are defined through a shared responsibility matrix that’s documented and reviewed regularly. The cloud provides a way to cut down on the enterprise capital expenses (Capex) on racks, servers, cooling, electricity, and the IT service professionals for managing the infrastructure. The cloud provides a more efficient pricing model and lowers both capital and operational expenditure.
• Flexibility: Businesses need to adjust the IT resources based on the market demands. They need to balance performance, security, availability, and scale based on the business requirements. The cloud provides a seamless and efficient way to manage availability, resilience, and security with flexibility to move workloads across on-premise, private, and public infrastructures and services.
• Resiliency: Improved risk management through improved business resiliency. Improved time to market and acceleration of innovation projects. Cloud computing makes data backup, disaster recovery, and business continuity seamless and inexpensive with multiple availability zones on a cloud provider’s network.
• Efficiency and global scale: The benefits of cloud computing services include the ability to scale elastically. That means rapidly expanding to new geographies with the right amount of IT resources. The cloud not only optimizes the IT resources but also frees up time for skilled resources to focus on innovative and future-looking projects. The cloud helps significantly improve energy efficiency through sharing and the optimal usage of resources. The cloud infrastructure and services are upgraded to the latest ones at a faster pace to provide fast and efficient computing hardware and services. This offers several benefits over traditional data centers, including reduced network latency for applications with multiple availability zones and greater economies of scale.

Most enterprises start with something under their control to optimize what is behind their firewalls. So, the initial interest was tremendously geared toward private clouds – in both large enterprises and the mid-market. There was great interest initially in public cloud services for infrastructure services especially. Businesses have become comfortable moving workloads externally with domain applications available on the public cloud. This has resulted in a proliferation of hybrid clouds with the need for businesses to integrate their private environments with public cloud services.

Enterprises are seeking to get a deeper understanding of their data and provide differentiated, personalized experiences for their employees, customers, and partners. This requires modern applications to be created that are more responsive and can be used by clients across different types of devices. This also requires collecting a lot more data and applying artificial intelligence and machine learning to create personalized insights. This experience must be highly scalable, available, and made available for large set of users. This means it has to be built and managed on hybrid multi-cloud platforms leveraging an automated DevOps pipeline. We will discuss the impact of this digital transformation across architecture, application, data, integration, management, automation, development, and operations. Security and compliance are important cross-cutting concerns that needs to be addressed for each of these areas as part of this transformation.

Application modernization

The key opportunities and challenges with application modernization in the context of hybrid cloud are discussed in the following diagram:

Figure 1.5 – Application modernization

The key trends in application modernization and migration to the cloud are listed as follows:

• Cloud-native applications: Companies are building new capabilities as cloud-native applications that are based on microservices. Taking a cloud-native approach provides high availability. Applications are able to launch to the market quickly and scale efficiently by taking a cloud-native approach. Microservices have emerged as the preferred architectural style for building cloud-native applications. This structures an application as a collection of services that are aligned to business requirements, loosely coupled, independently deployable, scalable, and highly available.
• Modernize and migrate existing applications to the cloud: Companies are looking at various options to modernize their legacy applications and make it ready for the digital era. They are considering the following:
  • Lifting and shifting to the cloud, where the application and associated data is moved directly to cloud without any changes or redesigning.
  • Re-platforming to a virtualized platform.
  • Exposing application data through APIs.
  • Containerizing the existing application.
  • Refactoring or completely rewriting into new microservices.
  • Strangling the monolith and moving to microservices over time, where a large legacy application is restrained from adding new features and incrementally transformed to new services based on cloud-native principles.
• Built-in intelligence: Modern apps include cognitive capabilities and participate in automated workflows. We are seeing this in all the domains across banking, telecom, insurance, retail, HR, and government. Bots or applications answer human queries and provide a better experience. This positive technology also is playing a key soldier role in the healthcare sector to help mankind fight the pandemic.
• Modernizing batch functions in the cloud: This is another area that is growing rapidly. Some capabilities such as serverless computing and code engines help to build great applications in shorter time frames. These concepts will be discussed in detail in a later chapter. Essentially, these capabilities or application styles help offload otherwise long-running and resource-hungry tasks to asynchronous jobs. Thus, the cloud provides optimized scale and cost efficiency. Batch jobs that take lot of runtime and processing power or cost in costly legacy systems may be done at a much lower cost with these computing styles.
• Runtimes: As part of their modernization strategies, enterprises are evaluating how to minimize the cost of ownership and operations of the apps as well. To this end, some of the traditional applications are getting rearchitected to leverage cloud-native capabilities such as cloud functions or code engines. Another big set of applications are those that needs to be sunset or retired because the compilers or runtimes are going out of service. There is a demand for newer runtimes and faster virtual machines with lighter footprints in terms of resource usage.
• Tools: 80% of the core applications are still on legacy platforms. For businesses to move them to the cloud requires greater automation. Many intelligent tools for the discovery and extraction of logic and business rules that are more domain- or industry-focused exist.

Data modernization and the emergence of data fabric

We are at a tipping point in history where technology is transforming the way that business gets done. Businesses are intelligently leveraging analytics, artificial intelligence, and machine learning. How businesses collect, organize, and analyze data and infuse artificial intelligence (AI) will be the key to this journey being successful.

There are several data transformation use case scenarios for a hybrid multi-cloud architecture. Many companies have to modernize monoliths to cloud-native with distributed data management techniques to deliver digital personalized experiences. This involves transforming legacy architecture characterized by data monoliths, data silos, the tight coupling of data, high Total Cost of Ownership (TCO), and low speeds into new technologies identified by increased data velocity, variety, and veracity. Data governance is another key use case where enterprises need visibility and control over the data spread across hybrid multi-cloud solutions, data in motion, and data permeating enterprise boundaries (for example, the blockchain). Enterprises need to better efficiency and resiliency, with improved security for their data middleware itself and the workloads.

The greatest trend is related to delivering personal and empathetic customer service and experiences anchored by individual preferences. This is nothing new but how we enable this in a hybrid cloud is important. Customers have their System of Records (SoRs) locked in filesystems and traditional databases. There is no easy way to introduce new digital products that leverage these legacy data sources. What is required is to have a strategy to unlock the data in the legacy system to participate in these digital plays. To do this, enterprises take multiple approaches, which are listed as follows:

• One is to leave the SoRs or the Source of Truth (SoT), wherever it is, and add the ability to access the data through APIs in order to participate in digital transactions.
• A second option is to move the core itself to the cloud, getting rid of the constraints to introduce new products more quickly with data on the cloud in the process. In this model, a single legacy database could end up as multiple databases on the cloud. This will remove the dependency on data for the new development teams, thus speeding up the delivery of new digital capabilities.

Another key observation is that the line between transactional and analytical worlds is becoming thinner. Traditionally, enterprises handled their transactional data and analytics separately. While one side was looking into how to scale rapids to support transactions from multiple users and devices, on the other side, analytics systems were looking to provide insights that could drive personalization further. Insights and AI help optimize production processes and act to balance quality, cost, and throughput. This use case also involves using artificial intelligence services to transform the customer experience and guarantee satisfaction at every touchpoint. Within the industrial, manufacturing, travel and transportation, and process sectors, it is all about getting greater insight from the data that’s being produced. This data is used to drive immediate real-time actions. So, we see operation analytics (asset management), smarter processes (airport or port management), information from edge deployments, security monitoring driving new insights, and the insights, in turn, driving new types of business.

The emergence of data fabric

Businesses are looking to optimize their investment in analytics to drive real-time results to deliver personalized experiences with improved reliability and drive optimization in operations and response. This is their objective – to raise maturity on the journey to becoming a data-driven organization. Companies invest in industry-based analytics platforms that will drive this agenda related to cognitive capabilities to modernize the ecosystem and enable higher value business functions.

Given the intermingling of transactional data and analytics data, something called the data fabric emerges. It simply means companies don’t need to worry about where the data is, how it is stored, and what format it is available in. The data fabric does the hard work of making it available to the systems at the time they need it and in the form required.

The concept and value of a data fabric bridging the transactional and analytics world is detailed in the following diagram:

Figure 1.6 – The data fabric

Cloud, data hub, data fabric, and data quality initiatives will be intertwined as data and analytics leaders strive for greater operational efficiency across their data management landscape. Data fabrics remain an emerging key area of interest, enabled by active metadata.

Various architecture aspects of this fabric leverage hybrid multi-cloud containerized capabilities to accelerate the integration of AI infusion aspects with transaction systems.

Lowering the cost of data acquisition and management is another key area that enterprises are dealing with. They are required to manage heterogenous transactional and analytical data environments. So, they need help with decision data and analytics workload dispositions to the cloud. Cost optimization through adopting new cloud-based database technologies is accelerating the move from on-premises databases to cloud-based database technologies. A hybrid cloud strategy providing efficient solutions for how data is stored, managed, and accessed is a prerequisite to these activities.

Transactional and master data management within an organization changes from application to application in terms of its interpretation and granularity. This is another evolving theme of central versus distributed management of data. The fast-adopted model is to synchronize access to this data across the organization orchestrated through a data fabric or DataOps.

Integration, coexistence, and interoperability

The majority of enterprise clients rely on a huge integration landscape. The traditional enterprise integration landscape is still the core of many businesses powering their existing Business to Business (B2B), Business to Consumer (B2C), and Business to Employee (B2E) integrations. Enterprises need the agility of their integration layer to align with the parallel advances occurring in their application delivery.

When creating new systems of engagement, cloud engineering teams are looking for new ways to integrate with systems of records. They want to rearchitect, refactor, or reconstruct this middle layer. The demands on this middle layer are increasing as end users demand more responsive apps. Squads building new applications in the form of APIs and microservices like to be in complete control end-to-end, including at the integration layer.

The following diagram shows how the integration layer is modernized to support the needs of a hybrid cloud environment covering coexistence and interoperability aspects as well:

Figure 1.7 – Integration modernization, coexistence, and interoperability

Enterprises are adopting newer hybrid cloud architecture patterns to reach a higher level of maturity with optimized and agile integration. They are also looking for better ways to deploy and manage integrations. There is a need for an optimized CI/CD pipeline for integrations and the capability to roll out increments. Another requirement is controlling the roll-out of an application and integrating code for different target environments. Thus, we see tools for GitOps being extended to the integration layer as well.

Event hubs and intelligent workflows

Clients are looking to tap into interesting business events that occur within their enterprise and an efficient way to respond or react in real-time. The deployment of an event hub and event-based integrations are becoming the new norm. This also is the key enabler for building some of the intelligent workflows for a cognitive enterprise.

This integration model plays an important role in the hybrid cloud world, bringing together otherwise siloed and static processes. This also enables enterprises to go beyond their boundaries to create a platform for customers, partners, and vendors to collaborate and drive actions derived from real-time data. The applications (event consumers and producers) could be within or outside the enterprise. This space is set for exponential growth in the near future.

Coexistence and interoperability

Many enterprises have a clear understanding of their current landscape and the vision for their target multi-cloud architecture. However, they lack the knowledge of what their next logical step to reach the target is. The critical gap that needs to be addressed by their modernization strategy is related to technical and cultural coexistence and interoperability.

In an incremental modernization journey that is continuously iterative, coexistence and interoperability is a natural byproduct. Coexistence architecture isolates the modernized state from the current state. This layer also hides the complexities and gaps, as well as minimizes any changes to current state processing. Coexistence provides a simple, predictable experience and outcome for an organization to transition from old to new, while interoperability is the ability to execute the same functionality seamlessly in the legacy and the modernized side.

Businesses prioritize their investments and focus on acquiring new capabilities and transforming existing capabilities. Coexistence and interoperability help clients to focus on business transformation with logical steps towards realizing their target architecture. With many moving parts, as well as each of them being transformed and deployed on multicloud environments, this is an even more challenging problem to solve. This layer also needs to be carefully crafted and defined for successful multi-cloud adoption.

DevOps

DevOps is a core element of developing and deploying to the cloud. It combines the development and operation of both technological and cultural tools and practices with a perspective to drive the efficiency and speed of delivering cloud applications. It is important to understand the trends in the space and the impact of the adoption of the hybrid cloud.

Optimization of operations

Large enterprises want to ensure uninterrupted business operations with 99.99% application availability. There are traditional huge monolithic applications that can benefit from re-platforming and re-architecturing to the hybrid cloud. However, the goal is not about transformation in itself but about ensuring the cost of operations is minimized. For the FTEs who work on keeping the application alive, businesses need a better way to reduce the effort involved in releases and change management.

Over the years, these enterprises have created solid engineering teams with tons of scripts and automation to build, deploy, and maintain their existing applications. Some of these may be reused, but there is still more automation to be done to reduce manual work and the associated processes.

Leveraging observability for a better customer experience

Over the years, there has been a lot of software or application intelligence gathered by the build and deploy systems. Enterprises are looking to see how to leverage this information to provide a better experience to their clients. This is where observability comes in. Business needs to tap into this telemetry data to determine the aspects of the application that are heavily accessed following client consumption patterns. This also provides problem areas in the application that need to be addressed. Telemetry data-driven integrated operations to monitor across application environments are a requirement in this space. When observability is combined with the efforts of SREs, we have a beautiful way to create highly available solution components on the cloud that can provide a better user experience.

Several enterprises already use AI in this space to manage and understand immense volumes of data better. This includes understanding how the IT events are related and learn system behaviors. Predictive insights into this observability data with AIOps, where artificial intelligence is leveraged for operations and addressing end user engagement with ChatOps or bot-assisted self-service, is a major trend.

Automation, automation, automation

From the provisioning of new sandbox environments through infrastructure as code, development, and testing, automation is the key to do more with fewer developers. As companies build their CI/CD pipeline, automation drives the consistent process to build, run, test, merge, and deploy to multiple target environments. Automation brings QA and developer teams together to move from build, integration, and delivery to deployment. However much automation is done, there will still be a lot of areas in which to improve for enterprises and automation will always be a requirement for this.

Building pipeline of pipelines for hybrid multi-cloud

Continuous planning, continuous integration (CI), and continuous delivery (CD) pipeline, as shown in the following diagram, are critical components required to operate software delivery in a hybrid cloud:

Figure 1.8 – A pipeline of pipelines

Companies need more than one CI/CD pipeline to build and deploy to multiple environments. These DevOps pipelines need to cater to building and deploying to existing and multi-cloud environments. Few are thinking of combining pipelines that can achieve a composite model for hybrid multi-cloud deployment. This is true for applications that have huge middleware and messaging components, which are also going through transformation. Bringing legacy systems into the existing pipeline with common tools and other modern tools is key to enabling application modernization. A robust coexistence and interoperability design sets the direction for an enterprise-wide pipeline. The integration of the pipeline with the service management toolset is another important consideration. Enterprises have to automize with the right set of tools that can also be customized for the target clouds based on the requirements.

Security is a cross-cutting concern across the hybrid enterprise. In this section, we will discuss the challenges and opportunities for hybrid cloud security in the context of digitization trends. This will cover security aspects related to infrastructure, application, data, integration, secure engineering, and operations.

App modernization and security

Businesses must consider security from the beginning when building applications on cloud. They are also looking to take advantage of application modernization as an opportunity to better their overall security posture. threat modeling and secure engineering were always part of traditional application development practices. These practices merit even more importance in a hybrid cloud world, as the attack surfaces and threats are bigger. Clients adopting cloud-native and exponential technologies such as containers, blockchains, and AI need protection and look to secure these technologies as part of their modernization.

There is a need to catch advanced threats and be in a position to respond to them – you need to shift left with threat management in the agile DevOps pipeline. Traditionally, threat management and responding to security incidents are extended components of Security Information and Event Management (SIEM). In the modernization context, we will need to plan for all the ways to combat the different threats during the application development phase itself.

Data security

This important aspect will include how to manage security, protection, and governance of the data on cloud. The data security domain includes controls mapped to security threat landscapes and business contexts – encryption and key management, Identity and Access Management (IAM), vulnerability management, and data activity monitoring.

Another emerging aspect to protect sensitive data in all formats – in transit, at rest, and even in use – is becoming a necessity. This will ensure data in memory is also secure. Use cases such as cryptocurrency and digital assets drive this requirement for performing computation in a hardware-based trusted execution environment. Use cases such as the mining of cryptocurrency shifting the focus from data to computational issues drive the need for tighter security for data in use.

Security for integration, coexistence, and interoperability

Integration and interoperability are key enablers for enterprises to build business processes and enable data movement across internal, partner, and supplier systems. In a hybrid multi-cloud environment, integration cuts across applications deployed on-premises, applications in the same cloud, and applications deployed on different clouds, as well as SaaS solutions. Within integration modernization trends, companies are also building the shared intelligence backbone enabled through event streams. The secure transformation of this integration landscape is an essential part of enterprise’s digital journey. This will involve securing the different integration patterns leveraged for the purpose – such as API enablement of legacy systems and event-driven architecture, as well as the coexistence and interoperability components.

Shift left security – from DevOps to DevSecOps

With a move from just a few releases a year to weekly feature releases, security can no longer be ensured manually. Security needs to be part of the DevOps pipeline and be automated. There are plenty of security tools out there from various vendors than can integrate with the pipeline. The key things to be addressed in a DevSecOps pipeline include security tools that address the following.

Securing cloud-native development and operations

Securing the DevOps pipeline involves catching security errors early in the cycle and addressing the vulnerabilities of deployable artifacts, as well as performing configuration checks. These aspects are discussed in the following sections.

Helping developers address issues in the code early

A proper Integrated Development Environment (IDE) should come with source code analysis and code coverage tools that analyze the source code to find security flaws. The usage of security testing tools for identifying potential vulnerabilities (such as OWASP) is a critical element of DevSecOps. There are several open source and vendor tools to that integrate with your pipeline that can secure your application before you deploy to production and ensure they are vulnerability free.

Information

The Open Web Application Security Project® (OWASP) is a non-profit foundation that works to improve the security of software. The OWASP Top 10 is a book or referential document outlining the 10 most critical security concerns for web application security.

Securing deployable artefacts

Artifacts such as containers and third-party libraries need to be scanned for vulnerabilities. The cloud service providers and repository engines typically provide built-in security scanning for these artifacts. In the container space, you will see a lot of open source tools (among others) available that provide security for your end-to-end CI/CD pipeline. These tools can also benchmark your security against standards and best practices such as CIS security standards.

Configuration management

Cloud configuration checks are really important when you are continuously deploying to a hybrid multi-cloud environment. Businesses don’t want to leave this down to a developer’s wisdom or chance but ensure the environment is set up correctly instead. This includes checking that all the infrastructure, services, IAM (roles and access), and other service configurations are set correctly for the application. All clouds provide tools or scripts that you can leverage in your automation to perform these configuration checks. This confirms that you have secured your storage, compute, and network as well as your service on the cloud correctly. Another opportunity to shift security left is to check for certificate expiry before discovering it through an application outage. Data vulnerability and cloud configuration checks may also be automated as part of every release.

Security Orchestration, Automation, and Response

Security Orchestration, Automation, and Response (SOAR) is a core part of automation – identifying the risks, integrating the data that needs to be monitored, detecting the key event of interest, and being able to respond to it. Enterprises have many security tools in their landscape. Threat information from these various sources is one place for analysis. This calls for leveraging open standards such as the Open Cyber Security Alliance (https://opencybersecurityalliance.org/) and building a standards-based open ecosystem where cybersecurity products can share threat information without the need for customized integrations.

Integrated security and continuous compliance

Enterprises that have to meet industry and government regulations are taking a compliance-led approach to moving their workloads to the cloud. Clients have to deal with multiple certifications and compliance as separate efforts. When you have several controls to put in place and verify, it makes sense to automate the whole process. For the same reasons, the cloud providers offer a centralized facility such as the security and compliance center where clients can centrally manage compliance with organization and regulatory guidelines. In the cloud, it is possible to predefine groups of controls as profiles and use the results as a report for audit evidence and continuous compliance. To gain trust in the cloud, it should provide all the evidence and controls to meet industry-specific compliance and security requirements, specifically in industries such as banking, healthcare, and government.

Zero-trust architecture and security models

This is one of the most significant trends related to cloud security. This model considers a different model to addressing security compared to the traditional approach. More driven by multi-cloud scenarios, this is a new approach where the traditional perimeter-based protection with firewalls is changed to context-based access. This also supports one of the other macro trends that companies are trying to address – secure and remote access for the employees to enterprise resources.

The important aspects of this is you don’t trust anything – people, processes, technology, networks, computes, or storage – until it proves that it is trustable. Some of the capabilities you need to build a security system based on zero trust are adaptive identity, context-specific and policy-enforced data security, policy-driven access control, and secured zones.

Several companies have built systems and platforms on top of the zero trust principle to establish integrity and trust levels explicitly. This is based on an organization’s risk threshold and tolerance to provide access to assets and data systems. This requirement has been amplified during COVID times when most employees are accessing corporate resources from different places. Thin virtualized apps and workspaces are likely to back in demand to meet the security needs of employees logging in through the internet to access enterprise applications.

NIST has identified this requirement to access cloud resources from anywhere, on any device, and to expect a reliable and secure experience as able to be achieved through a zero-trust security model. The NIST publication discusses the various aspects of this model focused on protecting resources (assets, services, workflows, network accounts, and so on) and discusses general deployment models and use cases. Industries such as government and defense drive confidential computing and multi-level security requirements, which intersects with the overall zero-trust approach.

In this chapter, we learned about what the cloud is and the different deployment, delivery, and consumption models of the cloud. Enterprises recognize the business value of the cloud and how a hybrid multi-cloud approach is emerging as a key ingredient in successful digital transformation journeys. There are major observable changes across how applications are built, data is managed, integrated, and operated at scale. Security is a cross-cutting concern in all these areas.

All the security components and architecture required for this digital transformation journey needs to be carefully rethought – identities, networks, apps, data, integrations, devices, and analytics for cloud-based workloads. We need consistent, repeatable methods of architecting, designing, and integrating security for hybrid cloud applications to provide context-based access to resources based on a zero-trust model. However, this is a complex topic that needs to be discussed across various areas for a hybrid multi-cloud infrastructure.

We can look at the benefits of taking a pattern approach to break down this complexity and address the problem context. By specifying right architecture building blocks for the context, we can build reusable solutions as the basis for providing effective security for digital transformation journeys.

In the next chapter, we will learn how to work with patterns better, how to use them, when, why, and what trade-offs to consider. We will cover the security architecture, domains, roles, and responsibilities that we need to know.