Cloud-native automation with Google Cloud Build

This brings us to Cloud Build, a Google Cloud offering that is a serverless platform. This means that teams are not responsible for maintaining, scaling workers, or assuring the availability of the platform. Cloud Build is a managed service as well because Google Cloud is responsible for ensuring that the service is available to the customer. This allows an organization to focus purely on their application code and build pipeline configuration. Google Cloud is responsible for scaling the platform as needed and the billing is priced (https://cloud.google.com/build/pricing) per build minute.

Cloud Build takes advantage of the inherent security of Google Cloud, where the API is only accessible by individuals and automation systems that are granted access. This will be a common theme throughout the book because Cloud Build is a serverless platform. We are able to take advantage of what Google Cloud has to offer in areas such as security, compute resources, integrations, and API- and language-specific client SDKs to access the platform.

As noted in previous sections, workers handle the steps (or jobs) in a build pipeline and Cloud Build is responsible for making sure there are workers available to complete the pipeline from end to end. Cloud Build offers workers in different flavors, noted as a default pool and a private pool. In a default pool, the workers are hosted by Google and not visible to the end user. While the build is running, it would have access to the public internet, but not privately hosted resources. Depending on a build that you need, this may be all that is necessary. However, if during a build you need access to resources on a private network, such as dependencies, localized datasets, higher concurrency limits, more machine type configurations, or integration testing with a private environment, the option would be a private pool. Private pools allow for connectivity to private networks using Google Cloud Platform (GCP) Virtual Private Cloud (VPC) peering capabilities. The following is a table comparing capabilities of default and private pools:

Figure 1.4 – Feature comparison between default and private pools

The data in the figure is up to date as of 2022-06-16 ( https://cloud.google.com/build/docs/private-pools/private-pools-overview).

The steps in a Cloud Build configuration typically remain the same regardless of the type of pool selected: default or private.

In Cloud Build, a single worker is responsible for handling all the steps in a pipeline. The worker and machine type (compute, memory, and storage) can be configured, providing flexibility for each build process, depending on the criticality. Exposing the build process to a machine with more resources may allow for faster builds, where machine resources have been identified as a bottleneck for slower builds.

This is a significant advantage of a serverless platform in Cloud Build, where the options are defined as configuration options. It is the responsibility of Google Cloud to provide the resources based on what is specified in the valid configuration. The flexibility of the cloud is available to the team, by specifying different machine types for the workers based on the pipeline requirements.

If you recall the example of Jenkins (in the Who needs to manage all of this? section), it provides plugins to add integrations or functionality to complete a pipeline. In the case of Cloud Build, each individual step consists of a container image known as a cloud builder. This provides a significant amount of flexibility as it allows for using Google Cloud-provided builders, community-developed builders, or custom builders, which can be container images created in-house. These custom builders can be any container image that executes a script or binary, which can then be incorporated as a build step.

Organizations may place different restrictions on the residency of code; for example, it may not reside on a developer’s workstation, it may only reside on organizationally maintained SCM systems. Cloud Build can integrate with privately hosted/non-GCP solutions, such as GitHub Enterprise. By using private pools, the worker of the build can interact with resources within a private network. This is very important as it allows Cloud Build to support many different scenarios that an organization may have.

Earlier in this chapter, we discussed the value of automation, which includes the automation of build steps, but also how a build itself can be triggered. Triggering a build can be based on a cron schedule using Cloud Scheduler (https://cloud.google.com/build/docs/automating-builds/create-scheduled-triggers), async messages through Cloud Pub/Sub (https://cloud.google.com/build/docs/automating-builds/create-pubsub-triggers), and webhooks, but is usually driven by a commit of code to an SCM system. Webhooks open the door to many other options; for instance, a build can also be invoked with an API or CLI call through an existing tool. This makes the integration and automation with existing components possible.

GCP service integrations

Integration with existing components is critical, but another advantage of Cloud Build is its native integration with Google Cloud services:

• Artifact Registry – store built artifacts (that is, language-specific, container images): https://cloud.google.com/artifact-registry/docs/configure-cloud-build.
• Secret Manager – natively manage sensitive information: https://cloud.google.com/build/docs/securing-builds/use-secrets.
• VPC Service Controls – mechanism to protect against data exfiltration, for instance, where a build can be triggered from or policies that require only private pools be used: https://cloud.google.com/build/docs/private-pools/using-vpc-service-controls.
• Binary Authorization (preview) – sign image attestations at build time with a provided attestor: https://cloud.google.com/build/docs/securing-builds/use-provenance-and-binary-authorization.

If you recall earlier, it was noted that a Cloud Build worker is responsible for executing the steps designated in the configured pipeline. The identity of this worker is established by a GCP resource known as a service account. Cloud Build has a default service account that it uses, or a user-specified service account may also be specified.

This book focuses on the build aspect of Cloud Build and a built resource does not always need to be deployed when using Cloud Build. However, Cloud Build also provides integrations with instructions and images to deploy built container images to a GCP runtime platform, such as the following:

• Google Kubernetes Engine
• Anthos-enabled clusters
• Cloud Run
• App Engine
• Cloud Functions
• Firebase

Organizations are able to use Cloud Build for both the code build and IaC automation capabilities to provide more autonomy to their teams. One example is noted here: https://cloud.google.com/customers/loveholidays.

As a serverless platform, visibility into what is happening with the build is important. Audit logs and build logs are stored by default in GCP’s Cloud Logging service. Audit logs cover API calls to the Cloud Build service, such as creation and cancelation. The actual build logs are stored by default in both Cloud Logging and a Google-created Cloud Storage bucket. The logs from Cloud Build are configurable to destinations of your choice within GCP (that is, only Google Cloud Storage, only Cloud Logging, or log buckets in other projects).

Cloud Build, being a serverless platform, also offers an SLA. As with GCP services, Cloud Build has a monthly uptime percentage of 99.95% (at the time of this writing this book – https://cloud.google.com/build/sla). This is noted as the service-level objective (SLO). Please refer to the previous link for additional details related to the SLA. For services that are provided by a cloud provider, this is an important measurement factoring in the reliability of the service. As noted in an earlier section, if a team were responsible for managing build infrastructure, they would also need to offer a reliability metric. The built solution would typically have to be designed, implemented, and tested to meet the offered metric. In GCP and its applicable services where SLAs are offered, GCP has this responsibility as a service provider.

Organizations have come a long way from having to manage their own infrastructure and their own pipeline software. Service providers provide a way for the infrastructure to be managed in their data center to offload the hardware, the OS, and sometimes even the management of the build pipeline platform itself. The next iteration is serverless, which is what we’ve been talking about, where it’s just an API that is exposed for clients to use. Define your desired configuration and execute the API in your desired mechanism and language.

Cloud Build is a serverless build pipeline platform that allows organizations to focus on the code at hand. Google Cloud takes care of ensuring the platform is available when needed and that there is sufficient compute for the workers. Each step, represented as a container image, provides the utmost flexibility in handling pipeline needs.