Book Image

Connecting the Data: Data Integration Techniques for Building an Operational Data Store (ODS)

By : Angelo Bobak
Book Image

Connecting the Data: Data Integration Techniques for Building an Operational Data Store (ODS)

By: Angelo Bobak

Overview of this book

When organizations change or enhance their internal structures, business data integration is a complex problem that they must resolve. This book describes the common hurdles you might face while working with data integration and shows you various ways to overcome these challenges. The book begins by explaining the foundational concepts of ODS. Once familiar with schema integration, you?ll learn how to reverse engineer each data source for creating a set of data dictionary reports. These reports will provide you with the metadata necessary to apply the schema integration process. As you progress through the chapters, you will learn how to write scripts for populating the source databases and spreadsheets, as well as how to use reports to create Extract, Transform, and Load (ETL) specifications. By the end of the book, you will have the knowledge necessary to design and build a small ODS.
Table of Contents (17 chapters)
Free Chapter
1
Section 1: Site Reliability Engineering – A Prescriptive Way to Implement DevOps
6
Section 2: Google Cloud Services to Implement DevOps via CI/CD
Appendix: Getting Ready for Professional Cloud DevOps Engineer Certification

Summary

In this chapter, we discussed the suite of tools that are part of Cloud Operations. Cloud Operations is critical for forming the feedback loop of the CI/CD process and is fundamental to establishing observability on GCP. Observability is key to ensuring that an SRE's technical practices – specifically, SLIs, SLOs, SLAs, and Error Budgets – are not violated. This is achieved by gathering logs, metrics, and traces from multiple sources and by visualizing this information on dashboards. This information is used to establish performance and reliability indicators. These indicators can then be tracked with configurable alerts. These alerts trigger when there is a potential violation, and the alerts will be notified on the configurable notification channels. Cloud Operations also offers services that allow us to debug the application, without slowing down, and capture trace information. The end goal is to ensure that the service is reliable. We concluded this chapter...