Book Image

Connecting the Data: Data Integration Techniques for Building an Operational Data Store (ODS)

By : Angelo Bobak
Book Image

Connecting the Data: Data Integration Techniques for Building an Operational Data Store (ODS)

By: Angelo Bobak

Overview of this book

When organizations change or enhance their internal structures, business data integration is a complex problem that they must resolve. This book describes the common hurdles you might face while working with data integration and shows you various ways to overcome these challenges. The book begins by explaining the foundational concepts of ODS. Once familiar with schema integration, you?ll learn how to reverse engineer each data source for creating a set of data dictionary reports. These reports will provide you with the metadata necessary to apply the schema integration process. As you progress through the chapters, you will learn how to write scripts for populating the source databases and spreadsheets, as well as how to use reports to create Extract, Transform, and Load (ETL) specifications. By the end of the book, you will have the knowledge necessary to design and build a small ODS.
Table of Contents (17 chapters)
Free Chapter
1
Section 1: Site Reliability Engineering – A Prescriptive Way to Implement DevOps
6
Section 2: Google Cloud Services to Implement DevOps via CI/CD
Appendix: Getting Ready for Professional Cloud DevOps Engineer Certification

Cloud Logging

A log is defined as a record of a status or event. Logging essentially describes what happened and provides data so that we can investigate an issue. It is critical to be able to read and parse logs across a distributed infrastructure involving multiple services and products. Cloud Logging is a GCP service that allows you to store, search, analyze, monitor, and alert others about logging data and events from Google Cloud and AWS, third-party applications, or custom application code. The information in the log entry is structured as a payload. This payload consists of information related to a timestamp, a resource that the log entry applies to, and a log name. The maximum size of a log entry is 256 KB. Each log entry indicates the source of the resource, labels, namespaces, and status codes. Cloud Logging is also the source of input for other Cloud Operations services, such as Cloud Debug and Cloud Error Reporting.

The following are the key features of Cloud Logging...