Book Image

Connecting the Data: Data Integration Techniques for Building an Operational Data Store (ODS)

By : Angelo Bobak
Book Image

Connecting the Data: Data Integration Techniques for Building an Operational Data Store (ODS)

By: Angelo Bobak

Overview of this book

When organizations change or enhance their internal structures, business data integration is a complex problem that they must resolve. This book describes the common hurdles you might face while working with data integration and shows you various ways to overcome these challenges. The book begins by explaining the foundational concepts of ODS. Once familiar with schema integration, you?ll learn how to reverse engineer each data source for creating a set of data dictionary reports. These reports will provide you with the metadata necessary to apply the schema integration process. As you progress through the chapters, you will learn how to write scripts for populating the source databases and spreadsheets, as well as how to use reports to create Extract, Transform, and Load (ETL) specifications. By the end of the book, you will have the knowledge necessary to design and build a small ODS.
Table of Contents (17 chapters)
Free Chapter
1
Section 1: Site Reliability Engineering – A Prescriptive Way to Implement DevOps
6
Section 2: Google Cloud Services to Implement DevOps via CI/CD
Appendix: Getting Ready for Professional Cloud DevOps Engineer Certification

Cloud Tasks

Cloud Tasks is a fully managed service from Google Cloud that allows you to separate out pieces of work that could be performed independently and asynchronously outside of a user or a service-to-service request. An independent piece of work is referred to as a task. Cloud Tasks is essentially used when an application accepts inputs from users and needs to initiate background tasks accordingly to perform automated asynchronous execution.

The following is a summary of the critical features of Cloud Tasks:

  • Cloud Tasks is aimed at explicit invocation, where the publisher retains full control of execution.
  • Cloud Tasks is most appropriate where the task producer can have control over the execution.

The core difference between Cloud Tasks and Pub/Sub is the notion of explicit versus implicit invocation. As mentioned, Cloud Tasks is aimed at explicit invocation. In contrast, Pub/Sub supports implicit invocation, where a publisher implicitly causes the subscriber...