Book Image

Connecting the Data: Data Integration Techniques for Building an Operational Data Store (ODS)

By : Angelo Bobak
Book Image

Connecting the Data: Data Integration Techniques for Building an Operational Data Store (ODS)

By: Angelo Bobak

Overview of this book

When organizations change or enhance their internal structures, business data integration is a complex problem that they must resolve. This book describes the common hurdles you might face while working with data integration and shows you various ways to overcome these challenges. The book begins by explaining the foundational concepts of ODS. Once familiar with schema integration, you?ll learn how to reverse engineer each data source for creating a set of data dictionary reports. These reports will provide you with the metadata necessary to apply the schema integration process. As you progress through the chapters, you will learn how to write scripts for populating the source databases and spreadsheets, as well as how to use reports to create Extract, Transform, and Load (ETL) specifications. By the end of the book, you will have the knowledge necessary to design and build a small ODS.

Related Content you might be interested in

Current Title:

Small book image
Connecting the Data: Data Integration Techniques for Building an Operational Data Store (ODS)
Angelo Bobak
Jul, 2014 | 250 pages
No titles found
Table of Contents (17 chapters)
Preface
Preface
Who this book is for
What this book covers
To get the most out of this book
Download the example code files
Download the color images
Conventions used
Get in touch
Reviews
Free Chapter
1
Section 1: Site Reliability Engineering – A Prescriptive Way to Implement DevOps
Section 1: Site Reliability Engineering – A Prescriptive Way to Implement DevOps
2
Chapter 1: DevOps, SRE, and Google Cloud Services for CI/CD
Chapter 1: DevOps, SRE, and Google Cloud Services for CI/CD
Understanding DevOps, its evolution, and life cycle
SRE's evolution; technical and cultural practices
Cloud-native approach to implementing DevOps using Google Cloud
Summary
Points to remember
Further reading
Practice test
Answers
3
Chapter 2: SRE Technical Practices – Deep Dive
Chapter 2: SRE Technical Practices – Deep Dive
Defining SLAs
Defining reliability expectations via SLOs
Exploring SLIs
Understanding error budgets
Eliminating toil through automation
Illustrating the impact of SLAs, SLOs, and error budgets relative to SLI
Summary
Points to remember
Further reading
Practice test
Answers
4
Chapter 3: Understanding Monitoring and Alerting to Target Reliability
Chapter 3: Understanding Monitoring and Alerting to Target Reliability
Understanding monitoring
Alerting
Time series
Summary
Points to remember
Further reading
Practice test
Answers
5
Chapter 4: Building SRE Teams and Applying Cultural Practices
Chapter 4: Building SRE Teams and Applying Cultural Practices
Building SRE teams
Incident management
Being on call
Psychological safety
Sharing vision and knowledge and fostering collaboration
Summary
Points to remember
Further reading
Practice test
Answers
6
Section 2: Google Cloud Services to Implement DevOps via CI/CD
Section 2: Google Cloud Services to Implement DevOps via CI/CD
7
Chapter 5: Managing Source Code Using Cloud Source Repositories
Chapter 5: Managing Source Code Using Cloud Source Repositories
Technical requirements
Introducing the key features
One-way sync from GitHub/Bitbucket to CSR
Common operations in CSR
Hands-on lab – integrating with Cloud Functions
Summary
Further reading
Practice test
Answers
8
Chapter 6: Building Code Using Cloud Build, and Pushing to Container Registry
Chapter 6: Building Code Using Cloud Build, and Pushing to Container Registry
Technical requirements
Key terminology (prerequisites)
Understanding the need for automation
Building and creating container images – Cloud Build
Managing build artifacts – Container Registry
Hands-on lab – building, creating, pushing, and deploying a container to Cloud Run using Cloud Build triggers
Summary
Points to remember
Further reading
Practice test
Answers
9
Chapter 7: Understanding Kubernetes Essentials to Deploy Containerized Applications
Chapter 7: Understanding Kubernetes Essentials to Deploy Containerized Applications
Technical requirements
Kubernetes – a quick introduction
Kubernetes cluster anatomy
Kubernetes objects
Scheduling and interacting with Pods
Kubernetes deployment strategies
Summary
Points to remember
Further reading
Practice test
Answers
10
Chapter 8: Understanding GKE Essentials to Deploy Containerized Applications
Chapter 8: Understanding GKE Essentials to Deploy Containerized Applications
Technical requirements
Google Kubernetes Engine (GKE) – introduction
GKE – core features
GKE Autopilot – hands-on lab
Summary
Points to remember
Further reading
Practice test
Answers
11
Chapter 9: Securing the Cluster Using GKE Security Constructs
Chapter 9: Securing the Cluster Using GKE Security Constructs
Technical requirements
Essential security patterns in Kubernetes
Hardening cluster security in GKE
Points to remember
Further reading
Practice test
Answers
12
Chapter 10: Exploring GCP Cloud Operations
Chapter 10: Exploring GCP Cloud Operations
Cloud Monitoring
Cloud Logging
Cloud Debugger
Cloud Trace
Cloud Profiler
Binding SRE and Cloud Operations
Summary
Points to remember
Further reading
Practice test
Answers
13
Mock Exam 1
Total Number of Questions: 50
Answers
14
Mock Exam 2
Total Number of Questions: 50
Answers
Why subscribe?
15
Other Books You May Enjoy
Other Books You May Enjoy
Packt is searching for authors like you
Leave a review - let other readers know what you think
Appendix: Getting Ready for Professional Cloud DevOps Engineer Certification
Appendix: Getting Ready for Professional Cloud DevOps Engineer Certification
Cloud Deployment Manager
Cloud Tasks
Spinnaker

Building and creating container images – Cloud Build

Cloud Build is a service to build and create artifacts based on the commits made to source code repositories. The artifacts produced by Cloud Build can either be container or non-container artifacts. Cloud Build can integrate with GCP's CSR as well as popular external repositories such as GitHub and Bitbucket. Key features of Cloud Build include the following:

  • Serverless platform: Cloud Build removes the need to pre-provision servers or pay in advance for computing power or storage required to build the code and produce artifacts. Based on the number of commits being made in parallel, scaling up or scaling down is an inherent process and doesn't require manual intervention.
  • Access to builder images: Cloud Build provides cloud builders, which are pre-baked ready-to-use container images with support for multiple common languages and tools installed. For example, Docker Cloud Builders run the Docker tool...
Previous Section
End of Section 5
Next Section