Sign In Start Free Trial
Account

Add to playlist

Create a Playlist

Modal Close icon
You need to login to use this feature.
  • Book Overview & Buying Data Observability for Data Engineering
  • Table Of Contents Toc
Data Observability for Data Engineering

Data Observability for Data Engineering

By : Michele Pinto, Sammy El Khammal
5 (2)
close
close
Data Observability for Data Engineering

Data Observability for Data Engineering

5 (2)
By: Michele Pinto, Sammy El Khammal

Overview of this book

In the age of information, strategic management of data is critical to organizational success. The constant challenge lies in maintaining data accuracy and preventing data pipelines from breaking. Data Observability for Data Engineering is your definitive guide to implementing data observability successfully in your organization. This book unveils the power of data observability, a fusion of techniques and methods that allow you to monitor and validate the health of your data. You’ll see how it builds on data quality monitoring and understand its significance from the data engineering perspective. Once you're familiar with the techniques and elements of data observability, you'll get hands-on with a practical Python project to reinforce what you've learned. Toward the end of the book, you’ll apply your expertise to explore diverse use cases and experiment with projects to seamlessly implement data observability in your organization. Equipped with the mastery of data observability intricacies, you’ll be able to make your organization future-ready and resilient and never worry about the quality of your data pipelines again.
Table of Contents (17 chapters)
close
close
1
Part 1: Introduction to Data Observability
4
Part 2: Implementing Data Observability
8
Part 3: How to adopt Data Observability in your organization
12
Part 4: Appendix

Data Observability Techniques

As we have already learned, sometimes, data quality is still not considered a determining factor during the life cycle of data, or the design and development of new data pipelines.

When a data team has to design and build a new data pipeline, there are many aspects to focus on:

  • Data sources – input and output
  • Scalability, reliability, and performance
  • Total Cost of Ownership (TCO)
  • Security
  • Operation and maintainability
  • Compliance with data regulations

But, often, what is missing is the adoption of data quality and observability by design – in other words, defining the specifications regarding what and how to monitor from the beginning of the design of the new data pipeline. These expectations are often not well defined and agreed upon between producers and consumers and even though this might seem unusual, data teams often only develop this sense of need for data quality and observability over time –...

CONTINUE READING
83
Tech Concepts
36
Programming languages
73
Tech Tools
Icon Unlimited access to the largest independent learning library in tech of over 8,000 expert-authored tech books and videos.
Icon Innovative learning tools, including AI book assistants, code context explainers, and text-to-speech.
Icon 50+ new titles added per month and exclusive early access to books as they are being written.
Data Observability for Data Engineering
notes
bookmark Notes and Bookmarks search Search in title playlist Add to playlist download Download options font-size Font size

Change the font size

margin-width Margin width

Change margin width

day-mode Day/Sepia/Night Modes

Change background colour

Close icon Search
Country selected

Close icon Your notes and bookmarks

Confirmation

Modal Close icon
claim successful

Buy this book with your credits?

Modal Close icon
Are you sure you want to buy this book with one of your credits?
Close
YES, BUY

Submit Your Feedback

Modal Close icon
Modal Close icon
Modal Close icon