-
Book Overview & Buying
-
Table Of Contents
Data Observability for Data Engineering
By :
As we have already learned, sometimes, data quality is still not considered a determining factor during the life cycle of data, or the design and development of new data pipelines.
When a data team has to design and build a new data pipeline, there are many aspects to focus on:
But, often, what is missing is the adoption of data quality and observability by design – in other words, defining the specifications regarding what and how to monitor from the beginning of the design of the new data pipeline. These expectations are often not well defined and agreed upon between producers and consumers and even though this might seem unusual, data teams often only develop this sense of need for data quality and observability over time –...