Understanding DataOps principles
The DataOps principles are the set of guidelines that help deliver datasets and pipelines more efficiently. According to the DataOps manifesto (https://dataopsmanifesto.org/en/), 18 different principles are recommended. Those principles fall into three main pillars that form the basis of DataOps:
- People
- Delivery
- Confidence
We can see the pillars and a summary of the principles in Figure 3.1, and we will look at each principle in detail later in this chapter:
Figure 3.1 – The pillars and principles of the DataOps framework
But for now, we will investigate the themes of each pillar:
- The People pillar focuses on the culture that DataOps is trying to instill. It looks at both the data teams delivering the datasets and the end users who will consume the dataset. The principles in this pilar also encourage a close, iterative link between the end user and the data engineer, borrowing ideas from...