Data lineage
Data lineage is a very important first step in defining the governance around the data you are collecting. The data-collection System is also a very important piece in the overall architecture from the governance perspective. It is the place where the data you will be using in your application to support business functionality enters. Thus, it is also the place where we start defining our Data Governance and Handling Strategies. With General Data Protection Regulation (GDPR) defining rules around data-handling and consequences of non-compliance being very high, the governance of data has become more important than ever. Companies are required to prove that they are handling the data in accordance with GDPR, otherwise they face huge financial implications. This is where capturing the lineage of the data becomes important.
When you add a new data source into the system or you update the existing data source, you need to maintain some sort of mapping or relationship within the...