Data lake layers
Now that we have a broader business use case for setting up a data lake, let’s look at a use case that will help us define what the different layers of a typical data lake are and why they are required.
Use case for creating data lake layers
GreatFin has different LOBs, and within each of these LOBs, multiple personas have different tasks to perform on the data. Each persona may need specific access to different sets of data. They will all need the data to be formatted and stored in a certain way for them to do their day-to-day operations easily. For example, data engineers may need access to the raw source data so that they can profile the data and understand the quality of the data. Data scientists may need access to a standardized form of datasets so that they can do feature engineering for creating machine learning (ML) models. Data analysts may need access to business-friendly datasets so that they can derive insights from the data.
Before we get...