Data mesh on AWS
To translate the concepts of data mesh to a data platform built using AWS services, we need to look at how the data is ingested, proceeded, and shared for consumption. The core purpose-built AWS analytics services remain the same, each performing specific tasks in the data platform. However, instead of placing all such services inside a single AWS account, they are all spread into different AWS accounts, owned and managed by different teams or business units. These accounts are constantly producing and/or consuming data, with the eventual goal of deriving value for the whole organization.
All the analytics services and architectures we’ve discussed in this book remain the same – it’s just the design philosophy around data production, data sharing, and data governance all become distributed and completely decoupled in nature. Instead of point-to-point data sharing across AWS accounts using bucket and IAM policies, a completely different mechanism...