Book Image

Engineering Data Mesh in Azure Cloud

By : Aniruddha Deswandikar
Book Image

Engineering Data Mesh in Azure Cloud

By: Aniruddha Deswandikar

Overview of this book

Decentralizing data and centralizing governance are practical, scalable, and modern approaches to data analytics. However, implementing a data mesh can feel like changing the engine of a moving car. Most organizations struggle to start and get caught up in the concept of data domains, spending months trying to organize domains. This is where Engineering Data Mesh in Azure Cloud can help. The book starts by assessing your existing framework before helping you architect a practical design. As you progress, you’ll focus on the Microsoft Cloud Adoption Framework for Azure and the cloud-scale analytics framework, which will help you quickly set up a landing zone for your data mesh in the cloud. The book also resolves common challenges related to the adoption and implementation of a data mesh faced by real customers. It touches on the concepts of data contracts and helps you build practical data contracts that work for your organization. The last part of the book covers some common architecture patterns used for modern analytics frameworks such as artificial intelligence (AI). By the end of this book, you’ll be able to transform existing analytics frameworks into a streamlined data mesh using Microsoft Azure, thereby navigating challenges and implementing advanced architecture patterns for modern analytics workloads.
Table of Contents (23 chapters)
Free Chapter
1
Part 1: Rolling Out the Data Mesh in the Azure Cloud
9
Part 2: Practical Challenges of Implementing a Data Mesh
16
Part 3: Popular Data Product Architectures
17
Chapter 14: Advanced Analytics Using Azure Machine Learning, Databricks, and the Lakehouse Architecture
19
Chapter 16: Event-Driven Analytics Using Azure Event Hubs, Azure Stream Analytics, and Azure Machine Learning

Part 1: Rolling Out the Data Mesh in the Azure Cloud

Part 1 starts with the theory of the data mesh architecture as described by Zhamak Dehghani in her original whitepaper (https://www.thoughtworks.com/insights/whitepapers/the-data-mesh-shift) and maps it to Microsoft Azure’s Well-Architected Framework, Cloud Adoption Framework, and cloud-scale analytics framework. Crossing this chasm is difficult for companies. This section will make it easier to understand the theory and apply it to your Microsoft Azure-based analytical systems. Whether you already have an existing central analytical system that you wish to migrate to a data mesh architecture or you are building an analytical system from the ground up, this part of the book will help you pave the way forward to adopt the data mesh architecture on Microsoft Azure.

This part has the following chapters:

  • Chapter 1, Introducing Data Meshes
  • Chapter 2, Building a Data Mesh Strategy
  • Chapter 3, Deploying a Data Mesh Using the Azure Cloud-Scale Analytics Framework
  • Chapter 4, Building a Data Mesh Governance Framework Using Microsoft Azure Services
  • Chapter 5, Security Architecture for Data Meshes
  • Chapter 6, Automating Deployment through Azure Resource Manager and Azure DevOps
  • Chapter 7, Building a Self-Service Portal for Common Data Mesh Operations