Book Image

Scalable Data Analytics with Azure Data Explorer

By : Jason Myerscough
Book Image

Scalable Data Analytics with Azure Data Explorer

By: Jason Myerscough

Overview of this book

Azure Data Explorer (ADX) enables developers and data scientists to make data-driven business decisions. This book will help you rapidly explore and query your data at scale and secure your ADX clusters. The book begins by introducing you to ADX, its architecture, core features, and benefits. You'll learn how to securely deploy ADX instances and navigate through the ADX Web UI, cover data ingestion, and discover how to query and visualize your data using the powerful Kusto Query Language (KQL). Next, you'll get to grips with KQL operators and functions to efficiently query and explore your data, as well as perform time series analysis and search for anomalies and trends in your data. As you progress through the chapters, you'll explore advanced ADX topics, including deploying your ADX instances using Infrastructure as Code (IaC). The book also shows you how to manage your cluster performance and monthly ADX costs by handling cluster scaling and data retention periods. Finally, you'll understand how to secure your ADX environment by restricting access with best practices for improving your KQL query performance. By the end of this Azure book, you'll be able to securely deploy your own ADX instance, ingest data from multiple sources, rapidly query your data, and produce reports with KQL and Power BI.
Table of Contents (18 chapters)
1
Section 1: Introduction to Azure Data Explorer
5
Section 2: Querying and Visualizing Your Data
11
Section 3: Advanced Azure Data Explorer Topics

What this book covers

Chapter 1, Introduction to Azure Data Explorer, covers what ADX is, the core features of ADX, and where ADX fits in Microsoft's suite of data services. The chapter then discusses some of the different use cases of when to use ADX and demonstrates how to execute your first KQL query.

Chapter 2, Building Your Azure Data Explorer Environment, explains how to quickly deploy and configure ADX clusters and databases using the Azure portal, PowerShell, and Azure ARM templates. By the end of this chapter, you will be ready to start ingesting and analyzing your data.

Chapter 3, Exploring Azure Data Explorer UI, presents the ADX UI to you. You will spend the majority of your time using the ADX UI to query and analyze your data. By the end of this chapter, you will be familiar with the windows and panes in the ADX Web UI.

Chapter 4, Ingesting Data in Azure Data Explorer, discusses the concept of data ingestion and demonstrates how to ingest data from multiple data sources such as Blob storage and Azure Event Hubs, how to create new table schemas, and explains how data maps to those tables. At the end of this chapter, you will understand how ADX ingests data and how to configure the data ingestion.

Chapter 5, Introducing the Kusto Query Language, introduces you to KQL and demonstrates how to query data. The chapter begins by introducing the language, explains the basics of KQL such as searching, filtering, aggregating, and joining tables. By the end of the chapter, you will know enough KQL to comfortably query data.

Chapter 6, Introducing Time Series Analysis, introduces you to ADX's time series features, beginning by defining what time series analysis is, and then demonstrating how to query your time series data using the make-series operator. Finally, we discuss some of the most important and useful time series functions provided by ADX.

Chapter 7, Identifying Patterns, Anomalies, and Trends in Your Data, builds on the previous chapter by discussing how to detect anomalies and trends in your data. The chapter first begins by introducing some of the anomaly functions available within ADX and then covers some of the machine learning capabilities of ADX.

Chapter 8, Data Visualization with Azure Data Explorer and Power BI, explains and demonstrates how to integrate ADX with Power BI. Power BI is a powerful reporting tool used to share rich graphs and reports. By the end of the chapter, you will know how to integrate ADX with Power BI and how to create reports in Power BI powered by ADX datasets.

Chapter 9, Monitoring and Troubleshooting Azure Data Explorer, teaches you how to monitor your ADX clusters using Azure Monitor and ADX Insights. The chapter teaches you how to configure alerts using KQL and action groups and explains how to troubleshoot issues by enabling the ADX diagnostics and examining those logs using Log Analytics. In the troubleshooting section, we will demonstrate how to troubleshoot and resolve a data ingestion problem.

Chapter 10, Azure Data Explorer Security, discusses how to secure your ADX instances using both identity management and virtual networks with subnet delegation. We begin by explaining why security is important on the public cloud and then we discuss identity management at the management and data plane. Next, we will introduce securing ADX instances using virtual networks and subnet delegation and demonstrate how to filter network traffic using network security groups (NSGs).

Chapter 11, Performance Tuning in Azure Data Explorer, begins by explaining why performance matters, and then discusses the KQL best practices and revisits the ADX architecture to explain how time filtering can provide performance improvements. You will also learn how to monitor the performance of your clusters, queries, and external applications.

Chapter 12, Cost Management in Azure Data Explorer, discusses how to plan and manage production deployments. The chapter first discusses how to manage your clusters and what requirements you should take into consideration when planning your deployment, and finally discusses how to estimate your Azure costs.