Book Image

Scalable Data Analytics with Azure Data Explorer

By : Jason Myerscough
Book Image

Scalable Data Analytics with Azure Data Explorer

By: Jason Myerscough

Overview of this book

Azure Data Explorer (ADX) enables developers and data scientists to make data-driven business decisions. This book will help you rapidly explore and query your data at scale and secure your ADX clusters. The book begins by introducing you to ADX, its architecture, core features, and benefits. You'll learn how to securely deploy ADX instances and navigate through the ADX Web UI, cover data ingestion, and discover how to query and visualize your data using the powerful Kusto Query Language (KQL). Next, you'll get to grips with KQL operators and functions to efficiently query and explore your data, as well as perform time series analysis and search for anomalies and trends in your data. As you progress through the chapters, you'll explore advanced ADX topics, including deploying your ADX instances using Infrastructure as Code (IaC). The book also shows you how to manage your cluster performance and monthly ADX costs by handling cluster scaling and data retention periods. Finally, you'll understand how to secure your ADX environment by restricting access with best practices for improving your KQL query performance. By the end of this Azure book, you'll be able to securely deploy your own ADX instance, ingest data from multiple sources, rapidly query your data, and produce reports with KQL and Power BI.

Related Content you might be interested in

Current Title:

Small book image
Scalable Data Analytics with Azure Data Explorer
Jason Myerscough
Mar, 2022 | 364 pages
Small book image
Learn Azure Synapse Data Explorer
Pericles Peri Rocha
Feb, 2023 | 346 pages
Small book image
Azure Data Factory Cookbook
Dmitry Foshin | Dmitry Anoshin | Xenia Ireton | Tonya Chernyshova
Feb, 2024 | 532 pages
Table of Contents (18 chapters)
Preface
Preface
Who this book is for
What this book covers
To get the most out of this book
Download the example code files
Code in Action
Download the color images
Conventions used
Get in touch
Share Your Thoughts
1
Section 1: Introduction to Azure Data Explorer
Section 1: Introduction to Azure Data Explorer
Free Chapter
2
Chapter 1: Introducing Azure Data Explorer
Chapter 1: Introducing Azure Data Explorer
Technical requirements
Introducing the data analytics pipeline
What is Azure Data Explorer?
Azure Data Explorer use cases
Running your first query
Summary
3
Chapter 2: Building Your Azure Data Explorer Environment
Chapter 2: Building Your Azure Data Explorer Environment
Technical requirements
Creating an Azure subscription
Introducing Azure Cloud Shell
Creating and configuring ADX instances in the Azure portal
Introducing Infrastructure as Code
Creating and configuring ADX instances with PowerShell
Creating ADX clusters with ARM templates
Summary
Questions
4
Chapter 3: Exploring the Azure Data Explorer UI
Chapter 3: Exploring the Azure Data Explorer UI
Technical requirements
Ingesting the StormEvents sample dataset
Querying data in the Azure portal
Exploring the ADX Web UI
Summary
5
Section 2: Querying and Visualizing Your Data
Section 2: Querying and Visualizing Your Data
6
Chapter 4: Ingesting Data in Azure Data Explorer
Chapter 4: Ingesting Data in Azure Data Explorer
Technical requirements
Understanding data ingestion
Introducing schema mapping
Ingesting data using one-click ingestion
Ingesting data using KQL management commands
Ingesting data from Blob storage using Azure Event Grid
Summary
Questions
7
Chapter 5: Introducing the Kusto Query Language
Chapter 5: Introducing the Kusto Query Language
Technical requirements
What is KQL?
Introducing the basics of KQL
Introducing KQL's scalar operators
Joining tables in KQL
Introducing KQL's management commands
Summary
Questions
8
Chapter 6: Introducing Time Series Analysis
Chapter 6: Introducing Time Series Analysis
Technical requirements
What is time series analysis?
Creating a time series with KQL
Calculating statistics for time series data
Summary
Questions
9
Chapter 7: Identifying Patterns, Anomalies, and Trends in your Data
Chapter 7: Identifying Patterns, Anomalies, and Trends in your Data
Technical requirements
Calculating moving averages with KQL
Trend analysis with KQL
Anomaly detection and forecasting with KQL
Summary
Questions
10
Chapter 8: Data Visualization with Azure Data Explorer and Power BI
Chapter 8: Data Visualization with Azure Data Explorer and Power BI
Technical requirements
Introducing data visualization
Creating dashboards with Azure Data Explorer
Connecting Power BI to Azure Data Explorer
Summary
Questions
11
Section 3: Advanced Azure Data Explorer Topics
Section 3: Advanced Azure Data Explorer Topics
12
Chapter 9: Monitoring and Troubleshooting Azure Data Explorer
Chapter 9: Monitoring and Troubleshooting Azure Data Explorer
Technical requirements
Introducing monitoring and troubleshooting
Monitoring ADX
Troubleshooting ADX
Summary
Questions
13
Chapter 10: Azure Data Explorer Security
Chapter 10: Azure Data Explorer Security
Technical requirements
Introducing identity management
Introducing virtual networking and subnet delegation
Filtering traffic with NSGs
Summary
Questions
14
Chapter 11: Performance Tuning in Azure Data Explorer
Chapter 11: Performance Tuning in Azure Data Explorer
Technical requirements
Introducing performance tuning
Introducing workload groups
Introducing policy management
Monitoring queries
KQL best practices
Summary
Questions
15
Chapter 12: Cost Management in Azure Data Explorer
Chapter 12: Cost Management in Azure Data Explorer
Technical requirements
Scaling and cost management
Selecting the correct ADX cluster SKU
Introducing Azure Advisor
Introducing Cost Management + Billing
Summary
16
Chapter 13: Assessment
Chapter 1
Chapter 2
Chapter 4
Chapter 5
Chapter 6
Chapter 7
Chapter 8
Chapter 9
Chapter 10
Chapter 11
Why subscribe?
17
Other Books You May Enjoy
Other Books You May Enjoy
Packt is searching for authors like you
Share Your Thoughts

Chapter 1: Introducing Azure Data Explorer

Welcome to Scalable Data Analytics with Azure Data Explorer! More than 90% of today's data is digital and most of that data is considered unstructured, such as text messages and other forms of free text. So how can we analyze all our data? The answer is data analytics and Azure Data Explorer (ADX). Data analytics is a complex topic and Microsoft Azure provides a comprehensive selection of data analytics services, which can seem overwhelming when you are first starting your journey into data analytics.

In this chapter, we begin by introducing the data analytics pipeline and learning about each of the steps in the pipeline. These steps are required for taking raw data and producing reports and visuals as a result of your analysis, which will help you understand the workflow used by ADX.

Next, we will introduce some of the popular Azure data services and understand where they fit in the data analytics pipeline. Some of these services, such as Azure Event Hubs, will be used in later chapters when we learn about data ingestion.

We will also learn what ADX is, the features that make it a powerful data exploration platform, the architecture, and key components of ADX, such as the engine cluster, and understand some of the use cases for ADX, for example, in IoT monitoring, telemetry, and log analysis. Finally, we will get our feet wet and dive right into running your first Kusto Query Language (KQL) query using the Data Explorer UI.

In this chapter, we are going to cover the following main topics:

  • Introducing the data analytics pipeline
  • What is Azure Data Explorer?
  • Azure Data Explorer use cases
  • Running your first query
Previous Section
End of Section 1
Next Section