Book Image

Scalable Data Analytics with Azure Data Explorer

By : Jason Myerscough
Book Image

Scalable Data Analytics with Azure Data Explorer

By: Jason Myerscough

Overview of this book

Azure Data Explorer (ADX) enables developers and data scientists to make data-driven business decisions. This book will help you rapidly explore and query your data at scale and secure your ADX clusters. The book begins by introducing you to ADX, its architecture, core features, and benefits. You'll learn how to securely deploy ADX instances and navigate through the ADX Web UI, cover data ingestion, and discover how to query and visualize your data using the powerful Kusto Query Language (KQL). Next, you'll get to grips with KQL operators and functions to efficiently query and explore your data, as well as perform time series analysis and search for anomalies and trends in your data. As you progress through the chapters, you'll explore advanced ADX topics, including deploying your ADX instances using Infrastructure as Code (IaC). The book also shows you how to manage your cluster performance and monthly ADX costs by handling cluster scaling and data retention periods. Finally, you'll understand how to secure your ADX environment by restricting access with best practices for improving your KQL query performance. By the end of this Azure book, you'll be able to securely deploy your own ADX instance, ingest data from multiple sources, rapidly query your data, and produce reports with KQL and Power BI.
Table of Contents (18 chapters)
1
Section 1: Introduction to Azure Data Explorer
5
Section 2: Querying and Visualizing Your Data
11
Section 3: Advanced Azure Data Explorer Topics

Ingesting data using one-click ingestion

In this section, we are going to learn how to ingest data using the Data Explorer Web UI and from an Azure Storage account using the one-click ingestion method.

Note

If you have not already cloned the Git repository, please do so now, so you can follow the example. The repository can be found here: https://github.com/PacktPublishing/Scalable-Data-Analytics-with-Azure-Data-Explorer.git.

Data ingestion is a three-step process:

  1. Ingestion Preparation: During the preparation phase, the table and mapping schemas are created.
  2. Ingestion: The file is then pulled from the queue, which is temporarily stored on an internal storage account, https://9qwkstrldmyerscoughadx01.blob.core.windows.net/20210614-ingestdata-e5c334ee145d4b43a3a2d3a96fbac1df-0/1623671437639_season-1819_csv.csv, and then ingested.
  3. Data Preview: Once the data has been ingested, it can be previewed and is ready for you to begin querying.

The following section...