Book Image

Learn Azure Synapse Data Explorer

By : Pericles (Peri) Rocha

Book Image

Learn Azure Synapse Data Explorer

By: Pericles (Peri) Rocha

Overview of this book

Large volumes of data are generated daily from applications, websites, IoT devices, and other free-text, semi-structured data sources. Azure Synapse Data Explorer helps you collect, store, and analyze such data, and work with other analytical engines, such as Apache Spark, to develop advanced data science projects and maximize the value you extract from data. This book offers a comprehensive view of Azure Synapse Data Explorer, exploring not only the core scenarios of Data Explorer but also how it integrates within Azure Synapse. From data ingestion to data visualization and advanced analytics, you’ll learn to take an end-to-end approach to maximize the value of unstructured data and drive powerful insights using data science capabilities. With real-world usage scenarios, you’ll discover how to identify key projects where Azure Synapse Data Explorer can help you achieve your business goals. Throughout the chapters, you'll also find out how to manage big data as part of a software as a service (SaaS) platform, as well as tune, secure, and serve data to end users. By the end of this book, you’ll have mastered the big data life cycle and you'll be able to implement advanced analytical scenarios from raw telemetry and log data.

Preface

Who this book is for

What this book covers

To get the most out of this book

Download the example code files

Download the color images

Conventions used

Share Your Thoughts

Download a free PDF copy of this book

Part 1 Introduction to Azure Synapse Data Explorer

Part 1 Introduction to Azure Synapse Data Explorer

Free Chapter

Chapter 1: Introducing Azure Synapse Data Explorer

Chapter 1: Introducing Azure Synapse Data Explorer

Technical requirements

Understanding the lifecycle of data

The need for a fast and highly scalable data exploration service

What is Azure Synapse?

What is Azure Synapse Data Explorer?

Integrating Data Explorer pools with other Azure Synapse services

Exploring the Data Explorer pool infrastructure and scalability

What makes Azure Synapse Data Explorer unique?

When to use Azure Synapse Data Explorer

Chapter 2: Creating Your First Data Explorer Pool

Chapter 2: Creating Your First Data Explorer Pool

Technical requirements

Creating a free Azure account

Creating an Azure Synapse workspace

Creating a Data Explorer pool using Azure Synapse Studio

Creating a Data Explorer pool using the Azure portal

Creating a Data Explorer pool using the Azure CLI

Chapter 3: Exploring Azure Synapse Studio

Chapter 3: Exploring Azure Synapse Studio

Technical requirements

Exploring the user interface of Azure Synapse Studio

Running your first query

Managing and monitoring Data Explorer pools

Monitoring Data Explorer pools

Chapter 4: Real-World Usage Scenarios

Chapter 4: Real-World Usage Scenarios

Technical requirements

Building a multi-purpose end-to-end analytics environment

Managing IoT data

Processing and analyzing geospatial data

Enabling real-time analytics with big data

Performing time series analytics

Part 2 Working with Data

Part 2 Working with Data

Chapter 5: Ingesting Data into Data Explorer Pools

Chapter 5: Ingesting Data into Data Explorer Pools

Technical requirements

Understanding the data loading process

Defining a retention policy

Choosing a data load strategy

Performing data ingestion

Chapter 6: Data Analysis and Exploration with KQL and Python

Chapter 6: Data Analysis and Exploration with KQL and Python

Technical requirements

Analyzing data with KQL

Exploring Data Explorer pool data with Python

Chapter 7: Data Visualization with Power BI

Chapter 7: Data Visualization with Power BI

Technical requirements

Introduction to the Power BI integration

Creating a Power BI report

Adding data sources to your Power BI report

Connecting Power BI with your Azure Synapse workspace

Authoring Power BI reports from Azure Synapse Studio

Chapter 8: Building Machine Learning Experiments

Chapter 8: Building Machine Learning Experiments

Technical requirements

Understanding the application of ML

Introducing ML into your projects with AutoML

Exploring additional ML capabilities in Azure Synapse

Chapter 9: Exporting Data from Data Explorer Pools

Chapter 9: Exporting Data from Data Explorer Pools

Technical requirements

Understanding data export scenarios

Exporting data with client tools

Using server-side export to pull data

Performing robust exports with server-side data push

Configuring continuous data export

Part 3 Managing Azure Synapse Data Explorer

Part 3 Managing Azure Synapse Data Explorer

Chapter 10: System Monitoring and Diagnostics

Chapter 10: System Monitoring and Diagnostics

Technical requirements

Monitoring your environment

Setting up alerts

Chapter 11: Tuning and Resource Management

Chapter 11: Tuning and Resource Management

Technical requirements

Implementing resource governance with workload groups

Speeding up queries using cache policies

Chapter 12: Securing Your Environment

Chapter 12: Securing Your Environment

Technical requirements

Security overview

Managing data encryption

Authenticating users

Configuring access to resources

Implementing network security

Protecting against external threats

Chapter 13: Advanced Data Management

Chapter 13: Advanced Data Management

Technical requirements

Managing extents

Purging personal data

Index

Other Books You May Enjoy

Other Books You May Enjoy

Packt is searching for authors like you

Share Your Thoughts

Download a free PDF copy of this book

Customer Reviews

5 star

0

4 star

0

3 star

0

2 star

0

1 star

0

What makes Azure Synapse Data Explorer unique?

Even though the underlying service of Data Explorer pools in Azure Synapse is the same as Azure Data Explorer, some capabilities are available exclusively in Azure Synapse. Let us investigate those differences, as follows:

Firewall: Azure Synapse workspaces include a firewall and allow you to configure IP firewall rules to grant or deny access to a workspace. This is not available in the standalone service.
Availability Zones: Enabled by default for Azure Synapse workspaces where Availability Zones are available. This can optionally be enabled when using Azure Data Explorer alone.
VM sizes for compute: Azure Data Explorer offers more than 20 different VM configurations to choose from. For Azure Synapse Data Explorer, a simplified subset of the VM configurations is offered, ranging from extra small (two cores) to large (16 cores).
Code control: As previously mentioned, in Azure Synapse you can connect your workspace with a Git repository, Azure DevOps, or GitHub. This option is not available when using the standalone service.
Pricing: For a Azure Synapse workspace, Data Explorer pools pricing is simplified to two meters: VCore and Storage. When using Azure Data Explorer as a standalone service, you may be charged by using multiple meters such as Compute, Storage, Networking, and the Azure Data Explorer IP markup, which is applied when you make use of fast data ingestion, caching, querying, and management capabilities. Additionally, Reserved Instances, which offer discounted prices when you make a commitment to use an Azure service for a certain period (typically 1 or 3 years), are only available for the standalone service Azure Data Explorer, and not for Azure Synapse Data Explorer.

As seen from the preceding points, there is no significant loss of functionality by using Azure Synapse Data Explorer when compared to the standalone service Azure Data Explorer. Azure Synapse Data Explorer includes the benefits seen on the standalone service, and it also incorporates the enterprise features offered with Synapse workspaces. However, is Azure Synapse Data Explorer the solution to every analytical problem? In the next section, you will find out how to decide whether you need Data Explorer pools or not.