What this book covers
Chapter 1, Introducing Azure Synapse Data Explorer, is the first of four chapters in Part 1, Introduction to Azure Synapse Data Explorer, where you will be introduced to the product and learn the basics that you need before you start to work with data. It welcomes you to Azure Synapse Data Explorer and elaborates on the need for a fast and highly scalable data exploration service for telemetry and log data. It introduces Azure Synapse and explains how the Data Explorer service fits under the Azure Synapse umbrella. Finally, it discusses the architecture and infrastructure of Data Explorer pools, and the scale of the service today.
Chapter 2, Creating Your First Data Explorer Pool, gets your hands busy by walking you through the creation of your first Azure Synapse workspace and a Data Explorer pool using the Azure portal, Azure Synapse Studio, or the Azure Command-Line Interface (CLI). If you are not familiar with Azure yet, don’t worry; this chapter guides you through the steps to create your first free Azure account, allowing you to follow the examples in the book.
Chapter 3, Exploring Azure Synapse Studio, introduces the development and management environment of Azure Synapse. You will learn about the user interface elements of Azure Synapse Studio, and where to find what you are looking for by navigating through the hubs. In addition to that, in this chapter, you will load some data into a database and run your first query to help you familiarize yourself with the query editor. This chapter closes with an overview of where to manage and monitor your environment using Azure Synapse Studio.
Chapter 4, Real-World Usage Scenarios, describes some example solution architectures you can use in common log and telemetry data analytics scenarios. It looks at five real-world use cases that integrate Azure Synapse Data Explorer with other Azure services and helps you understand the blueprints so that you can build your own.
Chapter 5, Ingesting Data into Data Explorer Pools, kicks off Part 2, Working with Data. It walks you through the data loading process, choosing your own data loading strategy, and walks you through different ways to load data into Data Explorer pools. This chapter builds the data assets that you will use in most chapters of the book.
Chapter 6, Data Exploration and Analysis with KQL and Python, is all about learning how to query, transform, and get insights from your data using Kusto Query Language (KQL) and Python. You will learn how to use KQL to explore the data you have at hand and familiarize yourself with the schema, plot simple charts in the query editor, obtain percentiles, and even use native KQL commands to look at trends in your data using linear regression. In the second half of this chapter, you will create an Azure Synapse notebook to explore and transform data using Python and create a lake database.
Chapter 7, Data Visualization with Power BI, complements the previous chapter by helping you configure Power BI integration with Azure Synapse and author new Power BI reports directly from Azure Synapse Studio. It walks you through the creation of reports that connect to data in Data Explorer pools, as well as to your new lake database.
Chapter 8, Building Machine Learning Experiments, provides an overview of applied machine learning, and how to introduce advanced analytics to your Azure Synapse projects using automated machine learning (AutoML). You will use Python to prepare your data for machine learning experiments, train a series of models, and find the best model to help you predict values.
Chapter 9, Exporting Data from Data Explorer Pools, closes Part 2, Working with Data, by walking you through data export scenarios. It explains scenarios where data exports are needed and walks you through different options you have available to perform data exports, including continuous data exports.
Chapter 10, System Monitoring and Diagnostics, is the first of four chapters in Part 3, Managing Azure Synapse Data Explorer. In this chapter, you will learn about managing a platform-as-a-service service such as Azure Synapse, and which parts of the service you should be concerned with. Through code examples and guidance through the user interface, you will learn how to stay on top of your Data Explorer pools and proactively monitor them. By setting up alerts, you’ll learn how to get notified on your phone if an event of interest happens in your environment.
Chapter 11, Tuning and Resource Management, introduces resources to help you provide predictable performance to end users and using cache policies to speed up queries. It walks you through the implementation of resource management to help you categorize user requests to prioritize the execution of critical workloads while queueing requests that can wait.
Chapter 12, Securing Your Environment, provides you with the information you need to make sure your data is secure at rest and in transit, and that only people who are intended to access your data have access to it. It walks you through an overview of the security issues you need to consider for your own implementations, how to double-encrypt your data for an added layer of security, how to authenticate and authorize users, and how to protect the network environment that transits your data.
Chapter 13, Advanced Data Management, covers how to adhere to governmental regulations for data handling, including how to permanently purge personal data. You will learn how to use extents, or data shards, in Azure Synapse Data Explorer to move large volumes of data quickly for archival.