Book Image

Azure Synapse Analytics Cookbook

By : Gaurav Agarwal, Meenakshi Muralidharan
Book Image

Azure Synapse Analytics Cookbook

By: Gaurav Agarwal, Meenakshi Muralidharan

Overview of this book

As data warehouse management becomes increasingly integral to successful organizations, choosing and running the right solution is more important than ever. Microsoft Azure Synapse is an enterprise-grade, cloud-based data warehousing platform, and this book holds the key to using Synapse to its full potential. If you want the skills and confidence to create a robust enterprise analytical platform, this cookbook is a great place to start. You'll learn and execute enterprise-level deployments on medium-to-large data platforms. Using the step-by-step recipes and accompanying theory covered in this book, you'll understand how to integrate various services with Synapse to make it a robust solution for all your data needs. Whether you're new to Azure Synapse or just getting started, you'll find the instructions you need to solve any problem you may face, including using Azure services for data visualization as well as for artificial intelligence (AI) and machine learning (ML) solutions. By the end of this Azure book, you'll have the skills you need to implement an enterprise-grade analytical platform, enabling your organization to explore and manage heterogeneous data workloads and employ various data integration services to solve real-time industry problems.

Related Content you might be interested in

Current Title:

Small book image
Azure Synapse Analytics Cookbook
Gaurav Agarwal | Meenakshi Muralidharan
Apr, 2022 | 238 pages
Small book image
Limitless Analytics with Azure Synapse
Prashant Kumar Mishra
Jun, 2021 | 392 pages
Small book image
Cloud Scale Analytics with Azure Data Services
Patrik Borosch
Jul, 2021 | 520 pages
Small book image
Azure Data Engineer Associate Certification Guide
Newton Alex
Feb, 2022 | 574 pages
Table of Contents (11 chapters)
Preface
Preface
Who this book is for
What this book covers
To get the most out of this book
Download the example code files
Download the color images
Conventions used
Get in touch
Share Your Thoughts
1
Chapter 1: Choosing the Optimal Method for Loading Data to Synapse
Chapter 1: Choosing the Optimal Method for Loading Data to Synapse
Choosing a data loading option
Achieving parallelism in data loading using PolyBase
Moving and transforming using a data flow
Adding a trigger to a data flow pipeline
Unsupported data loading scenarios
Data loading best practices
Free Chapter
2
Chapter 2: Creating Robust Data Pipelines and Data Transformation
Chapter 2: Creating Robust Data Pipelines and Data Transformation
Reading and writing data from ADLS Gen2 using PySpark
Visualizing data in a Synapse notebook
3
Chapter 3: Processing Data Optimally across Multiple Nodes
Chapter 3: Processing Data Optimally across Multiple Nodes
Working with the resource consumption model of Synapse SQL
Optimizing analytics with dedicated SQL pool and working on data distribution
Working with serverless SQL pool
Processing and querying very large datasets
Script for statistics in Synapse SQL
4
Chapter 4: Engineering Real-Time Analytics with Azure Synapse Link Using Cosmos DB
Chapter 4: Engineering Real-Time Analytics with Azure Synapse Link Using Cosmos DB
Integrating an Azure Synapse ETL pipeline with Cosmos DB
Setting up Azure Cosmos DB analytical store
Enabling Azure Synapse Link and connecting Azure Cosmos DB to Azure Synapse
IoT end-to-end solutions and getting real-time insights
Use cases using Synapse Link
5
Chapter 5: Data Transformation and Processing with Synapse Notebooks
Chapter 5: Data Transformation and Processing with Synapse Notebooks
Landing data in ADLS Gen2
Exploring data with ADLS Gen2 to pandas DataFrame in Synapse notebook
Processing data from a PySpark notebook within Synapse
Performing read-write operations to a Parquet file using Spark in Synapse
Analytics with Spark
6
Chapter 6: Enriching Data Using the Azure ML AutoML Regression Model
Chapter 6: Enriching Data Using the Azure ML AutoML Regression Model
Training a model using AutoML in Synapse
Building a regression model from Azure Machine Learning in Synapse Studio
Modeling and scoring using SQL pools
An overview of Spark MLlib and Azure Synapse
Integrating AI and Cognitive Services
7
Chapter 7: Visualizing and Reporting Petabytes of Data
Chapter 7: Visualizing and Reporting Petabytes of Data
Combining Power BI and aserverless SQL pool
Working on a composite model
Using materialized views to improve performance
8
Chapter 8: Data Cataloging and Governance
Chapter 8: Data Cataloging and Governance
Configuring your Azure Purview account for Synapse SQL pool
Scanning data using the Purview data catalog
Enumerating resources within Synapse Studio
9
Chapter 9: MPP Platform Migration to Synapse
Chapter 9: MPP Platform Migration to Synapse
Understanding data migration challenges
Configuring Azure Synapse Pathway
Evaluating a data source to be migrated
Generating a data migration assessment
Supported data sources for migration
Why subscribe?
10
Other Books You May Enjoy
Other Books You May Enjoy
Packt is searching for authors like you
Share Your Thoughts

What this book covers

Chapter 1, Choosing the Optimal Method for Loading Data to Synapse, will help you learn how to choose between different options when loading data into Synapse and the optimal way to perform different data loadings.

Chapter 2, Creating Robust Data Pipelines and Data Transformation, will help you understand Synapse notebooks and its interfaces to create a file that will contain the real code – the logic. You will also learn how to visualize data within a notebook and other big data scenarios.

Chapter 3, Processing Data Optimally across Multiple Nodes, explores the Synapse SQL architecture components and how to leverage the scale-out capabilities to distribute the computational processing of data across multiple nodes.

Chapter 4, Engineering Real-time Analytics with Azure Synapse Link Using Cosmos DB, describes how you can architect and perform real-time analytics with Synapse, integrate Synapse Link for Cosmos DB, and enable the Internet of Things (IoT).

Chapter 5, Data Transformation and Processing with Synapse Notebooks, teaches you how to use Python to read data from Azure Data Lake Storage Gen2 into a Spark DataFrame using Azure Synapse Analytics.

Chapter 6, Enriching Data Using the Azure ML AutoML Regression Model, helps you uncover the power of Azure Machine Learning along with Spark MLlib and Synapse Studio.

Chapter 7, Visualizing and Reporting Petabytes of Data, teaches you how to present data using visualizations with Power BI, integrate Power BI with Synapse, and use the power of the serverless SQL pool for data exploration.

Chapter 8, Data Cataloging and Governance, teaches you how to provide comprehensive data governance for an analytical workload and embed data discovery and classification with Synapse using Azure Purview integration.

Chapter 9, MPP Platform Migration to Synapse, teaches you how to get started with the migration of a legacy data warehouse using Azure Synapse Pathway.

Previous Section
Next Section