Book Image

Limitless Analytics with Azure Synapse

By : Prashant Kumar Mishra
Book Image

Limitless Analytics with Azure Synapse

By: Prashant Kumar Mishra

Overview of this book

Azure Synapse Analytics, which Microsoft describes as the next evolution of Azure SQL Data Warehouse, is a limitless analytics service that brings enterprise data warehousing and big data analytics together. With this book, you'll learn how to discover insights from your data effectively using this platform. The book starts with an overview of Azure Synapse Analytics, its architecture, and how it can be used to improve business intelligence and machine learning capabilities. Next, you'll go on to choose and set up the correct environment for your business problem. You'll also learn a variety of ways to ingest data from various sources and orchestrate the data using transformation techniques offered by Azure Synapse. Later, you'll explore how to handle both relational and non-relational data using the SQL language. As you progress, you'll perform real-time streaming and execute data analysis operations on your data using various languages, before going on to apply ML techniques to derive accurate and granular insights from data. Finally, you'll discover how to protect sensitive data in real time by using security and privacy features. By the end of this Azure book, you'll be able to build end-to-end analytics solutions while focusing on data prep, data management, data warehousing, and AI tasks.
Table of Contents (20 chapters)
1
Section 1: The Basics and Key Concepts
4
Section 2: Data Ingestion and Orchestration
8
Section 3: Azure Synapse for Data Scientists and Business Analysts
14
Section 4: Best Practices

Using various activities in Synapse pipelines

Synapse pipelines give you the option to add various transformations; however, we will try to cover just a couple of transformations in this section. Proceed as follows:

  1. Navigate to the Integrate tab on Synapse Studio and click on + to select Pipeline out of the other available options, as illustrated in the following screenshot:
    Figure 4.18 – Creating a Synapse pipeline in Synapse Studio

    Figure 4.18 – Creating a Synapse pipeline in Synapse Studio

  2. Fill in the name and description in the Properties window of the pipeline that you created in the preceding step and click on Publish all to save the changes.
  3. Let's add some activities to the canvas. We are going to select the Get Metadata activity from the list of all available activities to begin with, as illustrated in the following screenshot:
    Figure 4.19 – Adding the Get Metadata activity to the Synapse pipeline canvas

    Figure 4.19 – Adding the Get Metadata activity to the Synapse pipeline canvas

  4. Provide a name for this activity in the General tab. We are going to enter GetMetadataForZipFiles...