Book Image

Hands-On Data Warehousing with Azure Data Factory

By : Christian Cote, Michelle Gutzait, Giuseppe Ciaburro
Book Image

Hands-On Data Warehousing with Azure Data Factory

By: Christian Cote, Michelle Gutzait, Giuseppe Ciaburro

Overview of this book

ETL is one of the essential techniques in data processing. Given data is everywhere, ETL will always be the vital process to handle data from different sources. Hands-On Data Warehousing with Azure Data Factory starts with the basic concepts of data warehousing and ETL process. You will learn how Azure Data Factory and SSIS can be used to understand the key components of an ETL solution. You will go through different services offered by Azure that can be used by ADF and SSIS, such as Azure Data Lake Analytics, Machine Learning and Databrick’s Spark with the help of practical examples. You will explore how to design and implement ETL hybrid solutions using different integration services with a step-by-step approach. Once you get to grips with all this, you will use Power BI to interact with data coming from different sources in order to reveal valuable insights. By the end of this book, you will not only learn how to build your own ETL solutions but also address the key challenges that are faced while building them.
Table of Contents (12 chapters)

Azure Data Factory


Now, we'll create the factory. The goal of the exercise is to copy data from a SQL Server table and bring it in an Azure Blob storage.

Log in to the Azure portal (http://Portal.Azure.com). In the resource section, click the + New icon. Click on Data + Analytics and select Data Factory, as shown in the following screenshot:

The New data factory blade opens. As shown in the following screenshot, fill the textboxes with the following values:

  • Name: The name of the factory might be later registered as DNS. It should be unique if at all possible. To make it unique, we might use our initials in front of it. There are naming rules for data factories, which can be found at https://docs.microsoft.com/en-us/azure/data-factory/naming-rules.
  • Subscription: Should be filled with the active subscription used to create the data factory.
  • Resource Group: We're using the resource group created earlier in this chapter.
  • Version: Since this book talks about V2, we'll use V2 of the data factory. At...