What this book covers
Chapter 1, Getting Started with ADF, will briefly show you the Azure data platform. In this chapter, you will learn about the ADF interface and options as well as common use cases. You will perform hands-on exercises in order to find ADF in the Azure portal and create your first job.
Chapter 2, Orchestration and Control Flow, will introduce you to the building blocks of the data processing in Azure Data Factory. The chapter contains hands-on exercises which show you how to set up linked services and datasets for your data sources, use various types of activities, design data-processing workflows, and create triggers for the data transfers.
Chapter 3, Setting up a Cloud Data Warehouse, covers key features and benefits of cloud data warehousing and Azure Synapse Analytics. You will learn how to connect and configure Azure Synapse Analytics, load data, build transformation processes, and operate data flows.
Chapter 4, Working with Azure Data Lake, will go through the features of Azure Data Lake Storage Gen2. This is multi-modal cloud storage that is frequently used for big data analytics. We will load and manage the datasets that we will use for analytics in the next chapter.
Chapter 5, Working with Big Data – HDInsight and Databricks, is where we will actively engage with analytical tools from the Azure data services. We will start with munging data with Azure Databricks, then train some models on big data, and analyze them to draw business insights. Also, we will go through Stream Analytics.
Chapter 6, Integration with MS SSIS, covers using the Azure data platform and ADF on-premises. This chapter will help you leverage your on-premises infrastructure together with cloud-native tools to get relevant business insights.
Chapter 7, Data Migration – Azure Data Factory and Other Cloud Services, explains how to use Azure Data factory to transfer data between Azure and other cloud providers, such as AWS or Google Cloud, using ADF built-in connectors. We also show how to integrate a provider not currently supported by a built-in ADF connector, using Dropbox as an example.
Chapter 8, Working with Azure Services Integration, will cover how to do integrations of the most commonly used Azure services into ADF. You will also learn how Azure services can be useful in designing ETL pipelines.
Chapter 9, Managing Deployment Processes with Azure DevOps, will cover the key features of Azure DevOps. You will learn how to build CI/CD processes and continuous monitoring with Microsoft Azure. You will create a platform for application deployment and integrate it with ADF.
Chapter 10, Monitoring and Troubleshooting Data Pipelines, will teach readers how to use the Azure Data Factory Monitor interface to evaluate the progress of your data transfers, how to understand error messages and set up alerts for the pipelines. This chapter contains hands-on recipes highlighting the debugging capabilities of ADF.