So far in this book, we have seen that ETL can be done on-premises with an existing SSIS implementation. For cloud ETL, we used Azure Data Lake Analytics (ADLA). Sparks is one of the other major players when it comes to data integration on the cloud. Databricks is Sparks' latest addition in Azure services at the time of writing this book. It is meant to be simple to set up and use while delivering fast performance in a production environment.
In this chapter, we'll see how ADF can trigger a Databricks notebook run. Later in this book, we'll show how we can integrate this data into a Power BI report. This chapter introduces only the simple Databricks integration in ADF.