We now have laid down everything to trigger the notebook execution in ADF. Going back to the factory, we're going to add a linked service. So far, all the linked services we created in this book were connected to a data store: SQL Server, blob storage, and so on. This time, we're going to use a computation linked service: Azure Databricks.
As shown in the following screenshot, add a linked service. Click on the
Compute tab, select
Azure Databricks, and click on
We'll now enter the details of the cluster in the next step. We used
Azure Databricks for the
Type property. The following screenshot shows the properties to set:
The properties are explained as follows:
Connect via integration runtime: We use the
Defaultone. It has access to all Azure resources.
Account selection method:
From Azure subscription.
Select cluster: We're going to create a cluster on the fly and it will be running only for the duration of our job; therefore, we select...