Monitoring data pipeline performance
Similar to the data movement metrics we saw in the previous section, ADF provides metrics for overall pipelines too. In the Pipeline runs page under the Monitoring tab, if you hover over the pipeline runs, a small Consumption icon appears, as shown in the following screenshot:
Figure 13.19 – Consumption icon in the Pipeline runs screen
If you click on that icon, ADF shows the pipeline consumption details. Here is a sample screen:
Figure 13.20 – Pipeline resource consumption details screen
You can also get additional metrics about each of the runs from the Gantt chart section. You can change the view from List to Gantt, as shown in the following screenshot:
Figure 13.21 – Additional pipeline details in the Gantt chart page
Note
ADF only maintains pipeline execution details and metrics for 45 days. If you need to analyze the pipeline data for more than...