Constructing a data pipeline in Alteryx Designer

Now that we have a working extraction workflow, we need to convert this into a production pipeline for automation. We need to add the extra documentation and the validation for the workflow to do this.

The steps we will implement in relation to the workflow are as follows:

• Configure the annotations/names of tools, meaning the logs are more helpful.
• Add the initial documentation to the workflow.
• Calculate the first initial statistical values for the dataset.
• Add an output step to persist the processed dataset.

The previous steps are added to the workflow to support the DataOps deployment by building the Confidence pillar. In addition, these steps allow anyone to understand the workflow quickly in the future.

Figure 4.18 – Overview of the entire pipeline

The preceding diagram shows an overview of our example workflow. This workflow performs the following processes:

...