Book Image

Data Engineering with Alteryx

By : Paul Houghton
Book Image

Data Engineering with Alteryx

By: Paul Houghton

Overview of this book

Alteryx is a GUI-based development platform for data analytic applications. Data Engineering with Alteryx will help you leverage Alteryx’s code-free aspects which increase development speed while still enabling you to make the most of the code-based skills you have. This book will teach you the principles of DataOps and how they can be used with the Alteryx software stack. You’ll build data pipelines with Alteryx Designer and incorporate the error handling and data validation needed for reliable datasets. Next, you’ll take the data pipeline from raw data, transform it into a robust dataset, and publish it to Alteryx Server following a continuous integration process. By the end of this Alteryx book, you’ll be able to build systems for validating datasets, monitoring workflow performance, managing access, and promoting the use of your data sources.
Table of Contents (18 chapters)
Part 1: Introduction
Part 2: Functional Steps in DataOps
Part 3: Governance of DataOps

Writing to destinations

The requirements for managing the destinations are similar to those for connecting to internal datasets described in Chapter 4, Sourcing the Data. Managing the destinations falls into three categories:

  • Writing to local files
  • Writing to databases
  • Custom connection tools from Alteryx Gallery

Managing these three categories on a local machine is relatively simple, but should be done in a manner that works for other people or environments (such as your Alteryx Server). In this section, we will look at the default way in which you connect to local files and how we can make that connection process more robust, why those changes fit with our DataOps processes, and how we can leverage these changes for collaboration.

Writing to files

Saving files to a location on your computer is a good way to persist data during a pipeline development. When you do save the file, it is often saved as a full path, such as C:\Users\username\project\output_file...