Book Image

Data Engineering with Alteryx

By : Paul Houghton
Book Image

Data Engineering with Alteryx

By: Paul Houghton

Overview of this book

Alteryx is a GUI-based development platform for data analytic applications. Data Engineering with Alteryx will help you leverage Alteryx’s code-free aspects which increase development speed while still enabling you to make the most of the code-based skills you have. This book will teach you the principles of DataOps and how they can be used with the Alteryx software stack. You’ll build data pipelines with Alteryx Designer and incorporate the error handling and data validation needed for reliable datasets. Next, you’ll take the data pipeline from raw data, transform it into a robust dataset, and publish it to Alteryx Server following a continuous integration process. By the end of this Alteryx book, you’ll be able to build systems for validating datasets, monitoring workflow performance, managing access, and promoting the use of your data sources.

Related Content you might be interested in

Current Title:

Small book image
Data Engineering with Alteryx
Paul Houghton
Jun, 2022 | 366 pages
Small book image
Learning Alteryx
Mayur Ravindra Narkhede | Alok Khobragade | Renato Baruti
Dec, 2017 | 228 pages
Small book image
Alteryx Designer Cookbook
Alberto Guisande
Oct, 2023 | 740 pages
Small book image
Principles of Data Fabric
Sonia Mezzetta | DR. Tommy Dang
Apr, 2023 | 188 pages
Table of Contents (18 chapters)
Preface
Preface
Who this book is for
What this book covers
To get the most out of this book
Download the example workflow files
Download the color images
Conventions used
Get in touch
Share Your Thoughts
1
Part 1: Introduction
Part 1: Introduction
Free Chapter
2
Chapter 1: Getting Started with Alteryx
Chapter 1: Getting Started with Alteryx
Understanding the Alteryx platform
Using Alteryx Designer
Leveraging Alteryx Server and Alteryx Connect
Using this book in your data engineering work
Summary
3
Chapter 2: Data Engineering with Alteryx
Chapter 2: Data Engineering with Alteryx
What is a data engineer?
Using Alteryx products as a data engineer
Applying DataOps as an Alteryx data engineer
Summary
4
Chapter 3: DataOps and Its Benefits
Chapter 3: DataOps and Its Benefits
The benefits the DataOps framework brings to your organization
Understanding DataOps principles
Applying DataOps to Alteryx
Using Alteryx software with DataOps
General steps for deploying DataOps in your environment
Summary
5
Part 2: Functional Steps in DataOps
Part 2: Functional Steps in DataOps
6
Chapter 4: Sourcing the Data
Chapter 4: Sourcing the Data
Technical requirements
Accessing internal data sources
Integrating public data sources with Download tool use
Leveraging external data sources from authenticated APIs
Initial cleansing of datasets
Constructing a data pipeline in Alteryx Designer
Summary
7
Chapter 5: Data Processing and Transformations
Chapter 5: Data Processing and Transformations
Technical requirements
The data cleansing process
Profiling data with summary and statistical aggregations
Transforming our data pipeline
Summary
8
Chapter 6: Destination Management
Chapter 6: Destination Management
Technical requirements
Writing to destinations
Managing database connections
Accessing more data sources with custom connections
Integrating data pipelines across environments
Publishing the external data to a Snowflake destination
Summary
9
Chapter 7: Extracting Value
Chapter 7: Extracting Value
Technical requirements
Exploratory data analysis in Alteryx and surfacing the datasets for BI tools
Using Alteryx to deliver standard reports
Summary
10
Chapter 8: Beginning Advanced Analytics
Chapter 8: Beginning Advanced Analytics
Technical requirements
Implementing spatial analytics with Alteryx
Beginning the ML process in Alteryx
Summary
11
Part 3: Governance of DataOps
Part 3: Governance of DataOps
12
Chapter 9: Testing Workflows and Outputs
Chapter 9: Testing Workflows and Outputs
Technical requirements
Workflow tests and messages
Validating data outputs
Centralizing the monitoring outputs with Insights
Summary
13
Chapter 10: Monitoring DataOps and Managing Changes
Chapter 10: Monitoring DataOps and Managing Changes
Technical requirements
Using the Alteryx Server monitoring workflow
Creating an insight dashboard for workflow monitoring
Exporting the MongoDB database for custom analysis
Using Git and GitHub Actions for continuous integration
Summary
14
Chapter 11: Securing and Managing Access
Chapter 11: Securing and Managing Access
Technical requirements
Organizing content on Alteryx Server
Managing collections
Securing the data environment
Summary
15
Chapter 12: Making Data Easy to Use and Discoverable with Alteryx
Chapter 12: Making Data Easy to Use and Discoverable with Alteryx
Technical requirements
What is Alteryx Connect, and how does it help DataOps?
Publishing the data lineage to Alteryx Connect
Data nexus
Syncing the Connect data dictionary with other data catalogs
Summary
16
Chapter 13: Conclusion
Chapter 13: Conclusion
The Alteryx data engineer
The functional steps in DataOps
Governance of DataOps with Alteryx
Our Alteryx data pipeline
Final summary
Why subscribe?
17
Other Books You May Enjoy
Other Books You May Enjoy
Packt is searching for authors like you
Share Your Thoughts

Publishing the data lineage to Alteryx Connect

We have been looking at a data catalog where all the individual assets, such as the tables or fields, have the information already populated. For this information to be available, you need to populate the data asset information into Connect. There are three methods for populating the Connect data assets:

  • Loading metadata directly from Connect
  • Using the prebuilt workflow apps
  • Creating a custom-built data source with the Connect APIs

These methods allow you to populate your Connect data dictionary with the information that exists in your data asset. For databases, Alteryx will extract any comment information in addition to tables, field names, and field types. Let's take a look at them one by one.

Loading metadata directly from Connect

The first method for loading metadata is directly from Alteryx Connect. To initialize the metadata collection from Connect, there are two prerequisites:

  • You must be...
Previous Section
End of Section 4
Next Section