Book Image

Data Engineering with Alteryx

By : Paul Houghton
Book Image

Data Engineering with Alteryx

By: Paul Houghton

Overview of this book

Alteryx is a GUI-based development platform for data analytic applications. Data Engineering with Alteryx will help you leverage Alteryx’s code-free aspects which increase development speed while still enabling you to make the most of the code-based skills you have. This book will teach you the principles of DataOps and how they can be used with the Alteryx software stack. You’ll build data pipelines with Alteryx Designer and incorporate the error handling and data validation needed for reliable datasets. Next, you’ll take the data pipeline from raw data, transform it into a robust dataset, and publish it to Alteryx Server following a continuous integration process. By the end of this Alteryx book, you’ll be able to build systems for validating datasets, monitoring workflow performance, managing access, and promoting the use of your data sources.

Related Content you might be interested in

Current Title:

Small book image
Data Engineering with Alteryx
Paul Houghton
Jun, 2022 | 366 pages
Small book image
Learning Alteryx
Renato Baruti
Dec, 2017 | 228 pages
Small book image
Alteryx Designer Cookbook
Alberto Guisande
Oct, 2023 | 740 pages
Small book image
Principles of Data Fabric
Sonia Mezzetta
Apr, 2023 | 188 pages
Table of Contents (18 chapters)
Preface
Preface
Who this book is for
What this book covers
To get the most out of this book
Download the example workflow files
Download the color images
Conventions used
Get in touch
Share Your Thoughts
1
Part 1: Introduction
Part 1: Introduction
Free Chapter
2
Chapter 1: Getting Started with Alteryx
Chapter 1: Getting Started with Alteryx
Understanding the Alteryx platform
Using Alteryx Designer
Leveraging Alteryx Server and Alteryx Connect
Using this book in your data engineering work
Summary
3
Chapter 2: Data Engineering with Alteryx
Chapter 2: Data Engineering with Alteryx
What is a data engineer?
Using Alteryx products as a data engineer
Applying DataOps as an Alteryx data engineer
Summary
4
Chapter 3: DataOps and Its Benefits
Chapter 3: DataOps and Its Benefits
The benefits the DataOps framework brings to your organization
Understanding DataOps principles
Applying DataOps to Alteryx
Using Alteryx software with DataOps
General steps for deploying DataOps in your environment
Summary
5
Part 2: Functional Steps in DataOps
Part 2: Functional Steps in DataOps
6
Chapter 4: Sourcing the Data
Chapter 4: Sourcing the Data
Technical requirements
Accessing internal data sources
Integrating public data sources with Download tool use
Leveraging external data sources from authenticated APIs
Initial cleansing of datasets
Constructing a data pipeline in Alteryx Designer
Summary
7
Chapter 5: Data Processing and Transformations
Chapter 5: Data Processing and Transformations
Technical requirements
The data cleansing process
Profiling data with summary and statistical aggregations
Transforming our data pipeline
Summary
8
Chapter 6: Destination Management
Chapter 6: Destination Management
Technical requirements
Writing to destinations
Managing database connections
Accessing more data sources with custom connections
Integrating data pipelines across environments
Publishing the external data to a Snowflake destination
Summary
9
Chapter 7: Extracting Value
Chapter 7: Extracting Value
Technical requirements
Exploratory data analysis in Alteryx and surfacing the datasets for BI tools
Using Alteryx to deliver standard reports
Summary
10
Chapter 8: Beginning Advanced Analytics
Chapter 8: Beginning Advanced Analytics
Technical requirements
Implementing spatial analytics with Alteryx
Beginning the ML process in Alteryx
Summary
11
Part 3: Governance of DataOps
Part 3: Governance of DataOps
12
Chapter 9: Testing Workflows and Outputs
Chapter 9: Testing Workflows and Outputs
Technical requirements
Workflow tests and messages
Validating data outputs
Centralizing the monitoring outputs with Insights
Summary
13
Chapter 10: Monitoring DataOps and Managing Changes
Chapter 10: Monitoring DataOps and Managing Changes
Technical requirements
Using the Alteryx Server monitoring workflow
Creating an insight dashboard for workflow monitoring
Exporting the MongoDB database for custom analysis
Using Git and GitHub Actions for continuous integration
Summary
14
Chapter 11: Securing and Managing Access
Chapter 11: Securing and Managing Access
Technical requirements
Organizing content on Alteryx Server
Managing collections
Securing the data environment
Summary
15
Chapter 12: Making Data Easy to Use and Discoverable with Alteryx
Chapter 12: Making Data Easy to Use and Discoverable with Alteryx
Technical requirements
What is Alteryx Connect, and how does it help DataOps?
Publishing the data lineage to Alteryx Connect
Data nexus
Syncing the Connect data dictionary with other data catalogs
Summary
16
Chapter 13: Conclusion
Chapter 13: Conclusion
The Alteryx data engineer
The functional steps in DataOps
Governance of DataOps with Alteryx
Our Alteryx data pipeline
Final summary
Why subscribe?
17
Other Books You May Enjoy
Other Books You May Enjoy
Packt is searching for authors like you
Share Your Thoughts

Profiling data with summary and statistical aggregations

Once you have your dataset, getting an idea of what values are in it allows you to understand what the data looks like. Knowing what the data is like provides a reference when you are ready to compare across runs.

In Alteryx, there are a number of ways in which to investigate the range of values that appear in the dataset. In this section, we are going to look at the following three areas:

  • What is the variation in the dataset and the size of the range?
  • How is the dataset distributed?
  • What proportion of your records is missing values?

In each of these areas, Alteryx provides tools for answering the questions quickly and also has methods for those answers to be persisted in your logging systems.

Investigating the variation and size range of your dataset

The first area to investigate is the spread of the data. Understanding the aggregated spread of the records in each field will give you an understanding...

Previous Section
End of Section 4
Next Section