Book Image

Data Engineering with Alteryx

By : Paul Houghton
Book Image

Data Engineering with Alteryx

By: Paul Houghton

Overview of this book

Alteryx is a GUI-based development platform for data analytic applications. Data Engineering with Alteryx will help you leverage Alteryx’s code-free aspects which increase development speed while still enabling you to make the most of the code-based skills you have. This book will teach you the principles of DataOps and how they can be used with the Alteryx software stack. You’ll build data pipelines with Alteryx Designer and incorporate the error handling and data validation needed for reliable datasets. Next, you’ll take the data pipeline from raw data, transform it into a robust dataset, and publish it to Alteryx Server following a continuous integration process. By the end of this Alteryx book, you’ll be able to build systems for validating datasets, monitoring workflow performance, managing access, and promoting the use of your data sources.

Related Content you might be interested in

Current Title:

Small book image
Data Engineering with Alteryx
Paul Houghton
Jun, 2022 | 366 pages
Small book image
Learning Alteryx
Renato Baruti
Dec, 2017 | 228 pages
Small book image
Alteryx Designer Cookbook
Alberto Guisande
Oct, 2023 | 740 pages
Small book image
Principles of Data Fabric
Sonia Mezzetta
Apr, 2023 | 188 pages
Table of Contents (18 chapters)
Preface
Preface
Who this book is for
What this book covers
To get the most out of this book
Download the example workflow files
Download the color images
Conventions used
Get in touch
Share Your Thoughts
1
Part 1: Introduction
Part 1: Introduction
Free Chapter
2
Chapter 1: Getting Started with Alteryx
Chapter 1: Getting Started with Alteryx
Understanding the Alteryx platform
Using Alteryx Designer
Leveraging Alteryx Server and Alteryx Connect
Using this book in your data engineering work
Summary
3
Chapter 2: Data Engineering with Alteryx
Chapter 2: Data Engineering with Alteryx
What is a data engineer?
Using Alteryx products as a data engineer
Applying DataOps as an Alteryx data engineer
Summary
4
Chapter 3: DataOps and Its Benefits
Chapter 3: DataOps and Its Benefits
The benefits the DataOps framework brings to your organization
Understanding DataOps principles
Applying DataOps to Alteryx
Using Alteryx software with DataOps
General steps for deploying DataOps in your environment
Summary
5
Part 2: Functional Steps in DataOps
Part 2: Functional Steps in DataOps
6
Chapter 4: Sourcing the Data
Chapter 4: Sourcing the Data
Technical requirements
Accessing internal data sources
Integrating public data sources with Download tool use
Leveraging external data sources from authenticated APIs
Initial cleansing of datasets
Constructing a data pipeline in Alteryx Designer
Summary
7
Chapter 5: Data Processing and Transformations
Chapter 5: Data Processing and Transformations
Technical requirements
The data cleansing process
Profiling data with summary and statistical aggregations
Transforming our data pipeline
Summary
8
Chapter 6: Destination Management
Chapter 6: Destination Management
Technical requirements
Writing to destinations
Managing database connections
Accessing more data sources with custom connections
Integrating data pipelines across environments
Publishing the external data to a Snowflake destination
Summary
9
Chapter 7: Extracting Value
Chapter 7: Extracting Value
Technical requirements
Exploratory data analysis in Alteryx and surfacing the datasets for BI tools
Using Alteryx to deliver standard reports
Summary
10
Chapter 8: Beginning Advanced Analytics
Chapter 8: Beginning Advanced Analytics
Technical requirements
Implementing spatial analytics with Alteryx
Beginning the ML process in Alteryx
Summary
11
Part 3: Governance of DataOps
Part 3: Governance of DataOps
12
Chapter 9: Testing Workflows and Outputs
Chapter 9: Testing Workflows and Outputs
Technical requirements
Workflow tests and messages
Validating data outputs
Centralizing the monitoring outputs with Insights
Summary
13
Chapter 10: Monitoring DataOps and Managing Changes
Chapter 10: Monitoring DataOps and Managing Changes
Technical requirements
Using the Alteryx Server monitoring workflow
Creating an insight dashboard for workflow monitoring
Exporting the MongoDB database for custom analysis
Using Git and GitHub Actions for continuous integration
Summary
14
Chapter 11: Securing and Managing Access
Chapter 11: Securing and Managing Access
Technical requirements
Organizing content on Alteryx Server
Managing collections
Securing the data environment
Summary
15
Chapter 12: Making Data Easy to Use and Discoverable with Alteryx
Chapter 12: Making Data Easy to Use and Discoverable with Alteryx
Technical requirements
What is Alteryx Connect, and how does it help DataOps?
Publishing the data lineage to Alteryx Connect
Data nexus
Syncing the Connect data dictionary with other data catalogs
Summary
16
Chapter 13: Conclusion
Chapter 13: Conclusion
The Alteryx data engineer
The functional steps in DataOps
Governance of DataOps with Alteryx
Our Alteryx data pipeline
Final summary
Why subscribe?
17
Other Books You May Enjoy
Other Books You May Enjoy
Packt is searching for authors like you
Share Your Thoughts

Publishing the external data to a Snowflake destination

In this section, we will take the places data that we downloaded in Chapter 5, Data Processing and Transformations, and upload it to a Snowflake database using the bulk loader. We chose a Snowflake account for two reasons. First, as a Premier Alteryx Technology partner, they are committed to creating and maintaining connections and tools to use with Alteryx. Secondly, they offer a trial account allowing anyone to try the database for free and have an account set up in minutes.

To achieve this, we will need to do the following:

  • Ensure we have the drivers for our database installed.
  • Configure the DSN connection.
  • Configure the output bulk load options.

These three are all that we need to complete the upload of our places dataset.

Installing the drivers

The drivers for Snowflake can be found in two ways. The first way is from the Alteryx Data Sources page seen in the following screenshot and which can...

Previous Section
End of Section 7
Next Section