Book Image

SAP Data Services 4.x Cookbook

Book Image

SAP Data Services 4.x Cookbook

Overview of this book

Want to cost effectively deliver trusted information to all of your crucial business functions? SAP Data Services delivers one enterprise-class solution for data integration, data quality, data profiling, and text data processing. It boosts productivity with a single solution for data quality and data integration. SAP Data Services also enables you to move, improve, govern, and unlock big data. This book will lead you through the SAP Data Services environment to efficiently develop ETL processes. To begin with, you’ll learn to install, configure, and prepare the ETL development environment. You will get familiarized with the concepts of developing ETL processes with SAP Data Services. Starting from smallest unit of work- the data flow, the chapters will lead you to the highest organizational unit—the Data Services job, revealing the advanced techniques of ETL design. You will learn to import XML files by creating and implementing real-time jobs. It will then guide you through the ETL development patterns that enable the most effective performance when extracting, transforming, and loading data. You will also find out how to create validation functions and transforms. Finally, the book will show you the benefits of data quality management with the help of another SAP solution—Information Steward.
Table of Contents (19 chapters)
SAP Data Services 4.x Cookbook
About the Author
About the Reviewers

Building an external ETL audit and audit reporting

In this recipe, we will implement the external user-built ETL audit mechanism. Our ETL audit will include information about the start and stop times of the workflows running within the job, their statuses, names, and information about which job they belong to.

Getting ready…

We need to create an ETL audit table in our database where we will store the audit results.

Connect to the STAGE database using the SQL Server Management Studio and execute the following statement to create the ETL audit table:

create table dbo.etl_audit (
  job_run_id integer, 
  workflow_status varchar(50),
  job_name varchar(255),
  start_dt datetime,
  end_dt datetime,
  process_name varchar(255)

How to do it…

First, we need to choose objects for auditing. The following steps should be implemented for every workflow or dataflow that you want to collect auditing information about. In this particular example, we will enable ETL auditing for the job object itself.

  1. Create...