Book Image

SAP Data Services 4.x Cookbook

Book Image

SAP Data Services 4.x Cookbook

Overview of this book

Want to cost effectively deliver trusted information to all of your crucial business functions? SAP Data Services delivers one enterprise-class solution for data integration, data quality, data profiling, and text data processing. It boosts productivity with a single solution for data quality and data integration. SAP Data Services also enables you to move, improve, govern, and unlock big data. This book will lead you through the SAP Data Services environment to efficiently develop ETL processes. To begin with, you’ll learn to install, configure, and prepare the ETL development environment. You will get familiarized with the concepts of developing ETL processes with SAP Data Services. Starting from smallest unit of work- the data flow, the chapters will lead you to the highest organizational unit—the Data Services job, revealing the advanced techniques of ETL design. You will learn to import XML files by creating and implementing real-time jobs. It will then guide you through the ETL development patterns that enable the most effective performance when extracting, transforming, and loading data. You will also find out how to create validation functions and transforms. Finally, the book will show you the benefits of data quality management with the help of another SAP solution—Information Steward.

Related Content you might be interested in

Current Title:

Small book image
SAP Data Services 4.x Cookbook
Dec, 2015 | 430 pages
No titles found
Table of Contents (19 chapters)
SAP Data Services 4.x Cookbook
SAP Data Services 4.x Cookbook
Credits
Credits
About the Author
About the Author
About the Reviewers
About the Reviewers
www.PacktPub.com
www.PacktPub.com
Preface
Preface
Free Chapter
1
Introduction to ETL Development
Introduction to ETL Development
Introduction
Preparing a database environment
Creating a source system database
Defining and creating staging area structures
Creating a target data warehouse
2
Configuring the Data Services Environment
Configuring the Data Services Environment
Introduction
Creating IPS and Data Services repositories
Installing and configuring Information Platform Services
Installing and configuring Data Services
Configuring user access
Starting and stopping services
Administering tasks
Understanding the Designer tool
3
Data Services Basics – Data Types, Scripting Language, and Functions
Data Services Basics – Data Types, Scripting Language, and Functions
Introduction
Creating variables and parameters
Creating a script
Using string functions
Using date functions
Using conversion functions
Using database functions
Using aggregate functions
Using math functions
Using miscellaneous functions
Creating custom functions
4
Dataflow – Extract, Transform, and Load
Dataflow – Extract, Transform, and Load
Introduction
Creating a source data object
Creating a target data object
Loading data into a flat file
Loading data from a flat file
Loading data from table to table – lookups and joins
Using the Map_Operation transform
Using the Table_Comparison transform
Exploring the Auto correct load option
Splitting the flow of data with the Case transform
Monitoring and analyzing dataflow execution
5
Workflow – Controlling Execution Order
Workflow – Controlling Execution Order
Introduction
Creating a workflow object
Nesting workflows to control the execution order
Using conditional and while loop objects to control the execution order
Using the bypassing feature
Controlling failures – try-catch objects
Use case example – populating dimension tables
6
Job – Building the ETL Architecture
Job – Building the ETL Architecture
Introduction
Projects and jobs – organizing ETL
Using object replication
Migrating ETL code through the central repository
Migrating ETL code with export/import
Debugging job execution
Monitoring job execution
Building an external ETL audit and audit reporting
Using built-in Data Services ETL audit and reporting functionality
Auto Documentation in Data Services
7
Validating and Cleansing Data
Validating and Cleansing Data
Introduction
Creating validation functions
Using validation functions with the Validation transform
Reporting data validation results
Using regular expression support to validate data
Enabling dataflow audit
Data Quality transforms – cleansing your data
8
Optimizing ETL Performance
Optimizing ETL Performance
Introduction
Optimizing dataflow execution – push-down techniques
Optimizing dataflow execution – the SQL transform
Optimizing dataflow execution – the Data_Transfer transform
Optimizing dataflow readers – lookup methods
Optimizing dataflow loaders – bulk-loading methods
Optimizing dataflow execution – performance options
9
Advanced Design Techniques
Advanced Design Techniques
Introduction
Change Data Capture techniques
Automatic job recovery in Data Services
Simplifying ETL execution with system configurations
Transforming data with the Pivot transform
10
Developing Real-time Jobs
Developing Real-time Jobs
Introduction
Working with nested structures
The XML_Map transform
The Hierarchy_Flattening transform
Configuring Access Server
Creating real-time jobs
11
Working with SAP Applications
Working with SAP Applications
Introduction
Loading data into SAP ERP
12
Introduction to Information Steward
Introduction to Information Steward
Introduction
Exploring Data Insight capabilities
Performing Metadata Management tasks
Working with the Metapedia functionality
Creating a custom cleansing package with Cleansing Package Builder
Index
Index

Enabling dataflow audit

Auditing in Data Services allows the collection of additional information about the data migrated from the source to the target by a specific dataflow on which the audit is enabled, and even allows making decisions according to the rules applied on the audit data. In this recipe, we will see how audit can be enabled and utilized during the extraction of data from the source system.

Getting ready

For this recipe, you can use the dataflow DF_Extract_Address from the previous recipes of this chapter.

How to do it…

Perform the following steps to enable the auditing for the specific dataflow.

  1. Open DF_Extract_Address in the workspace window and select Tools | Audit from the top-level menu.

  2. In the newly opened window, select the Label tab, right-click in the empty space, and choose Show All Objects from the context menu.

  3. The Label tab displays the list of objects from within a dataflow. Enable auditing on the Query and Pass Query transform objects by right-clicking on them and...

Previous Section
End of Section 7
Next Section