5. Workflow – Controlling Execution Order | SAP Data Services 4.x Cookbook

Sign In Start Free Trial

Book Overview & Buying
Table Of Contents
Feedback & Rating

SAP Data Services 4.x Cookbook

By : Shomnikov

4 (1)

SAP Data Services 4.x Cookbook

4 (1)

By: Shomnikov

Overview of this book

Want to cost effectively deliver trusted information to all of your crucial business functions? SAP Data Services delivers one enterprise-class solution for data integration, data quality, data profiling, and text data processing. It boosts productivity with a single solution for data quality and data integration. SAP Data Services also enables you to move, improve, govern, and unlock big data. This book will lead you through the SAP Data Services environment to efficiently develop ETL processes. To begin with, you’ll learn to install, configure, and prepare the ETL development environment. You will get familiarized with the concepts of developing ETL processes with SAP Data Services. Starting from smallest unit of work- the data flow, the chapters will lead you to the highest organizational unit—the Data Services job, revealing the advanced techniques of ETL design. You will learn to import XML files by creating and implementing real-time jobs. It will then guide you through the ETL development patterns that enable the most effective performance when extracting, transforming, and loading data. You will also find out how to create validation functions and transforms. Finally, the book will show you the benefits of data quality management with the help of another SAP solution—Information Steward.

Preface

Preface

What this book covers

What you need for this book

Who this book is for

Sections

Conventions

Reader feedback

Customer support

Free Chapter

1. Introduction to ETL Development

1. Introduction to ETL Development

Introduction

Preparing a database environment

Creating a source system database

Defining and creating staging area structures

Creating a target data warehouse

2. Configuring the Data Services Environment

2. Configuring the Data Services Environment

Introduction

Creating IPS and Data Services repositories

Installing and configuring Information Platform Services

Installing and configuring Data Services

Configuring user access

Starting and stopping services

Administering tasks

Understanding the Designer tool

3. Data Services Basics – Data Types, Scripting Language, and Functions

3. Data Services Basics – Data Types, Scripting Language, and Functions

Introduction

Creating variables and parameters

Creating a script

Using string functions

Using date functions

Using conversion functions

Using database functions

Using aggregate functions

Using math functions

Using miscellaneous functions

Creating custom functions

4. Dataflow – Extract, Transform, and Load

4. Dataflow – Extract, Transform, and Load

Introduction

Creating a source data object

Creating a target data object

Loading data into a flat file

Loading data from a flat file

Loading data from table to table – lookups and joins

Using the Map_Operation transform

Using the Table_Comparison transform

Exploring the Auto correct load option

Splitting the flow of data with the Case transform

Monitoring and analyzing dataflow execution

5. Workflow – Controlling Execution Order

5. Workflow – Controlling Execution Order

Introduction

Creating a workflow object

Nesting workflows to control the execution order

Using conditional and while loop objects to control the execution order

Using the bypassing feature

Controlling failures – try-catch objects

Use case example – populating dimension tables

6. Job – Building the ETL Architecture

6. Job – Building the ETL Architecture

Introduction

Projects and jobs – organizing ETL

Using object replication

Migrating ETL code through the central repository

Migrating ETL code with export/import

Debugging job execution

Monitoring job execution

Building an external ETL audit and audit reporting

Using built-in Data Services ETL audit and reporting functionality

Auto Documentation in Data Services

7. Validating and Cleansing Data

7. Validating and Cleansing Data

Introduction

Creating validation functions

Using validation functions with the Validation transform

Reporting data validation results

Using regular expression support to validate data

Enabling dataflow audit

Data Quality transforms – cleansing your data

8. Optimizing ETL Performance

8. Optimizing ETL Performance

Introduction

Optimizing dataflow execution – push-down techniques

Optimizing dataflow execution – the SQL transform

Optimizing dataflow execution – the Data_Transfer transform

Optimizing dataflow readers – lookup methods

Optimizing dataflow loaders – bulk-loading methods

Optimizing dataflow execution – performance options

9. Advanced Design Techniques

9. Advanced Design Techniques

Introduction

Change Data Capture techniques

Automatic job recovery in Data Services

Simplifying ETL execution with system configurations

Transforming data with the Pivot transform

10. Developing Real-time Jobs

10. Developing Real-time Jobs

Introduction

Working with nested structures

The XML_Map transform

The Hierarchy_Flattening transform

Configuring Access Server

Creating real-time jobs

11. Working with SAP Applications

Introduction

Loading data into SAP ERP

12. Introduction to Information Steward

12. Introduction to Information Steward

Introduction

Exploring Data Insight capabilities

Performing Metadata Management tasks

Working with the Metapedia functionality

Creating a custom cleansing package with Cleansing Package Builder

Index

Index

Customer Reviews

4 (1)

5 star

0%

4 star

100%

3 star

0%

2 star

0%

1 star

0%

Nesting workflows to control the execution order

In this recipe, we will see how workflow objects are executed in the nested structure.

Getting ready

We will not create dataflow objects in this recipe, so to prepare an environment, just create an empty job object.

How to do it

We will create a nested structure of a few workflow objects, each of which, when executed, will run the script. It will display the current workflow name and the full path to the root job context. Follow these steps:

In the job workspace, create a new workflow object, WF_root, and open it.
In the Variables and Parameters window, when in the WF_root context, create one local variable $l_wf_name and one input parameter $p_wf_parent_name, both of the varchar(255) data type.

Also, inside WF_root, add the new script object named Script with the following code:

$l_wf_name = workflow_name();
print('INFO: running {$l_wf_name} (parent={$p_wf_parent_name})');
$l_wf_name = $p_wf_parent_name || ' > ' || $l_wf_name;

In the same WF_root...

Visually different images

CONTINUE READING

83

Tech Concepts

36

Programming languages

73

Tech Tools

Unlimited access to the largest independent learning library in tech of over 8,000 expert-authored tech books and videos.

Innovative learning tools, including AI book assistants, code context explainers, and text-to-speech.

50+ new titles added per month and exclusive early access to books as they are being written.

SAP Data Services 4.x Cookbook

Search

Your notes and bookmarks