Book Image

SAP Data Services 4.x Cookbook

Book Image

SAP Data Services 4.x Cookbook

Overview of this book

Want to cost effectively deliver trusted information to all of your crucial business functions? SAP Data Services delivers one enterprise-class solution for data integration, data quality, data profiling, and text data processing. It boosts productivity with a single solution for data quality and data integration. SAP Data Services also enables you to move, improve, govern, and unlock big data. This book will lead you through the SAP Data Services environment to efficiently develop ETL processes. To begin with, you’ll learn to install, configure, and prepare the ETL development environment. You will get familiarized with the concepts of developing ETL processes with SAP Data Services. Starting from smallest unit of work- the data flow, the chapters will lead you to the highest organizational unit—the Data Services job, revealing the advanced techniques of ETL design. You will learn to import XML files by creating and implementing real-time jobs. It will then guide you through the ETL development patterns that enable the most effective performance when extracting, transforming, and loading data. You will also find out how to create validation functions and transforms. Finally, the book will show you the benefits of data quality management with the help of another SAP solution—Information Steward.
Table of Contents (19 chapters)
SAP Data Services 4.x Cookbook
Credits
About the Author
About the Reviewers
www.PacktPub.com
Preface
Index

Optimizing dataflow readers – lookup methods


There are different ways in which to perform the lookup of a record from another table in Data Services. The three most popular ones are: a table join with a Query transform, using the lookup_ext() function, and using the sql() function.

In this recipe, we will take a look at all these methods and discuss how they affect the performance of ETL code execution and their impact on a database used to source data from.

Getting ready

We will be using the same dataflow as in the first recipe, the one which populates the PERSON_DETAILS stage table from multiple OLTP tables.

How to do it…

We will perform a lookup for the PHONENUMBER column of a person from the OLTP table PERSONPHONE in three different ways.

Lookup with the Query transform join

  1. Import the lookup table into a datastore and add the table object as a source in the dataflow where you need to perform the lookup.

  2. Use the Query transform to join your main dataset with the lookup table using the BUSINESSENTITYID...