Book Image

SAP Data Services 4.x Cookbook

Book Image

SAP Data Services 4.x Cookbook

Overview of this book

Want to cost effectively deliver trusted information to all of your crucial business functions? SAP Data Services delivers one enterprise-class solution for data integration, data quality, data profiling, and text data processing. It boosts productivity with a single solution for data quality and data integration. SAP Data Services also enables you to move, improve, govern, and unlock big data. This book will lead you through the SAP Data Services environment to efficiently develop ETL processes. To begin with, you’ll learn to install, configure, and prepare the ETL development environment. You will get familiarized with the concepts of developing ETL processes with SAP Data Services. Starting from smallest unit of work- the data flow, the chapters will lead you to the highest organizational unit—the Data Services job, revealing the advanced techniques of ETL design. You will learn to import XML files by creating and implementing real-time jobs. It will then guide you through the ETL development patterns that enable the most effective performance when extracting, transforming, and loading data. You will also find out how to create validation functions and transforms. Finally, the book will show you the benefits of data quality management with the help of another SAP solution—Information Steward.

Related Content you might be interested in

Current Title:

Small book image
SAP Data Services 4.x Cookbook
Dec, 2015 | 430 pages
No titles found
Table of Contents (14 chapters)
Preface
Preface
What this book covers
What you need for this book
Who this book is for
Sections
Conventions
Reader feedback
Customer support
Free Chapter
1
1. Introduction to ETL Development
1. Introduction to ETL Development
Introduction
Preparing a database environment
Creating a source system database
Defining and creating staging area structures
Creating a target data warehouse
2
2. Configuring the Data Services Environment
2. Configuring the Data Services Environment
Introduction
Creating IPS and Data Services repositories
Installing and configuring Information Platform Services
Installing and configuring Data Services
Configuring user access
Starting and stopping services
Administering tasks
Understanding the Designer tool
3
3. Data Services Basics – Data Types, Scripting Language, and Functions
3. Data Services Basics – Data Types, Scripting Language, and Functions
Introduction
Creating variables and parameters
Creating a script
Using string functions
Using date functions
Using conversion functions
Using database functions
Using aggregate functions
Using math functions
Using miscellaneous functions
Creating custom functions
4
4. Dataflow – Extract, Transform, and Load
4. Dataflow – Extract, Transform, and Load
Introduction
Creating a source data object
Creating a target data object
Loading data into a flat file
Loading data from a flat file
Loading data from table to table – lookups and joins
Using the Map_Operation transform
Using the Table_Comparison transform
Exploring the Auto correct load option
Splitting the flow of data with the Case transform
Monitoring and analyzing dataflow execution
5
5. Workflow – Controlling Execution Order
5. Workflow – Controlling Execution Order
Introduction
Creating a workflow object
Nesting workflows to control the execution order
Using conditional and while loop objects to control the execution order
Using the bypassing feature
Controlling failures – try-catch objects
Use case example – populating dimension tables
6
6. Job – Building the ETL Architecture
6. Job – Building the ETL Architecture
Introduction
Projects and jobs – organizing ETL
Using object replication
Migrating ETL code through the central repository
Migrating ETL code with export/import
Debugging job execution
Monitoring job execution
Building an external ETL audit and audit reporting
Using built-in Data Services ETL audit and reporting functionality
Auto Documentation in Data Services
7
7. Validating and Cleansing Data
7. Validating and Cleansing Data
Introduction
Creating validation functions
Using validation functions with the Validation transform
Reporting data validation results
Using regular expression support to validate data
Enabling dataflow audit
Data Quality transforms – cleansing your data
8
8. Optimizing ETL Performance
8. Optimizing ETL Performance
Introduction
Optimizing dataflow execution – push-down techniques
Optimizing dataflow execution – the SQL transform
Optimizing dataflow execution – the Data_Transfer transform
Optimizing dataflow readers – lookup methods
Optimizing dataflow loaders – bulk-loading methods
Optimizing dataflow execution – performance options
9
9. Advanced Design Techniques
9. Advanced Design Techniques
Introduction
Change Data Capture techniques
Automatic job recovery in Data Services
Simplifying ETL execution with system configurations
Transforming data with the Pivot transform
10
10. Developing Real-time Jobs
10. Developing Real-time Jobs
Introduction
Working with nested structures
The XML_Map transform
The Hierarchy_Flattening transform
Configuring Access Server
Creating real-time jobs
11
11. Working with SAP Applications
Introduction
Loading data into SAP ERP
12
12. Introduction to Information Steward
12. Introduction to Information Steward
Introduction
Exploring Data Insight capabilities
Performing Metadata Management tasks
Working with the Metapedia functionality
Creating a custom cleansing package with Cleansing Package Builder
13
Index
Index

Loading data into SAP ERP

We will not discuss the topic of configuring the SAP system to communicate with the Data Services environment as it would require another few chapters on the subject and, most importantly, it is not the purpose of this book. All this information can be found in detailed SAP documentation available at http://help.sap.com.

We presume that you have exactly the same Data Services and staging environments configured and created in the previous chapters of this book and have also installed and configured the SAP ERP system, which can communicate with the Data Services job server.

In this recipe, we will go through the steps of loading information into the SAP ERP system by using Data Services. In one of our preparation processes, we will be generating data records for insertion right in the dataflow, when usually, you have the data ready to be loaded in the staging area extracted from another system.

We will also review the main SAP transactions involved in the process of...

Previous Section
End of Section 1
Next Section