Book Image

SAP Data Services 4.x Cookbook

Book Image

SAP Data Services 4.x Cookbook

Overview of this book

Want to cost effectively deliver trusted information to all of your crucial business functions? SAP Data Services delivers one enterprise-class solution for data integration, data quality, data profiling, and text data processing. It boosts productivity with a single solution for data quality and data integration. SAP Data Services also enables you to move, improve, govern, and unlock big data. This book will lead you through the SAP Data Services environment to efficiently develop ETL processes. To begin with, you’ll learn to install, configure, and prepare the ETL development environment. You will get familiarized with the concepts of developing ETL processes with SAP Data Services. Starting from smallest unit of work- the data flow, the chapters will lead you to the highest organizational unit—the Data Services job, revealing the advanced techniques of ETL design. You will learn to import XML files by creating and implementing real-time jobs. It will then guide you through the ETL development patterns that enable the most effective performance when extracting, transforming, and loading data. You will also find out how to create validation functions and transforms. Finally, the book will show you the benefits of data quality management with the help of another SAP solution—Information Steward.

Related Content you might be interested in

Current Title:

Small book image
SAP Data Services 4.x Cookbook
Dec, 2015 | 430 pages
No titles found
Table of Contents (19 chapters)
SAP Data Services 4.x Cookbook
SAP Data Services 4.x Cookbook
Credits
Credits
About the Author
About the Author
About the Reviewers
About the Reviewers
www.PacktPub.com
www.PacktPub.com
Preface
Preface
Free Chapter
1
Introduction to ETL Development
Introduction to ETL Development
Introduction
Preparing a database environment
Creating a source system database
Defining and creating staging area structures
Creating a target data warehouse
2
Configuring the Data Services Environment
Configuring the Data Services Environment
Introduction
Creating IPS and Data Services repositories
Installing and configuring Information Platform Services
Installing and configuring Data Services
Configuring user access
Starting and stopping services
Administering tasks
Understanding the Designer tool
3
Data Services Basics – Data Types, Scripting Language, and Functions
Data Services Basics – Data Types, Scripting Language, and Functions
Introduction
Creating variables and parameters
Creating a script
Using string functions
Using date functions
Using conversion functions
Using database functions
Using aggregate functions
Using math functions
Using miscellaneous functions
Creating custom functions
4
Dataflow – Extract, Transform, and Load
Dataflow – Extract, Transform, and Load
Introduction
Creating a source data object
Creating a target data object
Loading data into a flat file
Loading data from a flat file
Loading data from table to table – lookups and joins
Using the Map_Operation transform
Using the Table_Comparison transform
Exploring the Auto correct load option
Splitting the flow of data with the Case transform
Monitoring and analyzing dataflow execution
5
Workflow – Controlling Execution Order
Workflow – Controlling Execution Order
Introduction
Creating a workflow object
Nesting workflows to control the execution order
Using conditional and while loop objects to control the execution order
Using the bypassing feature
Controlling failures – try-catch objects
Use case example – populating dimension tables
6
Job – Building the ETL Architecture
Job – Building the ETL Architecture
Introduction
Projects and jobs – organizing ETL
Using object replication
Migrating ETL code through the central repository
Migrating ETL code with export/import
Debugging job execution
Monitoring job execution
Building an external ETL audit and audit reporting
Using built-in Data Services ETL audit and reporting functionality
Auto Documentation in Data Services
7
Validating and Cleansing Data
Validating and Cleansing Data
Introduction
Creating validation functions
Using validation functions with the Validation transform
Reporting data validation results
Using regular expression support to validate data
Enabling dataflow audit
Data Quality transforms – cleansing your data
8
Optimizing ETL Performance
Optimizing ETL Performance
Introduction
Optimizing dataflow execution – push-down techniques
Optimizing dataflow execution – the SQL transform
Optimizing dataflow execution – the Data_Transfer transform
Optimizing dataflow readers – lookup methods
Optimizing dataflow loaders – bulk-loading methods
Optimizing dataflow execution – performance options
9
Advanced Design Techniques
Advanced Design Techniques
Introduction
Change Data Capture techniques
Automatic job recovery in Data Services
Simplifying ETL execution with system configurations
Transforming data with the Pivot transform
10
Developing Real-time Jobs
Developing Real-time Jobs
Introduction
Working with nested structures
The XML_Map transform
The Hierarchy_Flattening transform
Configuring Access Server
Creating real-time jobs
11
Working with SAP Applications
Working with SAP Applications
Introduction
Loading data into SAP ERP
12
Introduction to Information Steward
Introduction to Information Steward
Introduction
Exploring Data Insight capabilities
Performing Metadata Management tasks
Working with the Metapedia functionality
Creating a custom cleansing package with Cleansing Package Builder
Index
Index

About the Reviewers

Andrés Aguado Aranda is a 26-year-old computer engineer from Spain. His experience has given him a really technical background in databases, data warehouse, and business intelligence.

Andrés has worked in different business sectors, such as banking, public administrations, and energy, since 2012 in data-related positions.

This book is my first stint as a reviewer, and it has been really interesting and valuable to me, both personally and professionally.

I would like to thank my family and friends for always being willing to help me when I needed. Also, I would like to thank to my former coworker and currently friend, Antonio Martín-Cobos, a BI reporting analyst who really helped me get this opportunity.

Dick Groenhof started his professional career in 1990 after finishing his studies in business information science at Vrije Universiteit Amsterdam. Having worked as a software developer and service management consultant for the first part of his career, he became active as a consultant in the business intelligence arena since 2005.

Dick has been a lead consultant on numerous SAP BI projects, designing and implementing successful solutions for his customers, who regard him as a trusted advisor. His core competences include both frontend (such as Web Intelligence, Crystal Reports, and SAP Design Studio) and backend tools (such as SAP Data Services and Information Steward). Dick is an early adopter of the SAP HANA platform, creating innovative solutions using HANA Information Views, Predictive Analysis Library, and SQLScript.

He is a Certified Application Associate in SAP HANA and SAP BusinessObjects Web Intelligence 4.1. Currently, Dick works as senior HANA and big data consultant for a highly respected and innovative SAP partner in the Netherlands.

He is a strong believer in sharing his knowledge with regard to SAP HANA and SAP Data Services by writing blogs (at http://www.dickgroenhof.com and http://www.thenextview.nl/blog) and speaking at seminars.

Dick is happily married to Emma and is a very proud father of his son, Christiaan, and daughter, Myrthe.

Bernard Timbal Duclaux de Martin is a business intelligence architect and technical expert with more than 15 years of experience. He has been involved in several large business intelligence system deployments and administration in banking and insurance companies. In addition, Bernard has skills in modeling, data extraction, transformation, loading, and reporting design. He has authored four books, including two regarding SAP BusinessObjects Enterprise administration.

Meenakshi Verma has been a part of the IT industry since 1998. She is an experienced business systems specialist having the CBAP and TOGAF certifications. Meenakshi is well-versed with a variety of tools and techniques used for business analysis, such as SAP BI, SAP BusinessObjects, Java/J2EE technologies, and others. She is currently based in Toronto, Canada, and works with a leading utility company.

Meenakshi has helped technically review many books published by Packt Publishing across various enterprise solutions. Her earlier works include JasperReports for Java Developers, Java EE 5 Development using GlassFish Application Server, Practical Data Analysis and Reporting with BIRT, EJB 3 Developer Guide, Learning Dojo, and IBM WebSphere Application Server 8.0 Administration Guide.

I'd like to thank my father, Mr. Bhopal Singh, and mother, Mrs. Raj Bala, for laying a strong foundation in me and giving me their unconditional love and support. I also owe thanks and gratitude to my husband, Atul Verma, for his encouragement and support throughout the reviewing of this book and many others; my ten-year-old son, Prieyaansh Verma, for giving me the warmth of his love despite my hectic schedules; and my brother, Sachin Singh, for always being there for me.

Previous Section
Next Chapter