In this recipe, we will see how you can use regular expressions to validate your data. We will take a simple example of validating phone numbers extracted from the source OLTP table PERSONPHONE
located in the PERSON
schema. The validation rule would be to identify all records which have phone numbers different from this pattern: ddd-ddd-dddd (d being a numeral). Let's say that we do not want to reject any data. Our goal is to generate a dashboard report showing the percentage of records in the source table which do not comply with the specified requirement for the phone number pattern.
SAP Data Services 4.x Cookbook
SAP Data Services 4.x Cookbook
Overview of this book
Want to cost effectively deliver trusted information to all of your crucial business functions? SAP Data Services delivers one enterprise-class solution for data integration, data quality, data profiling, and text data processing. It boosts productivity with a single solution for data quality and data integration. SAP Data Services also enables you to move, improve, govern, and unlock big data.
This book will lead you through the SAP Data Services environment to efficiently develop ETL processes. To begin with, you’ll learn to install, configure, and prepare the ETL development environment. You will get familiarized with the concepts of developing ETL processes with SAP Data Services. Starting from smallest unit of work- the data flow, the chapters will lead you to the highest organizational unit—the Data Services job, revealing the advanced techniques of ETL design.
You will learn to import XML files by creating and implementing real-time jobs. It will then guide you through the ETL development patterns that enable the most effective performance when extracting, transforming, and loading data. You will also find out how to create validation functions and transforms.
Finally, the book will show you the benefits of data quality management with the help of another SAP solution—Information Steward.
Table of Contents (19 chapters)
SAP Data Services 4.x Cookbook
Credits
About the Author
About the Reviewers
www.PacktPub.com
Preface
Free Chapter
Introduction to ETL Development
Configuring the Data Services Environment
Data Services Basics – Data Types, Scripting Language, and Functions
Dataflow – Extract, Transform, and Load
Workflow – Controlling Execution Order
Job – Building the ETL Architecture
Validating and Cleansing Data
Optimizing ETL Performance
Advanced Design Techniques
Developing Real-time Jobs
Working with SAP Applications
Introduction to Information Steward
Index
Customer Reviews