Browse Library
Sign In Start Free Trial
Close icon Search icon
Account
Browse Library Sign In Start Free Trial

Add to playlist

Create a Playlist

Modal Close icon
You need to login to use this feature.
  • Book Overview & Buying SAP Data Services 4.x Cookbook
  • Table Of Contents Toc
SAP Data Services 4.x Cookbook

SAP Data Services 4.x Cookbook

By : Shomnikov
4 (1)
Buy this Book
close
close
SAP Data Services 4.x Cookbook

SAP Data Services 4.x Cookbook

4 (1)
By: Shomnikov
Buy this Book

Overview of this book

Want to cost effectively deliver trusted information to all of your crucial business functions? SAP Data Services delivers one enterprise-class solution for data integration, data quality, data profiling, and text data processing. It boosts productivity with a single solution for data quality and data integration. SAP Data Services also enables you to move, improve, govern, and unlock big data. This book will lead you through the SAP Data Services environment to efficiently develop ETL processes. To begin with, you’ll learn to install, configure, and prepare the ETL development environment. You will get familiarized with the concepts of developing ETL processes with SAP Data Services. Starting from smallest unit of work- the data flow, the chapters will lead you to the highest organizational unit—the Data Services job, revealing the advanced techniques of ETL design. You will learn to import XML files by creating and implementing real-time jobs. It will then guide you through the ETL development patterns that enable the most effective performance when extracting, transforming, and loading data. You will also find out how to create validation functions and transforms. Finally, the book will show you the benefits of data quality management with the help of another SAP solution—Information Steward.
Table of Contents (14 chapters)
close
close
close
Preface
Icon
Preface
Icon
What this book covers
Icon
What you need for this book
Icon
Who this book is for
Icon
Sections
Icon
Conventions
Icon
Reader feedback
Icon
Customer support
Lock Free Chapter
1
1. Introduction to ETL Development
chevron up
Icon
1. Introduction to ETL Development
Icon
Introduction
Icon
Preparing a database environment
Icon
Creating a source system database
Icon
Defining and creating staging area structures
Icon
Creating a target data warehouse
2
2. Configuring the Data Services Environment
Icon
2. Configuring the Data Services Environment
Icon
Introduction
Icon
Creating IPS and Data Services repositories
Icon
Installing and configuring Information Platform Services
Icon
Installing and configuring Data Services
Icon
Configuring user access
Icon
Starting and stopping services
Icon
Administering tasks
Icon
Understanding the Designer tool
3
3. Data Services Basics – Data Types, Scripting Language, and Functions
Icon
3. Data Services Basics – Data Types, Scripting Language, and Functions
Icon
Introduction
Icon
Creating variables and parameters
Icon
Creating a script
Icon
Using string functions
Icon
Using date functions
Icon
Using conversion functions
Icon
Using database functions
Icon
Using aggregate functions
Icon
Using math functions
Icon
Using miscellaneous functions
Icon
Creating custom functions
4
4. Dataflow – Extract, Transform, and Load
Icon
4. Dataflow – Extract, Transform, and Load
Icon
Introduction
Icon
Creating a source data object
Icon
Creating a target data object
Icon
Loading data into a flat file
Icon
Loading data from a flat file
Icon
Loading data from table to table – lookups and joins
Icon
Using the Map_Operation transform
Icon
Using the Table_Comparison transform
Icon
Exploring the Auto correct load option
Icon
Splitting the flow of data with the Case transform
Icon
Monitoring and analyzing dataflow execution
5
5. Workflow – Controlling Execution Order
Icon
5. Workflow – Controlling Execution Order
Icon
Introduction
Icon
Creating a workflow object
Icon
Nesting workflows to control the execution order
Icon
Using conditional and while loop objects to control the execution order
Icon
Using the bypassing feature
Icon
Controlling failures – try-catch objects
Icon
Use case example – populating dimension tables
6
6. Job – Building the ETL Architecture
Icon
6. Job – Building the ETL Architecture
Icon
Introduction
Icon
Projects and jobs – organizing ETL
Icon
Using object replication
Icon
Migrating ETL code through the central repository
Icon
Migrating ETL code with export/import
Icon
Debugging job execution
Icon
Monitoring job execution
Icon
Building an external ETL audit and audit reporting
Icon
Using built-in Data Services ETL audit and reporting functionality
Icon
Auto Documentation in Data Services
7
7. Validating and Cleansing Data
Icon
7. Validating and Cleansing Data
Icon
Introduction
Icon
Creating validation functions
Icon
Using validation functions with the Validation transform
Icon
Reporting data validation results
Icon
Using regular expression support to validate data
Icon
Enabling dataflow audit
Icon
Data Quality transforms – cleansing your data
8
8. Optimizing ETL Performance
Icon
8. Optimizing ETL Performance
Icon
Introduction
Icon
Optimizing dataflow execution – push-down techniques
Icon
Optimizing dataflow execution – the SQL transform
Icon
Optimizing dataflow execution – the Data_Transfer transform
Icon
Optimizing dataflow readers – lookup methods
Icon
Optimizing dataflow loaders – bulk-loading methods
Icon
Optimizing dataflow execution – performance options
9
9. Advanced Design Techniques
Icon
9. Advanced Design Techniques
Icon
Introduction
Icon
Change Data Capture techniques
Icon
Automatic job recovery in Data Services
Icon
Simplifying ETL execution with system configurations
Icon
Transforming data with the Pivot transform
10
10. Developing Real-time Jobs
Icon
10. Developing Real-time Jobs
Icon
Introduction
Icon
Working with nested structures
Icon
The XML_Map transform
Icon
The Hierarchy_Flattening transform
Icon
Configuring Access Server
Icon
Creating real-time jobs
11
11. Working with SAP Applications
Icon
Introduction
Icon
Loading data into SAP ERP
12
12. Introduction to Information Steward
Icon
12. Introduction to Information Steward
Icon
Introduction
Icon
Exploring Data Insight capabilities
Icon
Performing Metadata Management tasks
Icon
Working with the Metapedia functionality
Icon
Creating a custom cleansing package with Cleansing Package Builder
13
Index
Icon
Index

Creating a target data warehouse

Finally, this is the time to create our target data warehouse system. The data warehouse structures and tables will be used by end users with the help of various reporting tools to make sense of the data and analyze it. As a result, it should help business users to make strategic decisions, which will hopefully lead to business growth.

We should not forget that the main purpose of a data warehouse, and hence that of our ETL system, is to serve business needs.

Getting ready

The data warehouse created in this recipe will be used as a target database populated by the ETL processes developed in SAP Data Services. This is where the data modified and cleansed by ETL processes will be inserted in the end. Plus, this is the database that will mainly be accessed by business users and reporting tools.

How to do it…

Perform the following steps:

  1. AdventureWorks comes to the rescue again. Use another link to download the AdventureWorks data warehouse data file, which will be mapped in the same manner to our SQL Server Express database engine in order to create a local data warehouse for our own learning purposes. Go to the following URL and click on the AdventureWorksDW for SQL Server 2012 link:

    https://msftdbprodsamples.codeplex.com/releases/view/105902

  2. After you have successfully downloaded the AdventureWorksDW2012.zip file, unpack its contents into the same directory as the previous file:

    C:\AdventureWorks\

  3. There should be two files in the archive:
    • AdventureWorksDW2012_Data.mdf—the database data file
    • AdventureWorksDW2012_Log.ldf—the database transaction log file
  4. Open SQL Server Management Studio and click on the New Query… button in the uppermost tool bar.
  5. Enter and execute the following command in the SQL Query window:
    CREATE DATABASE AdventureWorks_DWH ON 
(FILENAME = 'C:\AdventureWorks\AdventureWorksDW2012_Data.mdf'), (FILENAME = 'C:\AdventureWorks\AdventureWorksDW2012_Log.ldf') FOR ATTACH;
  6. After a successful command execution, right-click on the Databases icon and choose the Refresh option in the opened menu list. This should refresh the contents of your object library, and you should see the following list of databases:
    • ODS
    • STAGE
    • AdventureWorks_OLTP
    • AdventureWorks_DWH

How it works…

Get yourself familiar with the tables of the created data warehouse. Throughout the whole book, you will be using them in order to insert, update, and delete data using Data Services.

There are also some diagrams available that could help you see the visual data warehouse structure. To get access to them, open SQL Server Management Studio, expand the Databases list in the Object Explorer window, then expand the AdventureWorks_DWH database object list, and finally open the Diagrams tree. Double-clicking on any diagram in the list opens a new window within Management Studio with the graphical presentation of tables, key columns, and links between the tables, which shows you the relationships between them.

There's more…

In the next recipe, we will have an overview of the knowledge resources that exist on the Web. We highly recommend that you get familiar with them in order to improve your data warehousing skills, learn about the data warehouse life cycle, and understand what makes a successful data warehouse project. In the meantime, feel free to open New Query in SQL Server Management Studio and start running the SELECT commands to explore the contents of the tables in your AdventureWorks_DWH database.

Note

The most important asset of any DWH architect or ETL developer is not the knowledge of a programming language or the available tools but the ability to understand the data that is, or will be, populating the data warehouse and the business needs and requirements for this data.

CONTINUE READING
83
Tech Concepts
36
Programming languages
73
Tech Tools
Icon Unlimited access to the largest independent learning library in tech of over 8,000 expert-authored tech books and videos.
Icon Innovative learning tools, including AI book assistants, code context explainers, and text-to-speech.
Icon 50+ new titles added per month and exclusive early access to books as they are being written.
SAP Data Services 4.x Cookbook
End of Chapter 1
Next Chapter
notes
bookmark Notes and Bookmarks search Search in title playlist Add to playlist download Download options
font-size Font size

Change the font size

margin-width Margin width

Change margin width

day-mode Day/Sepia/Night Modes

Change background colour

Close icon Search
Country selected
Close icon Your notes and bookmarks

Confirmation

Modal Close icon
claim successful
Order Page Read Now

Buy this book with your credits?

Modal Close icon
Are you sure you want to buy this book with one of your credits?
Close
YES, BUY

Submit Your Feedback

Modal Close icon
Modal Close icon
Modal Close icon