Oracle Data Integrator 11g Cookbook

Oracle Data Integrator 11g Cookbook

Overview of this book

Oracle Data Integrator (ODI) is Oracle's strategic data integration platform for high-speed data transformation and movement between different systems. From high-volume batches, to SOA-enabled data services, to trickle operations, ODI is a cutting-edge platform that offers heterogeneous connectivity, enterprise-level deployment, and strong administrative, diagnostic, and management capabilities."Oracle Data Integrator 11g Cookbook" will take you on a journey past your first steps with ODI to a new level of proficiency, lifting the cover on many of the internals of the product to help you better leverage the most advanced features.The first part of this book will focus on the administrative tasks required for a successful deployment, moving on to showing you how to best leverage Knowledge Modules with explanations of their internals and focus on specific examples. Next we will look into some advanced coding techniques for interfaces, packages, models, and a focus on XML. Finally the book will lift the cover on web services as well as the ODI SDK, along with additional advanced techniques that may be unknown to many users.Throughout "Oracle Data Integrator 11g Cookbook", the authors convey real-world advice and best practices learned from their extensive hands-on experience.

Oracle Data Integrator 11g Cookbook

Credits

Foreword

About the Authors

About the Reviewers

www.PacktPub.com

Preface

Free Chapter

Installation, Deployment, and Configuration

Introduction

Deploying and configuring a standalone agent manually

Deploying a JEE ODI Agent

Configuring a standalone agent with OPMN

Deploying JDBC drivers with a JEE ODI Agent

Defining the Oracle Data Integrator Security

Introduction

Setting up LDAP security for Oracle Data Integrator

Setting external authentication with Oracle Data Integrator

Creating users using generic profiles

Creating users using non-generic profiles

Creating new custom profiles in the Security navigator

Advanced Topology

Introduction

Creating a new technology

Modifying actions to get more from your diagrams

Modifying and expanding datatypes

Changing the case sensitivity for code generation

Best practice – using the Staging Area User to access your target schema

Using Variables

Introduction

Passing start-up parameters to a scenario using variables

Using table names that run in all contexts using getObjectName

Using variables to filter data based on a timestamp

Using variables in KM options (and reusing the variables in an interface, package, and so on)

Using variables in topology

Using variables to control loops inside packages

Knowledge Module Internals

Introduction

Using the substitution passes

Using Java variables in KMs

Using Java for condition code generation

Invoking Java from the KMs

Using substitution methods in Java

Combining substitution methods in a KM

Inside Knowledge Modules – SCD and CDC

Introduction

Implementing Slowly Changing Dimensions (SCD) using ODI

Modifying a Slowly Changing Dimension KM to allow undefined behaviors

Using Changed Data Capture (CDC) - simple

Using Changed Data Capture (CDC) - consistent set

Using one single interface to load changes that occur in any dimensions

Advanced Coding Techniques

Introduction

Using diagrams to develop and maintain models

Generating DDL from data models

Generating interfaces from data models or diagrams

Creating a temporary interface (subquery)

Loading data from an SQL query

Performing a pivot

Loading data using partition exchange

Package Loops and File Processing

Introduction

Defining packages and loops for near real-time execution using a hybrid loop

Using a file from a parameter variable

Detecting files with a variable name

Processing all files in a directory

Processing a large number of files in parallel

XML and Web Services

Introduction

Defining a connection to XML within ODI

Processing complex files with ODI

Processing XML data within an RDBMS not in memory

Invoking web services from ODI

Invoking asynchronous ODI web services with callbacks

Configuring container-based authentication with ODI web services

Advanced Coding Techniques Using the ODI SDK

Introduction

Creating the Master and Work repositories using the SDK

Creating a project using the SDK

Automating the import of artifacts using the SDK

Creating models and datastores using the SDK

Creating an interface using the SDK

Invoking and monitoring a scenario using the SDK

About the Authors

Christophe Dupupet is a Director in the Fusion Middleware Architects Team, where he leads the expertise on ODI. The team works closely with strategic customers that implement ODI, and helps define best practices on the product in terms of architecture, implementation, and operations.

Prior to Oracle, Christophe was part of the team that started the operations for Sunopsis in the US, where he lead the technical team (presales, support, and training). Sunopsis was acquired by Oracle in 2006.

Christophe holds an Operations Research degree from EISTI in France, a Masters Degree in Operations Research from Florida Tech, and a certificate in Management from Harvard University.

Christophe is a co-author of the book Getting Started with Oracle Data Integrator 11g: A Hands-on Tutorial.

There would be no book if we did not have a fantastic product and customers to trust us with this product. I want to particularly thank all the individuals that have helped this product become what it is today: the architects and software engineers that work and have worked on ODI for their vision and production; our support engineers that help our customers every day and help shape best practices with a forever growing knowledge base (support.oracle.com is truly a goldmine if you are looking for information on how to use ODI); our sales engineers and product managers that help customers and partners in their selection of our product.

Peter C. Boyd-Bowman is a Technical Director and Consultant with the Oracle Corporation. He has over 30 years of software engineering and database management experience, including 12 years of focused interest in data warehousing and business intelligence. Capitalizing on his extensive background in Oracle database technologies dating back to 1985, he has spent recent years specializing in data migration. After many successful project implementations using Oracle Warehouse Builder, and shortly after Oracle's acquisition of the Sunopsis Corporation, he switched his area of focus over to Oracle's flagship ETL product: Oracle Data Integrator. Peter holds a BS degree in Industrial Management and Computer Science from Purdue University and currently resides in North Carolina.

Denis Gray is a Director of Product Management for Data Integration at Oracle. Denis has over 15 years of experience in the data-integration field. For the past seven years, Denis has been an integral part of Oracle Development Organization as a Product Manager within Fusion Middleware, delivering data integration solutions. Prior to this, Denis was a data integration consult for Hyperion Solutions (Oracle). Here, Denis worked at many of the largest Fortune 100 companies, building data warehouses and implementing business intelligence solutions. Denis has a Bachelor's Degree in Computer Science from the University of Missouri and currently resides in St. Louis, MO. Denis also co-authored the Packt book Getting Started with Oracle Data Integrator 11g: A Hands-on Tutorial.

Julien Testut is a Product Manager in the Oracle Data Integration group focusing on Oracle Data Integrator. Julien has an extensive background in Data Integration and Data Quality solutions and is a co-author of Getting Started with Oracle Data Integrator 11g: A Hands-on Tutorial. Prior to joining Oracle, he was an Applications Engineer at Sunopsis, which was then acquired by Oracle. Julien holds a Masters degree in Software Engineering.

To my daughter Olivia, who was born while I was working on this book and who has been filling my life with joy since then. Thanks to my wife Emilie for her help and patience throughout this book's writing process. I couldn't have done it without you. I would also like to take this opportunity to express my gratitude to my co-authors. It's always a pleasure to collaborate with you! Finally, I would like to thank my family and friends for their support as well as my colleagues who help make Oracle Data Integrator a better product with every release. Thank you!

Oracle Data Integrator 11g Cookbook

Oracle Data Integrator 11g Cookbook

Overview of this book

Related Content you might be interested in

Current Title:

Oracle Data Integrator 11g Cookbook

About the Authors