In 1998, when Sunopsis first opened its door to begin development of the product that would become ODI, the technology landscape was very different from today. There were very few data warehouses over 10 terabytes, and both the underlying hardware and software were struggling to keep up with the load and query demands placed upon them. As a result, the warehouses only included the most critical transactional data from the few key systems. The prevailing ETL approach of the day was to hand-build highly optimized flows, which executed in dedicated hardware, and then spoon-feed the results into the data warehouse. In this context, the founding principles of ODI were extraordinarily bold: leverage the data warehouse itself as the transformation engine and enable developers to work at a much higher level of abstraction, counting on the tool to generate an optimized execution plan.

Viewed with hindsight 15 years later, these principles seem prescient. Data warehouses have grown to be petabytes in size, and the hardware that houses them is often the most powerful in the data center — 100s of processor cores, terabytes of RAM, and 10s of terabytes of Flash. After many years of concerted effort, and with thousands of mappings to show for it, many enterprises have added far more transaction systems into their warehouses, and they are now looking to bring in data sets that have hitherto been dark (for example, server logs, social media feeds), and may be best preprocessed on open source distributed frameworks such as Hadoop. The world has finally caught up to ODI.

Congratulations on your decision to take a different approach to bulk movement and transformation of data within your business. For developers experienced on traditional ETL tools, you will discover yourself doing far less mundane work once you grasp a few of the key ODI concepts such as topologies, knowledge modules, and set-based transformation. This book will be an excellent companion for you on this journey. Written by four experts on the product (with decades of experience among them), including key product managers who are continuing to drive ODI's evolution, this book complements the product documentation with a variety of practical recipes. In addition to all of the common tasks required in populating a data warehouse (for example, change data capture, slowly changing dimensions), readers will also find valuable information on using ODI within a web service environment, and how to use its powerful APIs to programmatically author ODI artifacts.

Since its acquisition, ODI has become the key bulk data technology within Oracle products and within our cloud offerings. I hope you find it as impactful in your business as it has been in ours.

Brad Adelberg

Vice President, Development for Data Integration

Oracle Corp.