-
Book Overview & Buying
-
Table Of Contents
Engineering Lakehouses with Open Table Formats
By :
In Chapter 1, Open Data Lakehouse – a New Architectural Paradigm, we introduced the storage engine as the critical component responsible for managing how data is stored, retrieved, and maintained in a data lakehouse architecture. In this chapter, we will deepen that understanding by exploring how the storage engine integrates with the open table format to provide transactional capabilities. Before diving into the specific capabilities of a storage engine, let’s first examine the various sub-components that make up the storage engine within a traditional database system. This will help clarify why the storage engine is so essential in an open lakehouse architecture. The following diagram illustrates the architecture of a database system, with the storage engine component highlighted (at the bottom):

In traditional DBMS, the storage engine is the core layer responsible...