Sign In Start Free Trial
Account

Add to playlist

Create a Playlist

Modal Close icon
You need to login to use this feature.
  • Book Overview & Buying Engineering Lakehouses with Open Table Formats
  • Table Of Contents Toc
Engineering Lakehouses with Open Table Formats

Engineering Lakehouses with Open Table Formats

By : Dipankar Mazumdar, Vinoth Govindarajan
close
close
Engineering Lakehouses with Open Table Formats

Engineering Lakehouses with Open Table Formats

By: Dipankar Mazumdar, Vinoth Govindarajan

Overview of this book

Engineering Lakehouses with Open Table Formats provides detailed insights into lakehouse concepts, and dives deep into the practical implementation of open table formats such as Apache Iceberg, Apache Hudi, and Delta Lake. You’ll explore the internals of a table format and learn in detail about the transactional capabilities of lakehouses. You’ll also get hands on with each table format with exercises using popular computing engines, such as Apache Spark, Flink, Trino, and Python-based tools. The book addresses advanced topics, including performance optimization techniques and interoperability among different formats, equipping you to build production-ready lakehouses. With step-by-step explanations, you’ll get to grips with the key components of lakehouse architecture and learn how to build, maintain, and optimize them. By the end of this book, you’ll be proficient in evaluating and implementing open table formats, optimizing lakehouse performance, and applying these concepts to real-world scenarios, ensuring you make informed decisions in selecting the right architecture for your organization’s data needs.
Table of Contents (15 chapters)
close
close
13
Other Books You May Enjoy
14
Index

Evaluating and Selecting Open Table Formats

While Apache Iceberg, Apache Hudi, and Delta Lake share foundational capabilities such as ACID transactions, schema evolution, and time travel, their architectural designs and optimization strategies cater to distinct use cases. Selecting the right table format is a foundational decision in building a scalable and efficient lakehouse architecture. This choice extends far beyond storage; it dictates how data is ingested, updated, and queried, while also shaping the broader ecosystem of compute engines, metadata catalogs, and downstream applications such as BI tools and ML platforms.

This chapter should provide a systematic approach to evaluating and selecting the most appropriate table format depending on workload type and size. It will bring together some of the core strengths of Iceberg, Hudi, and Delta Lake (discussed in Chapters 3, 4, and 5), how they integrate with different ecosystems, and real-world adoption patterns that illustrate...

Visually different images
CONTINUE READING
83
Tech Concepts
36
Programming languages
73
Tech Tools
Icon Unlimited access to the largest independent learning library in tech of over 8,000 expert-authored tech books and videos.
Icon Innovative learning tools, including AI book assistants, code context explainers, and text-to-speech.
Icon 50+ new titles added per month and exclusive early access to books as they are being written.
Engineering Lakehouses with Open Table Formats
notes
bookmark Notes and Bookmarks search Search in title playlist Add to playlist download Download options font-size Font size

Change the font size

margin-width Margin width

Change margin width

day-mode Day/Sepia/Night Modes

Change background colour

Close icon Search
Country selected

Close icon Your notes and bookmarks

Confirmation

Modal Close icon
claim successful

Buy this book with your credits?

Modal Close icon
Are you sure you want to buy this book with one of your credits?
Close
YES, BUY

Submit Your Feedback

Modal Close icon
Modal Close icon
Modal Close icon