Sign In Start Free Trial
Account

Add to playlist

Create a Playlist

Modal Close icon
You need to login to use this feature.
  • Book Overview & Buying Engineering Lakehouses with Open Table Formats
  • Table Of Contents Toc
Engineering Lakehouses with Open Table Formats

Engineering Lakehouses with Open Table Formats

By : Dipankar Mazumdar, Vinoth Govindarajan
close
close
Engineering Lakehouses with Open Table Formats

Engineering Lakehouses with Open Table Formats

By: Dipankar Mazumdar, Vinoth Govindarajan

Overview of this book

Engineering Lakehouses with Open Table Formats provides detailed insights into lakehouse concepts, and dives deep into the practical implementation of open table formats such as Apache Iceberg, Apache Hudi, and Delta Lake. You’ll explore the internals of a table format and learn in detail about the transactional capabilities of lakehouses. You’ll also get hands on with each table format with exercises using popular computing engines, such as Apache Spark, Flink, Trino, and Python-based tools. The book addresses advanced topics, including performance optimization techniques and interoperability among different formats, equipping you to build production-ready lakehouses. With step-by-step explanations, you’ll get to grips with the key components of lakehouse architecture and learn how to build, maintain, and optimize them. By the end of this book, you’ll be proficient in evaluating and implementing open table formats, optimizing lakehouse performance, and applying these concepts to real-world scenarios, ensuring you make informed decisions in selecting the right architecture for your organization’s data needs.
Table of Contents (15 chapters)
close
close
13
Other Books You May Enjoy
14
Index

Table evolution and versioning

Table evolution and versioning are critical aspects of lakehouse architecture, influencing how data evolves and how historical versions are retained. A table format’s ability to support flexible schema and partition evolution, and efficient versioning directly impacts data consistency and long-term maintainability. When selecting a table format, organizations must evaluate the following:

  • How seamlessly schema changes can be applied without requiring costly table rewrites
  • How efficiently past table versions can be accessed for rollback, auditing, or time travel
  • Ways to evolve partitions without a full data rewrite

Effectively managing table evolution and versioning ensures that data remains consistent, auditable, and easy to maintain over time. Choosing a format that handles these aspects efficiently lays the foundation for a resilient and future-proof lakehouse architecture.

Schema and partition evolution

...
CONTINUE READING
83
Tech Concepts
36
Programming languages
73
Tech Tools
Icon Unlimited access to the largest independent learning library in tech of over 8,000 expert-authored tech books and videos.
Icon Innovative learning tools, including AI book assistants, code context explainers, and text-to-speech.
Icon 50+ new titles added per month and exclusive early access to books as they are being written.
Engineering Lakehouses with Open Table Formats
notes
bookmark Notes and Bookmarks search Search in title playlist Add to playlist download Download options font-size Font size

Change the font size

margin-width Margin width

Change margin width

day-mode Day/Sepia/Night Modes

Change background colour

Close icon Search
Country selected

Close icon Your notes and bookmarks

Confirmation

Modal Close icon
claim successful

Buy this book with your credits?

Modal Close icon
Are you sure you want to buy this book with one of your credits?
Close
YES, BUY

Submit Your Feedback

Modal Close icon
Modal Close icon
Modal Close icon