Sign In Start Free Trial
Account

Add to playlist

Create a Playlist

Modal Close icon
You need to login to use this feature.
  • Book Overview & Buying Engineering Lakehouses with Open Table Formats
  • Table Of Contents Toc
Engineering Lakehouses with Open Table Formats

Engineering Lakehouses with Open Table Formats

By : Dipankar Mazumdar, Vinoth Govindarajan
close
close
Engineering Lakehouses with Open Table Formats

Engineering Lakehouses with Open Table Formats

By: Dipankar Mazumdar, Vinoth Govindarajan

Overview of this book

Engineering Lakehouses with Open Table Formats provides detailed insights into lakehouse concepts, and dives deep into the practical implementation of open table formats such as Apache Iceberg, Apache Hudi, and Delta Lake. You’ll explore the internals of a table format and learn in detail about the transactional capabilities of lakehouses. You’ll also get hands on with each table format with exercises using popular computing engines, such as Apache Spark, Flink, Trino, and Python-based tools. The book addresses advanced topics, including performance optimization techniques and interoperability among different formats, equipping you to build production-ready lakehouses. With step-by-step explanations, you’ll get to grips with the key components of lakehouse architecture and learn how to build, maintain, and optimize them. By the end of this book, you’ll be proficient in evaluating and implementing open table formats, optimizing lakehouse performance, and applying these concepts to real-world scenarios, ensuring you make informed decisions in selecting the right architecture for your organization’s data needs.
Table of Contents (15 chapters)
close
close
13
Other Books You May Enjoy
14
Index

Answers

  1. OLTP systems focus on real-time transactional processing, optimized for rapid data insertion, updates, and deletions with ACID compliance. OLAP systems target large-scale data analysis with high query performance, often using column-oriented file formats and denormalized schemas.
  2. Data lakes provide centralized storage for raw data in its native format, enabling scalability and cost-effectiveness. However, they face challenges such as lack of governance, slow query performance, and no native ACID support, which the lakehouse architecture addresses by combining the flexibility of data lakes with the reliability of data warehouses.
  3. Key attributes of a lakehouse architecture include open data architecture, unification of batch and streaming workloads, cost efficiency, reliable transactions, and multi-compute engine support.
  4. Table formats such as Apache Iceberg, Apache Hudi, and Delta Lake provide schema evolution, time travel, and ACID compliance, enabling better data management...
CONTINUE READING
83
Tech Concepts
36
Programming languages
73
Tech Tools
Icon Unlimited access to the largest independent learning library in tech of over 8,000 expert-authored tech books and videos.
Icon Innovative learning tools, including AI book assistants, code context explainers, and text-to-speech.
Icon 50+ new titles added per month and exclusive early access to books as they are being written.
Engineering Lakehouses with Open Table Formats
notes
bookmark Notes and Bookmarks search Search in title playlist Add to playlist download Download options font-size Font size

Change the font size

margin-width Margin width

Change margin width

day-mode Day/Sepia/Night Modes

Change background colour

Close icon Search
Country selected

Close icon Your notes and bookmarks

Confirmation

Modal Close icon
claim successful

Buy this book with your credits?

Modal Close icon
Are you sure you want to buy this book with one of your credits?
Close
YES, BUY

Submit Your Feedback

Modal Close icon
Modal Close icon
Modal Close icon