Sign In Start Free Trial
Account

Add to playlist

Create a Playlist

Modal Close icon
You need to login to use this feature.
  • Book Overview & Buying In-Memory Analytics with Apache Arrow
  • Table Of Contents Toc
In-Memory Analytics with Apache Arrow

In-Memory Analytics with Apache Arrow - Second Edition

By : Matthew Topol
5 (7)
close
close
In-Memory Analytics with Apache Arrow

In-Memory Analytics with Apache Arrow

5 (7)
By: Matthew Topol

Overview of this book

Apache Arrow is an open source, columnar in-memory data format designed for efficient data processing and analytics. This book harnesses the author’s 15 years of experience to show you a standardized way to work with tabular data across various programming languages and environments, enabling high-performance data processing and exchange. This updated second edition gives you an overview of the Arrow format, highlighting its versatility and benefits through real-world use cases. It guides you through enhancing data science workflows, optimizing performance with Apache Parquet and Spark, and ensuring seamless data translation. You’ll explore data interchange and storage formats, and Arrow's relationships with Parquet, Protocol Buffers, FlatBuffers, JSON, and CSV. You’ll also discover Apache Arrow subprojects, including Flight, SQL, Database Connectivity, and nanoarrow. You’ll learn to streamline machine learning workflows, use Arrow Dataset APIs, and integrate with popular analytical data systems such as Snowflake, Dremio, and DuckDB. The latter chapters provide real-world examples and case studies of products powered by Apache Arrow, providing practical insights into its applications. By the end of this book, you’ll have all the building blocks to create efficient and powerful analytical services and utilities with Apache Arrow.
Table of Contents (18 chapters)
close
close
Lock Free Chapter
1
Part 1: Overview of What Arrow is, Its Capabilities, Benefits, and Goals
5
Part 2: Interoperability with Arrow: The Power of Open Standards
12
Part 3: Real-World Examples, Use Cases, and Future Development

Summary

By composing these various pieces together (the C Data API, Compute API, Datasets API, and Acero), and gluing infrastructure on top, anyone should be able to create a rudimentary query and analysis engine that is fairly performant right away. The functionality provided allows for abstracting away a lot of the tedious work of interacting with different file formats and handling different location sources of data to provide a single interface that allows you to get right to work in building the specific logic you need. Once again, it’s the fact that all these things are built on top of Arrow as an underlying format, which is particularly efficient for these operations, that allows them to all be so easily interoperable. Not only that, but because of the standardization of Arrow, individual pieces of that stack could be swapped out and composed with other, pre-existing projects, such as DuckDB or Apache DataFusion, both of which can use the Arrow C Data API to communicate...

CONTINUE READING
83
Tech Concepts
36
Programming languages
73
Tech Tools
Icon Unlimited access to the largest independent learning library in tech of over 8,000 expert-authored tech books and videos.
Icon Innovative learning tools, including AI book assistants, code context explainers, and text-to-speech.
Icon 50+ new titles added per month and exclusive early access to books as they are being written.
In-Memory Analytics with Apache Arrow
notes
bookmark Notes and Bookmarks search Search in title playlist Add to playlist download Download options font-size Font size

Change the font size

margin-width Margin width

Change margin width

day-mode Day/Sepia/Night Modes

Change background colour

Close icon Search
Country selected

Close icon Your notes and bookmarks

Confirmation

Modal Close icon
claim successful

Buy this book with your credits?

Modal Close icon
Are you sure you want to buy this book with one of your credits?
Close
YES, BUY

Submit Your Feedback

Modal Close icon
Modal Close icon
Modal Close icon