Book Image

Protocol Buffers Handbook

By : Clément Jean
Book Image

Protocol Buffers Handbook

By: Clément Jean

Overview of this book

Explore how Protocol Buffers (Protobuf) serialize structured data and provides a language-neutral, platform-neutral, and extensible solution. With this guide to mastering Protobuf, you'll build your skills to effectively serialize, transmit, and manage data across diverse platforms and languages. This book will help you enter the world of Protocol Buffers by unraveling the intricate nuances of Protobuf syntax and showing you how to define complex data structures. As you progress, you’ll learn schema evolution, ensuring seamless compatibility as your projects evolve. The book also covers advanced topics such as custom options and plugins, allowing you to tailor validation processes to your specific requirements. You’ll understand how to automate project builds using cutting-edge tools such as Buf and Bazel, streamlining your development workflow. With hands-on projects in Go and Python programming, you’ll learn how to practically apply Protobuf concepts. Later chapters will show you how to integrate data interchange capabilities across different programming languages, enabling efficient collaboration and system interoperability. By the end of this book, you’ll have a solid understanding of Protobuf internals, enabling you to discern when and how to use and redefine your approach to data serialization.
Table of Contents (13 chapters)

Field metadata

So far, we haven’t talked too much about field tags. In this section, we’ll dive into how they are encoded and why they are encoded as such.

First, let’s get a small refresher on what field tags are. They are identifiers for fields that will help Protobuf know into which field to deserialize some data. So, let’s say we have the following field:

uint64 id = 1;

Protobuf decodes some specific data with an ID of 1 (tag), so it will know that this data is meant to be deserialized into the id field. All of this is an abstract explanation of what’s happening, so let’s understand concretely how the field for deserialization is selected.

First, we need to understand that Protobuf only serializes a combination of type, tag, and value. The name of a field is not serialized. We already know how integer values get serialized; later, we will see how it works for other types (string, repeated, and so on). For now, we can focus on...