Book Image

Protocol Buffers Handbook

By : Clément Jean
Book Image

Protocol Buffers Handbook

By: Clément Jean

Overview of this book

Explore how Protocol Buffers (Protobuf) serialize structured data and provides a language-neutral, platform-neutral, and extensible solution. With this guide to mastering Protobuf, you'll build your skills to effectively serialize, transmit, and manage data across diverse platforms and languages. This book will help you enter the world of Protocol Buffers by unraveling the intricate nuances of Protobuf syntax and showing you how to define complex data structures. As you progress, you’ll learn schema evolution, ensuring seamless compatibility as your projects evolve. The book also covers advanced topics such as custom options and plugins, allowing you to tailor validation processes to your specific requirements. You’ll understand how to automate project builds using cutting-edge tools such as Buf and Bazel, streamlining your development workflow. With hands-on projects in Go and Python programming, you’ll learn how to practically apply Protobuf concepts. Later chapters will show you how to integrate data interchange capabilities across different programming languages, enabling efficient collaboration and system interoperability. By the end of this book, you’ll have a solid understanding of Protobuf internals, enabling you to discern when and how to use and redefine your approach to data serialization.
Table of Contents (13 chapters)

ZigZag encoding

As we saw in the previous section, int32 and int64 are not efficient at storing negative numbers. They will always result in 10-byte-long payloads. To solve this specific use case of negative numbers, Protobuf introduces two other types: sint32 and sint64. The “s” stands for signed and they handle negative numbers.

The reason why they handle negative numbers more efficiently is that they add an extra step on top of varint encoding. This extra step, called ZigZag encoding, consists of turning all negative numbers into positive ones, and because varint encoding is very good at encoding positive numbers, we solved the problem.

Now, as usual, let’s see an example of ZigZag encoding. Let’s take our cherished 128. We have the following binary:

00000000 10000000

Now, let’s left shift by one:

00000001 00000000

We will then take the original binary and apply a right shift of 31 in the case of int32 and 63 in the case of int64...