Book Image

Protocol Buffers Handbook

By : Clément Jean
Book Image

Protocol Buffers Handbook

By: Clément Jean

Overview of this book

Explore how Protocol Buffers (Protobuf) serialize structured data and provides a language-neutral, platform-neutral, and extensible solution. With this guide to mastering Protobuf, you'll build your skills to effectively serialize, transmit, and manage data across diverse platforms and languages. This book will help you enter the world of Protocol Buffers by unraveling the intricate nuances of Protobuf syntax and showing you how to define complex data structures. As you progress, you’ll learn schema evolution, ensuring seamless compatibility as your projects evolve. The book also covers advanced topics such as custom options and plugins, allowing you to tailor validation processes to your specific requirements. You’ll understand how to automate project builds using cutting-edge tools such as Buf and Bazel, streamlining your development workflow. With hands-on projects in Go and Python programming, you’ll learn how to practically apply Protobuf concepts. Later chapters will show you how to integrate data interchange capabilities across different programming languages, enabling efficient collaboration and system interoperability. By the end of this book, you’ll have a solid understanding of Protobuf internals, enabling you to discern when and how to use and redefine your approach to data serialization.
Table of Contents (13 chapters)

The evolution rules

We saw multiple problems caused by the evolution of the proto file and it is important to keep these in mind. So, it is time to recapitulate and formulate rules on how to evolve schemas in Protobuf.

Here’s the list of rules that we should follow if we need to maintain backward and forward compatibility:

  • Never remove a reserved statement. This probably goes without saying but reserved statements are here to protect us. Do not remove or modify them; otherwise, you will get undefined behaviors.
  • Never modify a field tag. This will lead Protobuf to decode data into unknown fields between versions. This means you will not receive that data and get a default value.
  • Avoid reusing a field tag. As we saw, this can cause problems such as integer overflow or other problems due to type conversion. It is better to just add a new field to deal with the new data. Remember that tags are encoded as varints.
  • Always add a reserved tag when deleting a field...