Book Image

Deployment with Docker

By : Grubor
Book Image

Deployment with Docker

By: Grubor

Overview of this book

Deploying Docker into production is considered to be one of the major pain points in developing large-scale infrastructures, and the documentation available online leaves a lot to be desired. With this book, you will learn everything you wanted to know to effectively scale your deployments globally and build a resilient, scalable, and containerized cloud platform for your own use. The book starts by introducing you to the containerization ecosystem with some concrete and easy-to-digest examples; after that, you will delve into examples of launching multiple instances of the same container. From there, you will cover orchestration, multi-node setups, volumes, and almost every relevant component of this new approach to deploying services. Using intertwined approaches, the book will cover battle-tested tooling, or issues likely to be encountered in real-world scenarios, in detail. You will also learn about the other supporting components required for a true PaaS deployment and discover common options to tie the whole infrastructure together. At the end of the book, you learn to build a small, but functional, PaaS (to appreciate the power of the containerized service approach) and continue to explore real-world approaches to implementing even larger global-scale services.
Table of Contents (18 chapters)
Title Page
Credits
About the Author
Acknowledgments
About the Reviewer
www.PacktPub.com
Customer Feedback
Preface
Index

High availability pipelines


Previously, we spent the majority of our time working with socket-based communication between nodes on a cluster, which is generally something that makes sense to most people and has tooling built around it in almost every programming language. So, it is the first tool that people transitioning their classic infrastructure to containers usually go for, but for large-and-beyond scales where you are dealing with pure data processing, it simply does not work well due to the back-pressure caused by exceeding the capacity of a particular stage on the rest of the processing pipeline.

If you imagine each cluster service as a consecutive set of transformation steps, the socket-based system would go through a loop of steps similar to these:

  • Opening a listening socket.
  • Looping forever doing the following:
    • Waiting for data on a socket from the previous stage.
    • Processing that data.
    • Sending the processed data to the next stage's socket.

But what happens in that last step if the...