Book Image

Real-World SRE

By : Pavlos Ratis, Nat Welch
Book Image

Real-World SRE

By: Pavlos Ratis, Nat Welch

Overview of this book

Real-World SRE is the go-to survival guide for the software developer in the middle of catastrophic website failure. Site Reliability Engineering (SRE) has emerged on the frontline as businesses strive to maximize uptime. This book is a step-by-step framework to follow when your website is down and the countdown is on to fix it. Nat Welch has battle-hardened experience in reliability engineering at some of the biggest outage-sensitive companies on the internet. Arm yourself with his tried-and-tested methods for monitoring modern web services, setting up alerts, and evaluating your incident response. Real-World SRE goes beyond just reacting to disaster—uncover the tools and strategies needed to safely test and release software, plan for long-term growth, and foresee future bottlenecks. Real-World SRE gives you the capability to set up your own robust plan of action to see you through a company-wide website crisis. The final chapter of Real-World SRE is dedicated to acing SRE interviews, either in getting a first job or a valued promotion.
Table of Contents (16 chapters)
Real-World SRE
Contributors
Preface
Other Books You May Enjoy
Index

Calling all clear


The first step of declaring all clear is making sure that your app does the thing it should. Then you can begin solving the root cause. Note that your first goal is always to bring the system up. Your first goal is not to fix things or to figure out the root cause.

Figure 14: Screenshot of https://status.cloud.google.com/incident/cloud-networking/18004. Notice the two separate messages calling all clear: the first saying things should be becoming healthy, and the second saying that everything is all clear and defining when further communication will happen.

Once you have verified the solution, then post an update saying that things are resolved and mention whether there will be a follow-up. Post at least one more update when you have found and fixed the root cause of the outage. Often this is immediate, but sometimes it takes hours or days to figure out why something happened in your system.