Book Image

Real-World SRE

By : Pavlos Ratis, Nat Welch
Book Image

Real-World SRE

By: Pavlos Ratis, Nat Welch

Overview of this book

Real-World SRE is the go-to survival guide for the software developer in the middle of catastrophic website failure. Site Reliability Engineering (SRE) has emerged on the frontline as businesses strive to maximize uptime. This book is a step-by-step framework to follow when your website is down and the countdown is on to fix it. Nat Welch has battle-hardened experience in reliability engineering at some of the biggest outage-sensitive companies on the internet. Arm yourself with his tried-and-tested methods for monitoring modern web services, setting up alerts, and evaluating your incident response. Real-World SRE goes beyond just reacting to disaster—uncover the tools and strategies needed to safely test and release software, plan for long-term growth, and foresee future bottlenecks. Real-World SRE gives you the capability to set up your own robust plan of action to see you through a company-wide website crisis. The final chapter of Real-World SRE is dedicated to acing SRE interviews, either in getting a first job or a valued promotion.
Table of Contents (16 chapters)
Real-World SRE
Contributors
Preface
Other Books You May Enjoy
Index

Chapter 3. Incident Response

Incident response is the next level in the hierarchy we introduced in Chapter 1, Introduction. While introducing monitoring into your organization, or project, is mostly a technical exercise, incident response is almost all process and people. Incident response builds upon the world of data that we built using monitoring and starts our feedback loop, helping us to find and tighten our monitoring for our services. This shows us what is important, because if we do not get an alert and someone tells us that our service was not working, then we were not monitoring the right things.

The inverse of this is also true, although rarer—when we do not launch or deliver new features because we spend all of our time trying to reach a level of reliability that our service has trouble reaching.

Figure 1: Our current position in the hierarchy

In this chapter, we will define incidents and explain how to respond to them. The chapter will help you to establish processes of alerting...