Book Image

Real-World SRE

By : Nat Welch
Book Image

Real-World SRE

By: Nat Welch

Overview of this book

Real-World SRE is the go-to survival guide for the software developer in the middle of catastrophic website failure. Site Reliability Engineering (SRE) has emerged on the frontline as businesses strive to maximize uptime. This book is a step-by-step framework to follow when your website is down and the countdown is on to fix it. Nat Welch has battle-hardened experience in reliability engineering at some of the biggest outage-sensitive companies on the internet. Arm yourself with his tried-and-tested methods for monitoring modern web services, setting up alerts, and evaluating your incident response. Real-World SRE goes beyond just reacting to disaster—uncover the tools and strategies needed to safely test and release software, plan for long-term growth, and foresee future bottlenecks. Real-World SRE gives you the capability to set up your own robust plan of action to see you through a company-wide website crisis. The final chapter of Real-World SRE is dedicated to acing SRE interviews, either in getting a first job or a valued promotion.
Table of Contents (13 chapters)

Defining a plan

Defining a plan is a series of steps. Each step requires the evaluation of a question to discover the answer:

  1. What is our current capacity?
  2. When are we going to run out of capacity?
  3. How should we change our capacity?
  4. Execute plan!

Let us work through each step. Note that I am using long periods of time in a lot of the examples below. If you are using systems that can automatically add and remove capacity, you may be working with data on the scale of minutes instead of days.

What is our current capacity?

There are lots of ways to define the capacity of your infrastructure. You can use aggregated metrics such as CPU usage, disk storage availability, requests per minute, packets per second, or any application metric. Usually, the metrics you want to focus on are the resources that you use the most or the resources that are most important to you. What is most important often comes from your SLOs and the SLIs (Service Level Indicators) behind them. Note that the metrics do not have...