Book Image

Real-World SRE

By : Nat Welch
Book Image

Real-World SRE

By: Nat Welch

Overview of this book

Real-World SRE is the go-to survival guide for the software developer in the middle of catastrophic website failure. Site Reliability Engineering (SRE) has emerged on the frontline as businesses strive to maximize uptime. This book is a step-by-step framework to follow when your website is down and the countdown is on to fix it. Nat Welch has battle-hardened experience in reliability engineering at some of the biggest outage-sensitive companies on the internet. Arm yourself with his tried-and-tested methods for monitoring modern web services, setting up alerts, and evaluating your incident response. Real-World SRE goes beyond just reacting to disaster—uncover the tools and strategies needed to safely test and release software, plan for long-term growth, and foresee future bottlenecks. Real-World SRE gives you the capability to set up your own robust plan of action to see you through a company-wide website crisis. The final chapter of Real-World SRE is dedicated to acing SRE interviews, either in getting a first job or a valued promotion.
Table of Contents (13 chapters)
12
Index

Managing and maintaining monitoring data


OK, now you have the basics of a monitoring system all set up. You have data flowing from your service to a datastore. You can visualize how things change over time. As you accumulate data over time, you are going to have to maintain that service. There are a few tricks for dealing with that.

The first way is a classic, tried, and true method—pay someone else to do it. There are tons of companies that sell hosted monitoring software. Datadog, Honeycomb, GrafanaCloud, InfluxCloud, Librato, Instrumental, New Relic, and many others, sell products with all sorts of features and tools where you do not have to host your monitoring tools.

Sometimes though, you do not have that kind of money, you have special security requirements, you cannot deal with the restrictions that hosted services impose, or you have some other reason not to relegate your monitoring infrastructure to someone else. In that case, first and foremost, you should monitor your monitoring...