In this chapter, we discussed the need to shift some testing to the right into production where we continuously monitor key performance indicators and focus on the mean time to recovery in an effort to increase our confidence in the stability of our cloud-native systems. We leverage both real and synthetic traffic and fully instrument our cloud-native components to maximize the observability of the system. We discussed the need to alert liberally, but page judiciously on user-visible symptoms rather than causes. Once a problem is identified, we investigate the system's work metrics, resource metrics, and events to diagnose the root cause and determine the appropriate actions that must be taken to recover quickly. We also discussed how the observability of a cloud-native system enables teams to continuously tune and improve the performance of the system.
In the next...