Technology organizations, from the CTO to the system administrators, have spent countless hours over the years trying to prevent their database systems from experiencing failure. This is because failure typically meant downtime for the application or, even worse, a loss of critical data.

As we discussed in Chapter 1, Cassandra's Approach to High Availability, attempts to make these systems highly available often required a significant amount of human intervention to restore functionality in the case of a failure. Cassandra, as you have learned, was designed from the ground up to consider failure as a normal operational state. This is because in a large distributed system, the chance that at any given moment a piece of hardware will fail is relatively high, so the system must know how to deal with those problems.

But even a robust system such as Cassandra, which is designed to handle failure scenarios without losing data or compromising availability, requires vigilance...