This section captures the various situations and scenarios that can occur and cause failures in a Cassandra cluster. We will also equip you with the knowledge and talk about the steps to handle these situations. These situations are specific to version 1.1.6 but can be applied to others as well.
Say, this is the problem: you're running an n node, for example let's say there are three node clusters and from that one node goes down; this will result in unrecoverable hardware failure. The solution is this: replace the dead nodes with new nodes.
The following are the steps to achieve the solution:
Confirm the node failure using the
nodetool ringcommand:bin/nodetool ring -h hostnameThe dead node will be shown as
DOWN; let's assumenode3is down:192.168.1.54 datacenter1rack1 Up Normal 755.25 MB 50.00% 0 192.168.1.55 datacenter1rack1 Down Normal 400.62 MB 25.00% 42535295865117307932921825928971026432 192.168.1.56 datacenter1rack1 Up Normal 793.06 MB...