Once in a while, a dead node needs to be replaced. This means you just wanted an exact replacement instead of just removal. Here are the steps:
Note down the dead node's token.
Set the new node's initial token as the dead node's token minus one. This node is going to own the new data, so you must make sure that the data directories are empty to avoid any conflict.
Configure
cassandra.yaml
appropriately. Similar to the way we did when adding a new node. (Refer to the Adding nodes to a cluster section in this chapter.)Let the bootstrap complete and see the node appear in the
nodetool ring
listing.Perform a
nodetool repair
for each keyspace for integrity.Perform
nodetool removetoken
for the old node.
Let us see this in action. The example cluster here has three nodes and a replication factor of 2. One of the nodes is down. We will replace it with a new node.
Here is what the current ring looks like:
$ bin/nodetool -h 10.99.9.67 ring Address Status EO* Token ...