When Datanodes come up in large clusters with more than 200 nodes, Namenode will be overwhelmed by the block reports and this can cause Namenode to become unresponsive.
This recipe makes more sense for large clusters, not in terms of the number of nodes, but the number of blocks in the cluster.
ssh to Namenode and edit the
hdfs-site.xmlfile to add the following property to it:<property> <name>dfs.blockreport.initialDelay</name> <value>20</value> </property>
Copy
hdfs-site.xmlacross all nodes in the cluster.Restart HDFS daemons across the nodes for the property to take effect:
$ stop-dfs.sh $ start-dfs.sh