"Always code as if the guy who ends up maintaining your code will be a violent psychopath who knows where you live."
– Martin Golding
YARN or (Yet Another Resource Negotiator) was introduced in Hadoop version 2 to open distributed programming for all of the problems that may not necessarily be addressed using the MapReduce programming technique. Let's look at the key reasons behind introducing YARN in Hadoop:
- The older Hadoop used Job Tracker to coordinate running jobs whereas Task Tracker was used to run assigned jobs. This eventually became a bottleneck due to a single Job Tracker when working with a high number of Hadoop nodes.
- With traditional MapReduce, the nodes were assigned fixed numbers of Map and Reduce slots. Due to this nature, the utilization of the cluster resources was not optimal due to inflexibility between...