YARN is a new generation resource manager and plays the role of scheduling and executing the application over the Hadoop cluster. In this section, we will look at how we can run benchmarks against a YARN cluster.
Hadoop provides three different kinds of scheduling algorithms in the form of queues. They are called FIFO, Capacity and Fair schedulers. Each of these schedulers takes different factors like available capacity, fairness among different running jobs, and guaranteed resource availability. Now, an important point to decide is what type of queue is suitable for your workload in the production environment. This test helps you decide that. One thing to note here is that the simulator works on prediction. It does not run jobs on the entire cluster. It is always time-consuming and expensive to run it on a very large cluster. Moreover, very few organizations run on a very large cluster. This test predicts how well the queues fit into your workloads. The...