Similar to CapacityScheduler, Fair Scheduler was designed to enforce fair shares of cluster resources in a multiuser environment. In this recipe, we will outline steps to configure Fair Scheduler for a Hadoop cluster.
We assume that our Hadoop cluster has been configured properly and all the daemons are running without any problems.
Log in to the master node from the Hadoop administrator machine using the following command:
ssh hduser@master
Perform the following steps to configure Hadoop Fair Scheduler:
Enable fair scheduling by changing the following property in the file
$HADOOP_HOME/conf/mapred-site.xml
:<property> <name>mapred.jobtracker.taskScheduler</name> <value>org.apache.hadoop.mapred.FairScheduler</value> </property>
Create the Fair Scheduler configuration file,
$HADOOP_HOME/conf/fair-scheduler.xml
, with content similar to the following:<?xml version="1.0"?> <allocations...