In this recipe, we will be configuring users who can submit jobs to the queue and can also set rule for various job submissions.
Let's look at another use case where, if user hadoop
submits a job, it should go to the prod
queue and if any other users submits a job, it must go to dev
queue. How can we set up something like this?
Make sure that the user has a running cluster with HDFS and YARN configured. It's best to have gone through at least the previous recipe.
Connect to the
master1.cyrus.com
Namenode and switch as userhadoop
.Edit the
capacity-scheduler.xml
allocation file as shown next:<property> <name>yarn.scheduler.capacity.queue-mappings</name> <value>u:d1:dev,g:group1:default,u:hadoop:prod</value> </property>
Make the preceding changes and copy the file across all nodes and restart the YARN daemons.
Whenever the
d1
user submits a job, it should go to thedev
queue and for user...