In YARN, scheduling means allocation of cluster resources to applications running in the cluster. Scheduler is one of the core components of the ResourceManager service and is responsible for allocation of resources based on the container resource requirements. Random Access Memory (RAM) and the processor are the two critical resources required for execution of every container. With an increasing number of concurrent containers, fulfilling the resource requirements for each container and resource management for clusters becomes critical for the successful execution of applications.
In this chapter, you will learn about the scheduling mechanisms in YARN and how schedulers can be configured. You will also learn about queues and different scheduling algorithms available in YARN.
We will cover the following topics:
Introduction to scheduling in YARN
Introduction to queues and queue types
Capacity scheduler
Fair scheduler