YARN scheduler policies

As explained in the previous section, the ResourceManager acts as a pluggable global scheduler that manages and controls all the containers (resources). There are three different policies that can be applied over the scheduler, as per requirements and resource availability. They are as follows:

The FIFO scheduler
The Fair scheduler
The Capacity scheduler

The FIFO (First In First Out) scheduler

FIFO means First In First Out. As the name indicates, the job submitted first will get priority to execute; in other words, the job runs in the order of submission. FIFO is a queue-based scheduler. It is a very simple approach to scheduling and it does not guarantee performance efficiency, as each job would use a whole cluster for execution. So other jobs may keep waiting to finish their execution, although a shared cluster has a great capability to offer more-than-enough resources to many users.

The fair scheduler

Fair scheduling is the policy of scheduling that assigns resources...

Tech Concepts

Programming languages

Tech Tools

Unlimited access to the largest independent learning library in tech of over 8,000 expert-authored tech books and videos.

Innovative learning tools, including AI book assistants, code context explainers, and text-to-speech.

50+ new titles added per month and exclusive early access to books as they are being written.

YARN Essentials

By : Fasale, Nirmal Kumar

YARN Essentials

By: Fasale, Nirmal Kumar

Overview of this book

YARN scheduler policies

The FIFO (First In First Out) scheduler

The fair scheduler

YARN Essentials

By : Fasale, Nirmal Kumar

YARN Essentials

By: Fasale, Nirmal Kumar

Overview of this book

YARN scheduler policies

The FIFO (First In First Out) scheduler

The fair scheduler

Confirmation

Buy this book with your credits?

Submit Your Feedback

Create a Free Account To Continue Reading

Sign in to activate your 7-day free access