Configuring Capacity Scheduler

Capacity Scheduler is mainly designed for multitenancy, where multiple organizations collectively fund the cluster based on the computing needs. There is an added benefit that an organization can access any excess capacity not being used by others. This provides elasticity for the organizations in a cost-effective manner.

Getting ready

For this recipe, you will again need a running cluster with YARN and HDFS configured in the cluster. Readers are recommended to read the previous recipes in this chapter to understand this recipe better.

In Hadoop 2.x, the default scheduler is Capacity Scheduler and it is enabled by default, unless modified explicitly as seen in the previous recipes where we have configured Fair Scheduler.

How to do it...

1. Connect to the master1.cyrus.com master node and switch as user hadoop.

2. Modify the yarn-site.xml file by changing the following parameter:

   <property>
       <name>yarn.resourcemanager.scheduler.class</name>
       <value...