Omega

Omega is Google's next generation cluster management system.

Omega is specifically focused on a cluster scheduling architecture that uses parallelism, shared state, and optimistic concurrency control.

From the past experience, Google noticed that as the clusters and their workloads increase, the scheduler is at risk of becoming a scalability bottleneck.

Google's production job scheduler has experienced all of this. Over the years, it has evolved into a complicated, sophisticated system that is hard to change.

A schematic overview of the scheduling architectures can be seen in the following figure:

• contrib project to Hadoop 0.20 branch and is not a very large code base.

• Corona is integrated with the fair-scheduler.

• YARN is more interested in the capacity scheduler.

Google identified the following two prevalent scheduler architectures shown in the preceding figure:

• Monolithic schedulers: This uses a single, centralized scheduling algorithm for all jobs (our existing scheduler is one of these...