YARN stands for Yet Another Resource Negotiator, and was introduced with Apache Hadoop 2.0 to address the scalability and manageability issues that existed with the previous versions. In Hadoop 1.0, we have two major components for job execution: JobTracker and task tracker. JobTracker is responsible for managing resources and scheduling jobs. It is also responsible for tracking the status of each job and restarting them if there is any failure. The task trackers are responsible for running tasks and sending progress report to JobTracker. The JobTracker also reschedules failed tasks on different task trackers. As JobTracker could be overloaded with multiple tasks, Hadoop 1.0 made several changes in its architecture to eliminate the following limitations:
- Scalability: In Hadoop 1.0, the JobTracker is responsible for scheduling the jobs, monitoring each job, and...
