Writing your own application

YARN framework provides flexibility to run any application in a clustered environment. An application could be as simple as a Java process, a shell script, or a simple date command. The ResourceManager service manages the cluster resource allocation and the NodeManager services execute tasks as specified by the application framework; for example, the map and reduce tasks of Hadoop MapReduce jobs.

In this section, you will write your own applications to run in a distributed environment through YARN.

The complete process can be summarized in four simple steps, which are shown in the following diagram:

Step 1 – Create a new project and add Hadoop-YARN JAR files

We will create a new Java project in Eclipse and will use the YARN client APIs to write a simple YARN application. You can either create a simple Java project or a Maven project.

You need to add the following jar files to your project's build path:

• hadoop-yarn-client-2.5.1.jar

• hadoop-yarn-api-2.5.1.jar

• hadoop...