Storm on YARN

In Chapter 6, YARN – Bringing Other Paradigms to Hadoop, we built a YARN application that executes distributed shell commands. Storm is one such application that has been brought to YARN by Yahoo!. Any Hadoop cluster running YARN can now execute streaming workloads for low-latency real-time applications. The Application Master and client program to execute Storm are available for deployment. It is open sourced on GitHub at https://github.com/yahoo/storm-yarn.

Installing Apache Storm-on-YARN

Apache Storm-on-YARN can currently be installed from GitHub. This section assumes that the Hadoop 2.2.0 cluster is available.

Prerequisites

The following prerequisites are necessary to install Storm-on-YARN:

• Java 7

• Maven: This needs to be installed on the gateway machine to help compile and deploy the Storm-on-YARN Application Master and client:

  • wget http://mirror.symnds.com/software/Apache/maven/maven-3/3.1.1/binaries/apache-maven-3.1.1-bin.tar.gz

  • tar –zxvf apache-maven-3.1.1-bin.tar.gz

  • mkdir...