The fully-distributed mode

In the previous section, we highlighted the standalone Hadoop and YARN configurations, and in this section we'll focus on the fully-distributed mode setup. This section describes how to install, configure, and manage Hadoop and YARN in fully-distributed, very large clusters with thousands of nodes in them.

In order to start with fully-distributed mode, we first need to download the stable version of Hadoop from Apache mirrors. Installing Hadoop in distributed mode generally means unpacking the software distribution on each machine in the cluster or installing Red Hat Package Managers (RPMs). As Hadoop follows a master-slave architecture, one machine in the cluster is designated as the NameNode (NN), one as the ResourceManager (RM), and the rest of the machines, DataNodes (DN) and NodeManagers (NM), will typically acts as slaves.

After the successful unpacking of software distribution on each cluster machine or RPM installation, you need to take care of a very...