Managing the NameNode

Let's do some more risk reduction. In Chapter 6, When Things Break, I probably scared you when talking about the potential consequences of a failure of the host running the NameNode. If that section did not scare you, go back and re-read it—it should have. The summary is that the loss of the NameNode could see you losing every single piece of data on the cluster. This is because the NameNode writes a file called fsimage that contains all the metadata for the filesystem and records which blocks comprise which files. If the loss of the NameNode host makes the fsimage unrecoverable, all the HDFS data is likewise lost.

Configuring multiple locations for the fsimage class

The NameNode can be configured to simultaneously write fsimage to multiple locations. This is purely a redundancy mechanism, the same data is written to each location and there is no attempt to use multiple storage devices for increased performance. Instead, the policy is that mltiple copies of fsimage will...