Limitations of the older HDFS architecture
The older HDFS architecture had two main components, which are as follows:
- Namespace: This is the HDFS component that handles building blocks such as directories, files, and the actual file blocks. The Namespace component supports the create, delete, list, and update/modify operations on these building blocks. The Namespace component is within the NameNode daemon.
- Block Storage Service: This is the HDFS component that handles file block management. The block storage component is divided between the NameNode and DataNode. The DataNode part of the block storage service takes care of block storage on the local machine in a cluster. It provides read and write services for the blocks. The NameNode part of the block storage performs the following actions:
- Taking care of DataNode registrations, monitoring, and health reports.
- Digesting block reports from DataNodes and storing the location of the file blocks in memory.
- Dealing with the create, delete, list...
