Summary

Hadoop doesn't provide any out of the box monitoring capabilities, but it does reveal lots of information about the internal status of its components. You can use the existing open source tools to monitor these metrics, and alert you when critical events happen. Unlike a traditional database system, not every component failure should be treated with the same priority and you should keep this in mind when implementing a monitoring solution. Separating checks into critical and regular groups would allow you to monitor the Hadoop cluster with a decent degree of flexibility.

Until now, we were discussing how to build and host your own Hadoop cluster. There are many use cases where owning and maintaining the full Hadoop infrastructure is not the best choice. An alternative to this approach is using a cloud infrastructure. In the next chapter, we will review the options you have if you want to host your cluster in the cloud.