Chapter 7. Monitoring Flume
The User Guide for Flume states:
Monitoring in Flume is still a work in progress. Changes can happen very often. Several Flume components report metrics to the JMX platform MBean server. These metrics can be queried using JConsole.
While JMX is fine for causal browsing of metric values, the number of eyeballs looking at JConsole doesn't scale when you have hundreds or even thousands of servers sending data all over the place. What you need is a way to watch everything all at once. But what are the important things to look for? That is a very difficult question, but I'll try and cover several of the items I feel are important as we cover monitoring options in this chapter.
