Configuring Metrics2

For Hadoop version 2, which uses the YARN framework, the metrics can be configured using hadoop-metrics2.properties, in the $HADOOP_HOME folder:

*.sink.file.class=org.apache.hadoop.metrics2.sink.FileSink
*.period=10
namenode.sink.file.filename=namenode-metrics.out
datanode.sink.file.filename=datanode-metrics.out
jobtracker.sink.file.filename=jobtracker-metrics.out
tasktracker.sink.file.filename=tasktracker-metrics.out
maptask.sink.file.filename=maptask-metrics.out
reducetask.sink.file.filename=reducetask-metrics.out

We can also script it out and use it for metrics generation, shown as follows:

# namenode
[script://./bin/hadoop_metrics.sh http://192.168.1.70:50070/jmx]
disabled = 0
interval = 10
sourcetype = hadoop_metrics
index = hadoop_metrics

# datanode
[script://./bin/hadoop_metrics.sh http://192.168.1.70:50075/jmx]
disabled = 0
interval = 10
sourcetype = hadoop_metrics
index = hadoop_metrics

# jobtracker
[script://./bin/hadoop_metrics.sh http://192.168.1.70:50030...