How to Configure Metricbeat for Application and Server infrastructure

[box type="note" align="" class="" width=""]This article is an excerpt from a book written by Pranav Shukla and Sharath Kumar M N titled Learning Elastic Stack 6.0. This book provides detailed understanding in how you can employ Elastic Stack in performing distributed analytics along with resolving various data processing challenges.[/box]

In today’s tutorial, we will show the step-by-step configuration of Metricbeat, a Beats platform for monitoring server and application infrastructure.

Configuring Metricbeat  

The configurations related to Metricbeat are stored in a configuration file named metricbeat.yml, and it uses YAML syntax.

The metricbeat.yml file contains the following:

• Module configuration
• General settings
• Output configuration
• Processor configuration
• Path configuration
• Dashboard configuration
• Logging configuration

Let's explore some of these sections.

Module configuration

Metricbeat comes bundled with various modules to collect metrics from the system and applications such as Apache, MongoDB, Redis, MySQL, and so on.

Metricbeat provides two ways of enabling modules and metricsets:

• Enabling module configs in the modules.d directory
• Enabling module configs in the metricbeat.yml file
  

Enabling module configs in the modules.d directory

The modules.d directory contains default configurations for all the modules available in Metricbeat. The configuration specific to a module is stored in a .yml file with the name of the file being the name of the module. For example, the configuration related to the MySQL module would be stored in the mysql.yml file. By default, excepting the system module, all other modules are disabled. To list the modules that are available in Metricbeat, execute the following command:

Windows:
D:packtmetricbeat-6.0.0-windows-x86_64>metricbeat.exe modules list

Linux:
[locationOfMetricBeat]$./metricbeat modules list

The modules list command displays all the available modules and also lists which modules are currently enabled/disabled.

As each module comes with the default configurations, make the appropriate changes in the module configuration file.

The basic configuration for mongodb module will look as follows:

- module: mongodb
metricsets: ["dbstats", "status"]
period: 10s
hosts: ["localhost:27017"]
username: user
password: pass

To enable it, execute the modules enable command, passing one or more module name. For example:

Windows:
D:packtmetricbeat-6.0.0-windows-x86_64>metricbeat.exe modules enable
redis mongodb

Linux:
[locationOfMetricBeat]$./metricbeat modules enable redis mongodb

Similar to disable modules, execute the modules disable command, passing one or more module names to it. For example:

Windows:
D:packtmetricbeat-6.0.0-windows-x86_64>metricbeat.exe modules disable
redis mongodb

Linux:
[locationOfMetricBeat]$./metricbeat modules disable redis mongodb

To enable dynamic config reloading, set reload.enabled to true and to specify the frequency to look for config file changes. Set the reload.period parameter under the metricbeat.config.modules property.

For example:
#metricbeat.yml

metricbeat.config.modules:
path: ${path.config}/modules.d/*.yml
reload.enabled: true
reload.period: 20s

Enabling module config in the metricbeat.yml file

If one is used to earlier versions of Metricbeat, one can enable the modules and metricsets in
the metricbeat.yml file directly by adding entries to the metricbeat.modules list. Each entry in the list begins with a dash (-) and is followed by the settings for that module. For Example:

metricbeat.modules:
#------------------ Memcached Module -----------------------------
- module: memcached
metricsets: ["stats"]
period: 10s
hosts: ["localhost:11211"]

#------------------- MongoDB Module ------------------------------
- module: mongodb
metricsets: ["dbstats", "status"]
period: 5s

It is possible to specify the module multiple times and specify a different period to use for one or more metricset. For example:

#------- Couchbase Module -----------------------------
- module: couchbase
metricsets: ["bucket"]
period: 15s
hosts: ["localhost:8091"]

- module: couchbase
metricsets: ["cluster", "node"]
period: 30s
hosts: ["localhost:8091"]

General settings

This section contains configuration options and some general settings to control the behavior of Metricbeat.

Some of the configuration options/settings are:

• name: The name of the shipper that publishes the network data. By default, hostname is used for this field:

name: "dc1-host1"

• tags: The list of tags that will be included in the tags field of every event Metricbeat ships. Tags make it easy to group servers by different logical properties and help when filtering events in Kibana and Logstash:

tags: ["staging", "web-tier","dc1"]

• max_procs: The maximum number of CPUs that can be executing simultaneously. The default is the number of logical CPUs available in the System:

max_procs: 2

Output configuration

This section is used to configure outputs where the events need to be shipped. Events can be sent to single or multiple outputs simultaneously. The allowed outputs are Elasticsearch, Logstash, Kafka, Redis, file, and console.

Some of the outputs that can be configured are as follows:

• elasticsearch: It is used to send the events directly to Elasticsearch. A sample Elasticsearch output configuration is shown in the following code snippet:

output.elasticsearch:
enabled: true
hosts: ["localhost:9200"]

Using the enabled setting, one can enable or disable the output. hosts accepts one or more Elasticsearch node/server. Multiple hosts can be defined for failover purposes. When multiple hosts are configured, the events are distributed to these nodes in round robin order. If Elasticsearch is secured, then the credentials can be passed using the username and password settings:

output.elasticsearch:
enabled: true
hosts: ["localhost:9200"]
username: "elasticuser"
password: "password"

To ship the events to the Elasticsearch ingest node pipeline so that they can be pre-processed before being stored in Elasticsearch, the pipeline information can be provided using the pipleline setting:

output.elasticsearch:
enabled: true
hosts: ["localhost:9200"]
pipeline: "ngnix_log_pipeline"

The default index the data gets written to is of the format metricbeat-%{[beat.version]}-%{+yyyy.MM.dd}. This will create a new index every day. For example if today is December 2, 2017 then all the events are placed in the metricbeat-6.0.0-2017-12-02 index. One can override the index name or the pattern using the index setting. In the following configuration snippet, a new index is created for every month:

output.elasticsearch:
hosts: ["http://localhost:9200"]
index: "metricbeat-%{[beat.version]}-%{+yyyy.MM}"

Using the indices setting, one can conditionally place the events in the appropriate index that matches the specified condition. In the following code snippet, if the message contains the DEBUG string, it will be placed in the debug-%{+yyyy.MM.dd} index. If the message contains the ERR string, it will be placed in the error-%{+yyyy.MM.dd} index. If the message contains neither of these
texts, then those events will be pushed to the logs-%{+yyyy.MM.dd} index as specified in the index parameter:

output.elasticsearch:
hosts: ["http://localhost:9200"]
index: "logs-%{+yyyy.MM.dd}"
indices:
- index: "debug-%{+yyyy.MM.dd}"
when.contains:
message: "DEBUG"
- index: "error-%{+yyyy.MM.dd}"
when.contains:
message: "ERR"

When the index parameter is overridden, disable templates and dashboards by adding the following setting in:

setup.dashboards.enabled: false
setup.template.enabled: false

Alternatively, provide the value for setup.template.name and setup.template.pattern in the metricbeat.yml configuration file, or else Metricbeat will fail to run.

• logstash: It is used to send the events to Logstash.

To use Logstash as the output, Logstash needs to be configured with the Beats input plugin to receive incoming Beats events.

A sample Logstash output configuration is as follows:

output.logstash:
enabled: true
hosts: ["localhost:5044"]

Using the enabled setting, one can enable or disable the output. hosts accepts one or more Logstash servers. Multiple hosts can be defined for failover purposes. If the configured host is unresponsive, then the event will be sent to one of the other configured hosts. When multiple hosts are configured, the events are distributed in random order. To enable load balancing of events across the Logstash hosts, use the loadbalance flag, set to true:

output.logstash:
hosts: ["localhost:5045", "localhost:5046"]
loadbalance: true

• console: It is used to send the events to stdout. The events are written in JSON format. It is useful during debugging or testing. A sample console configuration is as follows:

output.console:
enabled: true
pretty: true

Logging

This section contains the options for configuring the Filebeat logging output. The logging system can write logs to syslog or rotate log files. If logging is not explicitly configured, file output is used on Windows systems, and syslog output is used on Linux and OS X.

A sample configuration is as follows:

logging.level: debug
logging.to_files: true
logging.files:
path: C:logsmetricbeat
name: metricbeat.log
keepfiles: 10

Some of the configuration options are:

• level: To specify the logging level.
• to_files: To write all logging output to files. The files are subject to file rotation. This is the default value.
• to_syslog: To write the logging output to syslogs if this setting is set to true.
• files.path, files.name, and files.keepfiles: These are used to specify the location of the file, the name

We successfully configured Beat Library, MetricBeat and developed good transmission of operational metrics to Elasticsearch, making it easy to monitor systems and services on servers with much ease.

If you found this tutorial useful, do check out the book Learning Elastic Stack 6.0 to examine the fundamentals of Elastic Stack in detail and start developing solutions for problems like logging, site search, app search, metrics and more.