Installing the ELK Stack

For a complete installation of ELK Stack, we first need to install individual components that are explained one by one in the following sections.

Elasticsearch

Elasticsearch 6.0 requires that we have Java 8 at the least. Before you proceed with the installation of Elasticsearch, please ensure which version of Java is present in your system by executing the following command:

java -version
echo $JAVA_HOME

After the setup is complete, we can go ahead and run Elasticsearch. You can find the binaries at www.elastic.co/downloads.

Installing Elasticsearch using a TAR file

First, we will download Elasticsearch 6.1.3.tar, as shown in the following code block:

curl -L -O https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.1.3.tar.gz

Then, extract it as follows:

tar -xvf elasticsearch-6.1.3.tar.gz

You will then see that a bunch of files and folders have been created. We can now proceed to the bin directory, as follows:

cd elasticsearch-6.1.3/bin

 We are now ready to start our node and a single cluster:

./elasticsearch

Installing Elasticsearch with Homebrew

You can also install Elasticsearch on macOS through Homebrew, as follows:

brew install elasticsearch

Installing Elasticsearch with MSI Windows Installer

Windows users are recommended to use the MSI Installer package. This package includes a graphical user interface (GUI) that guides the users through the installation process.

First, download the Elasticsearch 6.1.3 MSI from https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.1.3.msi.

Launch the GUI by double-clicking on the downloaded file. On the first screen, select the deployment directories:

Installing Elasticsearch with the Debian package

On Debian, before you can proceed with the installation process, you may need to install the apt-transport-https package first:

sudo apt-get install apt-transport-https

Save the repository definition to /etc/apt/sources.list.d/elastic-6.x.list:

echo "deb https://artifacts.elastic.co/packages/6.x/apt stable main" | sudo tee -a /etc/apt/sources.list.d/elastic-6.x.list

You can install the elasticsearch Debian package with the following code:

sudo apt-get update && sudo apt-get install elasticsearch

Installing Elasticsearch with the RPM package

Download and install the public signing key:

rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch

Create a file named elasticsearch.repo in the /etc/yum.repos.d/ directory for Red Hat-based distributions or in the /etc/zypp/repos.d/ directory for openSUSE-based distributions, containing the following code:

[elasticsearch-6.x]
 name=Elasticsearch repository for 6.x packages
 baseurl=https://artifacts.elastic.co/packages/6.x/yum
 gpgcheck=1
 gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
 enabled=1
 autorefresh=1
 type=rpm-md

Your repository is now ready for use. You can now install Elasticsearch with one of the following commands:

You can use yum on CentOS and older Red Hat-based distributions:

sudo yum install elasticsearch

You can use dnf on Fedora and other newer Red Hat distributions:

sudo dnf install elasticsearch

You can use zypper on openSUSE-based distributions:

sudo zypper install elasticsearch

Elasticsearch can be started and stopped using the service command:

sudo -i service elasticsearch start
sudo -i service elasticsearch stop

Logstash

Logstash requires at least Java 8. Before you go ahead with the installation of Logstash, please check the version of Java in your system by running the following command:

java -version
echo $JAVA_HOME

Using apt package repositories

Download and install the public signing key:

wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -

You may need to install the apt-transport-https package on Debian before proceeding, as follows:

sudo apt-get install apt-transport-https

Save the repository definition to /etc/apt/sources.list.d/elastic-6.x.list, as follows:

echo "deb https://artifacts.elastic.co/packages/6.x/apt stable main" | sudo tee -a /etc/apt/sources.list.d/elastic-6.x.list

Run sudo apt-get update and the repository will be ready for use. You can install it using the following code:

sudo apt-get update && sudo apt-get install logstash

Using yum package repositories

Download and install the public signing key:

rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch

Add the following in your /etc/yum.repos.d/ directory in a file with a .repo suffix (for example, logstash.repo):

[logstash-6.x]
 name=Elastic repository for 6.x packages
 baseurl=https://artifacts.elastic.co/packages/6.x/yum
 gpgcheck=1
 gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
 enabled=1
 autorefresh=1
 type=rpm-md

Your repository is now ready for use. You can install it using the following code:

sudo yum install logstash

Kibana

Starting with version 6.0.0, Kibana only supports 64-bit operating systems.

Installing Kibana using .tar.gz

The Linux archive for Kibana v6.1.3 can be downloaded and installed as follows:

wget https://artifacts.elastic.co/downloads/kibana/kibana-6.1.3-linux-x86_64.tar.gz

Compare the SHA produced by sha1sum or shasum with the published SHA:

sha1sum kibana-6.1.3-linux-x86_64.tar.gz
tar -xzf kibana-6.1.3-linux-x86_64.tar.gz

This directory is known as $KIBANA_HOME: 

cd kibana-6.1.3-linux-x86_64/

Installing Kibana using the Debian package

Download and install the public signing key:

wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -

You may need to install the apt-transport-https package on Debian before proceeding:

sudo apt-get install apt-transport-https

Save the repository definition to /etc/apt/sources.list.d/elastic-6.x.list:

echo "deb https://artifacts.elastic.co/packages/6.x/apt stable main" | sudo tee -a /etc/apt/sources.list.d/elastic-6.x.list

You can install the Kibana Debian package with the following:

sudo apt-get update && sudo apt-get install kibana

Installing Kibana using rpm

Download and install the public signing key, as follows:

rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch

Create a file named kibana.repo in the /etc/yum.repos.d/ directory for Red Hat-based distributions, or in the /etc/zypp/repos.d/ directory for openSUSE-based distributions, containing the following code:

[kibana-6.x]
 name=Kibana repository for 6.x packages
 baseurl=https://artifacts.elastic.co/packages/6.x/yum
 gpgcheck=1
 gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
 enabled=1
 autorefresh=1
 type=rpm-md

Your repository is now ready for use. You can now install Kibana with one of the following commands:

• You can use yum on CentOS and older Red Hat-based distributions:

sudo yum install kibana

• You can use dnf on Fedora and other newer Red Hat distributions:

sudo dnf install kibana

• You can use zypper on openSUSE-based distributions:

sudo zypper install kibana

Installing Kibana on Windows

Download the .zip Windows archive for Kibana v6.1.3 from https://artifacts.elastic.co/downloads/kibana/kibana-6.1.3-windows-x86_64.zip.

Unzipping it will create a folder named kibana-6.1.3-windows-x86_64, which we will refer to as $KIBANA_HOME. In your Terminal, CD to the $KIBANA_HOME directory; for instance:

CD c:\kibana-6.1.3-windows-x86_64

Kibana can be started from the command line as follows:

.\bin\kibana

Beats

After installing and configuring the ELK Stack, you need to install and configure your Beats.

Each Beat is a separately installable product. To get up and running quickly with a Beat, see the getting started information for your Beat:

• Packetbeat
• Metricbeat
• Filebeat
• Winlogbeat
• Heartbeat

Packetbeat

The value of a network packet analytics system such as Packetbeat can be best understood by trying it on your traffic.

To download and install Packetbeat, use the commands that work with your system (deb for Debian/Ubuntu, rpm for Red Hat/CentOS/Fedora, macOS for OS X, Docker for any Docker platform, and win for Windows):

• Ubuntu:

sudo apt-get install libpcap0.8
curl -L -O https://artifacts.elastic.co/downloads/beats/packetbeat/packetbeat-6.2.1-amd64.deb
 sudo dpkg -i packetbeat-6.2.1-amd64.deb

• Red Hat:

sudo yum install libpcap
 curl -L -O https://artifacts.elastic.co/downloads/beats/packetbeat/packetbeat-6.2.1-x86_64.rpm
 sudo rpm -vi packetbeat-6.2.1-x86_64.rpm

• macOS:

curl -L -O https://artifacts.elastic.co/downloads/beats/packetbeat/packetbeat-6.2.1-darwin-x86_64.tar.gz
 tar xzvf packetbeat-6.2.1-darwin-x86_64.tar.gz

• Windows:
  1. Download and install WinPcap from this page. WinPcap is a library that uses a driver to enable packet capturing.
  2. Download the Packetbeat Windows ZIP file from the downloads page.
  3. Extract the contents of the ZIP file into C:\Program Files.
  4. Rename the packetbeat-<version>-windows directory to Packetbeat.
  5. Open a PowerShell prompt as an administrator (right-click the PowerShell icon and select Run as administrator). If you are running Windows XP, you may need to download and install PowerShell.
  6. From the PowerShell prompt, run the following commands to install Packetbeat as a Windows service:

PS > cd 'C:\Program Files\Packetbeat'
PS C:\Program Files\Packetbeat> .\install-service-packetbeat.ps1

Before starting Packetbeat, you should look at the configuration options in the configuration file; for example, C:\Program Files\Packetbeat\packetbeat.yml or /etc/packetbeat/packetbeat.yml.

Metricbeat

Metricbeat should be installed as close as possible to the service that needs to be monitored. For example, if there are four servers running MySQL, it's strongly recommended that you run Metricbeat on each service. This gives Metricbeat access to your service from localhost and in turn does not cause any additional network traffic or prevent Metricbeat from collecting metrics when there are network problems. Metrics from multiple Metricbeat instances will be combined on the Elasticsearch server.

To download and install Metricbeat, use the commands that work with your system (deb for Debian/Ubuntu, rpm for Red Hat/CentOS/Fedora, macOS for OS X, Docker for any Docker platform, and win for Windows), as follows:

• Ubuntu:

curl -L -O https://artifacts.elastic.co/downloads/beats/metricbeat/metricbeat-6.2.1-amd64.deb
 sudo dpkg -i metricbeat-6.2.1-amd64.deb

• Red Hat:

curl -L -O https://artifacts.elastic.co/downloads/beats/metricbeat/metricbeat-6.2.1-x86_64.rpm
 sudo rpm -vi metricbeat-6.2.1-x86_64.rpm

• macOS:

curl -L -O https://artifacts.elastic.co/downloads/beats/metricbeat/metricbeat-6.2.1-darwin-x86_64.tar.gz
 tar xzvf metricbeat-6.2.1-darwin-x86_64.tar.gz

• Windows:
  1. Download the Metricbeat Windows ZIP file from the downloads page.
  2. Extract the contents of the ZIP file into C:\Program Files.
  3. Rename the metricbeat-<version>-windows directory to Metricbeat.
  4. Open a PowerShell prompt as an administrator (right-click the PowerShell icon and select Run as administrator). If you are running Windows XP, you may need to download and install PowerShell.
  5. From the PowerShell prompt, run the following commands to install Metricbeat as a Windows service:

PS > cd 'C:\Program Files\Metricbeat'
PS C:\Program Files\Metricbeat> .\install-service-metricbeat.ps1

Before starting Metricbeat, you should look at the configuration options in the configuration file; for example, C:\Program Files\Metricbeat\metricbeat.yml.

Filebeat

To download and install Filebeat, use the commands that work with your system (deb for Debian/Ubuntu, rpm for Red Hat/CentOS/Fedora, macOS for OS X, Docker for any Docker platform, and win for Windows), as follows:

• Ubuntu:

curl -L -O https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-6.2.1-amd64.deb
 sudo dpkg -i filebeat-6.2.1-amd64.deb

• Red Hat:

curl -L -O https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-6.2.1-x86_64.rpm
 sudo rpm -vi filebeat-6.2.1-x86_64.rpm

• macOS:

curl -L -O https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-6.2.1-darwin-x86_64.tar.gz
 tar xzvf filebeat-6.2.1-darwin-x86_64.tar.gz

• Windows:
  1. Download the Filebeat Windows ZIP file from the downloads page.
  2. Extract the contents of the ZIP file into C:\Program Files.
  3. Rename the filebeat-<version>-windows directory to Filebeat.
  4. Open a PowerShell prompt as an administrator (right-click the PowerShell icon and select Run as administrator). If you are running Windows XP, you may need to download and install PowerShell.
  5. From the PowerShell prompt, run the following commands to install Filebeat as a Windows service:

PS > cd 'C:\Program Files\Filebeat'
PS C:\Program Files\Filebeat> .\install-service-filebeat.ps1

Winlogbeat

In order to install Winlogbeat, we need to follow these steps:

1. Download the Winlogbeat ZIP file from the downloads page.
2. Extract the contents into C:\Program Files.
3. Rename the winlogbeat-<version> directory to Winlogbeat.
4. Open a PowerShell prompt as an administrator (right-click on the PowerShell icon and select Run as administrator). If you are running Windows XP, you may need to download and install PowerShell.
5. From the PowerShell prompt, run the following commands to install the service:

PS C:\Users\Administrator> cd 'C:\Program Files\Winlogbeat'
PS C:\Program Files\Winlogbeat> .\install-service-winlogbeat.ps1

Security warning: Only run scripts that you trust. Although scripts from the internet can be useful, they can potentially harm your computer. If you trust the script, use Unblock-File  to allow the script to run without this warning message:

Do you want to run
 C:\Program Files\Winlogbeat\install-service-winlogbeat.ps1?
 [D] Do not run [R] Run once [S] Suspend [?] Help (default is "D"): R
 
 Status Name DisplayName
 ------ ---- -----------
 Stopped winlogbeat winlogbeat

Before starting winlogbeat, you should look at the configuration options in the configuration file; for example, C:\Program Files\Winlogbeat\winlogbeat.yml. There's also a full example configuration file named winlogbeat.reference.yml.

Heartbeat

Unlike most Beats, which we install on edge nodes, we typically install Heartbeat as part of a monitoring service that runs on a separate machine and possibly even outside of the network where the services that you want to monitor are running.

To download and install Heartbeat, use the commands that work with your system (deb for Debian/Ubuntu, rpm for Red Hat/CentOS/Fedora, macOS for OS X, Docker for any Docker platform, and win for Windows):

• Ubuntu:

curl -L -O https://artifacts.elastic.co/downloads/beats/heartbeat/heartbeat-6.2.1-amd64.deb
 sudo dpkg -i heartbeat-6.2.1-amd64.deb

• Red Hat:

curl -L -O https://artifacts.elastic.co/downloads/beats/heartbeat/heartbeat-6.2.1-x86_64.rpm
 sudo rpm -vi heartbeat-6.2.1-x86_64.rpm

• macOS:

curl -L -O https://artifacts.elastic.co/downloads/beats/heartbeat/heartbeat-6.2.1-darwin-x86_64.tar.gz
 tar xzvf heartbeat-6.2.1-darwin-x86_64.tar.gz

• Windows:
  1. Download the Heartbeat Windows ZIP file from the downloads page.
  2. Extract the contents of the ZIP file into C:\Program Files.
  3. Rename the heartbeat-<version>-windows directory to Heartbeat.
  4. Open a PowerShell prompt as an administrator (right-click the PowerShell icon and select Run as administrator). If you are running Windows XP, you may need to download and install PowerShell.
  5. From the PowerShell prompt, run the following commands to install Heartbeat as a Windows service:

PS > cd 'C:\Program Files\Heartbeat'
PS C:\Program Files\Heartbeat> .\install-service-heartbeat.ps1

 

Before starting Heartbeat, you should look at the configuration options in the configuration file; for example, C:\Program Files\Heartbeat\heartbeat.yml or /etc/heartbeat/heartbeat.yml.