Installing and Configuring X-pack on Elasticsearch and Kibana

[box type="note" align="" class="" width=""]This article is an excerpt from a book written by Pranav Shukla and Sharath Kumar M N titled Learning Elastic Stack 6.0. This book provides detailed coverage on fundamentals of Elastic Stack, making it easy to search, analyze and visualize data across different sources in real-time.[/box]

In this short tutorial, we will show step-by-step installation and configuration of X-pack components in Elastic Stack to extend the functionalities of Elasticsearch and Kibana.

As X-Pack is an extension of Elastic Stack, prior to installing X-Pack, you need to have both Elasticsearch and Kibana installed. You must run the version of X-Pack that matches the version of Elasticsearch and Kibana.

Installing X-Pack on Elasticsearch

X-Pack is installed just like any plugin to extend Elasticsearch.

These are the steps to install X-Pack in Elasticsearch:

1. Navigate to the ES_HOME folder.

2. Install X-Pack using the following command:

$ ES_HOME> bin/elasticsearch-plugin install x-pack

During installation, it will ask you to grant extra permissions to X-Pack, which are required by Watcher to send email alerts and also to enable Elasticsearch to launch the machine learning analytical engine. Specify y to continue the installation or N to abort the installation.

You should get the following logs/prompts during installation:

-> Downloading x-pack from elastic

[=================================================] 100%

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

@ WARNING: plugin requires additional permissions @

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

* java.io.FilePermission .pipe* read,write

* java.lang.RuntimePermissionaccessClassInPackage.com.sun.activation.registries

* java.lang.RuntimePermission getClassLoader

* java.lang.RuntimePermission setContextClassLoader

* java.lang.RuntimePermission setFactory

* java.net.SocketPermission * connect,accept,resolve

* java.security.SecurityPermission createPolicy.JavaPolicy

* java.security.SecurityPermission getPolicy

* java.security.SecurityPermission putProviderProperty.BC

* java.security.SecurityPermission setPolicy

* java.util.PropertyPermission * read,write

* java.util.PropertyPermission sun.nio.ch.bugLevel write

See http://docs.oracle.com/javase/8/docs/technotes/guides/security/permissions.html for descriptions of what these permissions allow and the associated Risks.

Continue with installation? [y/N]y

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

@ WARNING: plugin forks a native controller @

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

This plugin launches a native controller that is not subject to the Java security manager nor to system call filters.

Continue with installation? [y/N]y

Elasticsearch keystore is required by plugin [x-pack], creating...

-> Installed x-pack

3. Restart Elasticsearch:

$ ES_HOME> bin/elasticsearch

4. Generate the passwords for the default/reserved users—elastic, kibana, and logstash_system—by executing this command:

$ ES_HOME>bin/x-pack/setup-passwords interactive

You should get the following logs/prompts to enter the password for the reserved/default users:

Initiating the setup of reserved user elastic,kibana,logstash_system passwords. You will be prompted to enter passwords as the process progresses.

Please confirm that you would like to continue [y/N]y

Enter password for [elastic]: elastic

Reenter password for [elastic]: elastic

Enter password for [kibana]: kibana

Reenter password for [kibana]:kibana

Enter password for [logstash_system]: logstash

Reenter password for [logstash_system]: logstash

Changed password for user [kibana]

Changed password for user [logstash_system]

Changed password for user [elastic]

Please make a note of the passwords set for the reserved/default users. You can choose any password of your liking. We have chosen the passwords as elastic, kibana, and logstash for elastic, kibana, and logstash_system users, respectively, and we will be using them throughout this chapter.

To verify the X-Pack installation and enforcement of security, point your web browser to http://localhost:9200/ to open Elasticsearch. You should be prompted to log in to Elasticsearch. To log in, you can use the built-in elastic user and the password elastic.

Upon a successful log in, you should see the following response:

{

name: "fwDdHSI",

cluster_name: "elasticsearch",

cluster_uuid: "08wSPsjSQCmeRaxF4iHizw",

version: {

number: "6.0.0",

build_hash: "8f0685b",

build_date: "2017-11-10T18:41:22.859Z",

build_snapshot: false,

lucene_version: "7.0.1",

minimum_wire_compatibility_version: "5.6.0",

minimum_index_compatibility_version: "5.0.0"

},

tagline: "You Know, for Search"

}

A typical cluster in Elasticsearch is made up of multiple nodes, and X-Pack needs to be installed on each node belonging to the cluster.

To skip the install prompt, use the—batch parameters during installation:

$ES_HOME>bin/elasticsearch-plugin install x-pack --batch.

Your installation of X-Pack will have created folders named x-pack in bin, config, and plugins found under ES_HOME. We shall explore these in later sections of the chapter.

Installing X-Pack on Kibana

X-Pack is installed just like any plugins to extend Kibana.

The following are the steps to install X-Pack in Kibana:

1. Navigate to the KIBANA_HOME folder.
2. Install X-Pack using the following command:

$KIBANA_HOME>bin/kibana-plugin install x-pack

You should get the following logs/prompts during installation:

Attempting to transfer from x-pack

Attempting to transfer from

https://artifacts.elastic.co/downloads/kibana-plugins/x-pack/x-pack

-6.0.0.zip

Transferring 120307264 bytes....................

Transfer complete

Retrieving metadata from plugin archive

Extracting plugin archive

Extraction complete

Optimizing and caching browser bundles...

Plugin installation complete

3. Add the following credentials in the kibana.yml file found under $KIBANA_HOME/config and save it:

elasticsearch.username: "kibana"

elasticsearch.password: "kibana"

If you have chosen a different password for the kibana user during password setup, use that value for the elasticsearch.password property.

4. Start Kibana:

$KIBANA_HOME>bin/kibana

To verify the X-Pack installation, go to http://localhost:5601/ to open Kibana. You should be prompted to log in to Kibana. To log in, you can use the built-in elastic user and the password elastic.

Your installation of X-Pack will have created a folder named x-pack in the plugins folder found under KIBANA_HOME.

You can also optionally install X-Pack on Logstash. However, X-Pack currently supports only monitoring of Logstash.

Uninstalling X-Pack

To uninstall X-Pack:

1. Stop Elasticsearch.
2. Remove X-Pack from Elasticsearch:

$ES_HOME>bin/elasticsearch-plugin remove x-pack

3. Restart Elasticsearch and stop Kibana 2. Remove X-Pack from Kibana:

$KIBANA_HOME>bin/kibana-plugin remove x-pack

4. Restart Kibana.

Configuring X-Pack

X-Pack comes bundled with security, alerting, monitoring, reporting, machine learning, and graph capabilities. By default, all of these features are enabled. However, one might not be interested in all the features it provides. One can selectively enable and disable the features that they are interested in from the elasticsearch.yml and kibana.yml configuration files.

Elasticsearch supports the following features and settings in the elasticsearch.yml file:

Kibana supports these features and settings in the kibana.yml file:

If X-Pack is installed on Logstash, you can disable the monitoring by setting the xpack.monitoring.enabled property to false in the logstash.yml configuration file.  

With this, we successfully explored how to install and configure the X-Pack components in order to bundle different capabilities of X-pack into one package of Elasticsearch and Kibana.

If you found this tutorial useful, do check out the book Learning Elastic Stack 6.0 to examine the fundamentals of Elastic Stack in detail and start developing solutions for problems like logging, site search, app search, metrics and more.