Installing plugins in Elasticsearch

One of the main features of Elasticsearch is the possibility to extend it with plugins. Plugins extend Elasticsearch features and functionalities in several ways.

In Elasticsearch, these plugins are native plugins. These are JAR files that contain application code, and are used for the following reasons:

• Script engines
• Custom analyzers, tokenizers, and scoring
• Custom mapping
• REST entry points
• Ingestion pipeline stages
• Supporting new storages (Hadoop, GCP Cloud Storage)
• Extending X-Pack (that is, with a custom authorization provider)

Getting ready

You need a working Elasticsearch installation, as we described in the Downloading and installing Elasticsearch recipe, as well as a prompt/shell to execute commands in the Elasticsearch install directory.

How to do it…

Elasticsearch provides a script for automatic downloads and for the installation of plugins in bin/directory called elasticsearch-plugin.

The steps that are required to install a plugin are as follows:

1. Calling the plugin and installing the Elasticsearch command with the plugin name reference.

For installing the ingested attachment plugin used to extract text from files, simply call and type the following command if you're using Linux:

bin/elasticsearch-plugin install ingest-attachment

And for Windows, type the following command:

elasticsearch-plugin.bat install ingest-attachment

2. If the plugin needs to change security permissions, a warning is prompted and you need to accept this if you want to continue.
3. During the node's startup, check that the plugin is correctly loaded.

In the following screenshot, you can see the installation and the startup of the Elasticsearch server, along with the installed plugin:

Remember that a plugin installation requires an Elasticsearch server restart.

How it works…

The elasticsearch-plugin.bat script is a wrapper for the Elasticsearch plugin manager. This can be used to install or remove a plugin (using the remove options).

There are several ways to install the plugin, for example:

• Passing the URL of the plugin (ZIP archive), as follows:

bin/elasticsearch-plugin install http://mywoderfulserve.com/plugins/awesome-plugin.zip

• Passing the file path of the plugin (ZIP archive), as follows:

bin/elasticsearch-plugin install file:///tmp/awesome-plugin.zip

• Using the install parameter with the GitHub repository of the plugin. The install parameter, which must be given, is formatted in the following way:

<username>/<repo>[/<version>]

During the installation process, Elasticsearch plugin manager is able to do the following:

• Download the plugin
• Create a plugins directory in ES_HOME/plugins, if it's missing
• Optionally, ask if the plugin wants special permission to be executed
• Unzip the plugin content in the plugin directory
• Remove temporary files

The installation process is completely automatic; no further actions are required. The user must only pay attention to the fact that the process ends with an Installed message to be sure that the install process has completed correctly.

Restarting the server is always required to be sure that the plugin is correctly loaded by Elasticsearch.

There's more…

If your current Elasticsearch application depends on one or more plugins, a node can be configured to start up only if these plugins are installed and available. To achieve this behavior, you can provide the plugin.mandatory directive in the elasticsearch.yml configuration file.

For the previous example (ingest-attachment), the config line to be added is as follows:

plugin.mandatory:ingest-attachment

There are also some hints to remember while installing plugins: updating some plugins in a node environment can cause malfunctions due to different plugin versions in different nodes. If you have a big cluster for safety, it's better to check for updates in a separate environment to prevent problems (and remember to upgrade the plugin in all the nodes).

To prevent the fact updating an Elasticsearch version server which could also break your custom binary plugins due to some internal API changes, in Elasticsearch 5.x or higher, the plugins need to have the same version of Elasticsearch server in their manifest.

Upgrading an Elasticsearch server version means upgrading all the installed plugins.

See also

On the Elasticsearch site, there is an updated list of available plugins: https://www.elastic.co/guide/en/elasticsearch/plugins/current/index.html.

The actual Elasticsearch documentation doesn't cover all available plugins. I suggest going to GitHub (https://github.com) and searching for them with these or similar queries: elasticsearch plugin, elasticsearch lang, and elasticsearch ingest.