Downloading and installing Elasticsearch

Elasticsearch has an active community and the release cycles are very fast.

Because Elasticsearch depends on many common Java libraries (Lucene, Guice, and Jackson are the most famous ones), the Elasticsearch community tries to keep them updated and fixes bugs that are discovered in them and in the Elasticsearch core. The large user base is also a source of new ideas and features for improving Elasticsearch use cases.

For these reasons, if possible, it's best to use the latest available release (usually the more stable and bug-free one).

Getting ready

To install Elasticsearch, you need a supported operating system (Linux/macOS X/Windows) with a Java Java virtual machine (JVM) 1.8 or higher installed (the Sun Oracle JDK is preferred. More information on this can be found at http://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html). A web browser is required to download the Elasticsearch binary release. At least 1 GB of free disk space is required to install Elasticsearch.

How to do it…

We will start by downloading Elasticsearch from the web. The latest version is always downloadable at https://www.elastic.co/downloads/elasticsearch. The versions that are available for different operating systems are as follows:

• elasticsearch-{version-number}.zip and elasticsearch-{version-number}.msi are for the Windows operating systems.
• elasticsearch-{version-number}.tar.gz is for Linux/macOS X, while elasticsearch-{version-number}.deb is for Debian-based Linux distributions (this also covers the Ubuntu family); this is installable with Debian using the dpkg -i elasticsearch-*.deb command.
• elasticsearch-{version-number}.rpm is for Red Hat-based Linux distributions (this also covers the Cent OS family). This is installable with the rpm -i elasticsearch-*.rpm command.

The preceding packages contain everything to start Elasticsearch. This book targets version 7.x or higher. The latest and most stable version of Elasticsearch was 7.0.0. To check out whether this is the latest version or not, visit https://www.elastic.co/downloads/elasticsearch.

Extract the binary content. After downloading the correct release for your platform, the installation involves expanding the archive in a working directory.

Choose a working directory that is safe to charset problems and does not have a long path. This prevents problems when Elasticsearch creates its directories to store index data.

For the Windows platform, a good directory in which to install Elasticsearch could be c:\es, on Unix and /opt/es on macOS X.

To run Elasticsearch, you need a JVM 1.8 or higher installed. For better performance, I suggest that you use the latest Sun/Oracle version.

If you are a macOS X user and you have installed Homebrew (http://brew.sh/ ), the first and the second steps are automatically managed by the brew install elasticsearch command.

Let's start Elasticsearch to check if everything is working. To start your Elasticsearch server, just access the directory, and for Linux and macOS X execute the following:

# bin/elasticsearch

Alternatively, you can type the following command line for Windows:

# bin\elasticserch.bat

Your server should now start up and show logs similar to the following:

[2018-10-28T16:19:41,189][INFO ][o.e.n.Node ] [] initializing ...
 [2018-10-28T16:19:41,245][INFO ][o.e.e.NodeEnvironment ] [fyBySLM] using [1] data paths, mounts [[/ (/dev/disk1s1)]], net usable_space [141.9gb], net total_space [465.6gb], types [apfs]
 [2018-10-28T16:19:41,246][INFO ][o.e.e.NodeEnvironment ] [fyBySLM] heap size [989.8mb], compressed ordinary object pointers [true]
 [2018-10-28T16:19:41,247][INFO ][o.e.n.Node ] [fyBySLM] node name derived from node ID [fyBySLMcR3uqKiYC32P5Sg]; set [node.name] to override
 [2018-10-28T16:19:41,247][INFO ][o.e.n.Node ] [fyBySLM] version[6.4.2], pid[50238], build[default/tar/04711c2/2018-09-26T13:34:09.098244Z], OS[Mac OS X/10.14/x86_64], JVM[Oracle Corporation/Java HotSpot(TM) 64-Bit Server VM/1.8.0_181/25.181-b13]
 [2018-10-28T16:19:41,247][INFO ][o.e.n.Node ] [fyBySLM] JVM arguments [-Xms1g, -Xmx1g,
... truncated ...
 [2018-10-28T16:19:42,511][INFO ][o.e.p.PluginsService ] [fyBySLM] loaded module [aggs-matrix-stats]
 [2018-10-28T16:19:42,511][INFO ][o.e.p.PluginsService ] [fyBySLM] loaded module [analysis-common]
 ...truncated...
[2018-10-28T16:19:42,513][INFO ][o.e.p.PluginsService ] [fyBySLM] no plugins loaded
 ...truncated...
[2018-10-28T16:19:46,776][INFO ][o.e.n.Node ] [fyBySLM] initialized
 [2018-10-28T16:19:46,777][INFO ][o.e.n.Node ] [fyBySLM] starting ...
 [2018-10-28T16:19:46,930][INFO ][o.e.t.TransportService ] [fyBySLM] publish_address {127.0.0.1:9300}, bound_addresses {[::1]:9300}, {127.0.0.1:9300}
 [2018-10-28T16:19:49,983][INFO ][o.e.c.s.MasterService ] [fyBySLM] zen-disco-elected-as-master ([0] nodes joined)[, ], reason: new_master {fyBySLM}{fyBySLMcR3uqKiYC32P5Sg}{-pUWNdRlTwKuhv89iQ6psg}{127.0.0.1}{127.0.0.1:9300}{ml.machine_memory=17179869184, xpack.installed=true, ml.max_open_jobs=20, ml.enabled=true}
 ...truncated...
[2018-10-28T16:19:50,452][INFO ][o.e.l.LicenseService ] [fyBySLM] license [b2754b17-a4ec-47e4-9175-4b2e0d714a45] mode [basic] - valid

How it works…

The Elasticsearch package generally contains the following directories:

• bin: This contains the scripts to start and manage Elasticsearch.
• elasticsearch.bat: This is the main executable script to start Elasticsearch.
• elasticsearch-plugin.bat: This is a script to manage plugins.
• config: This contains the Elasticsearch configs. The most important ones are as follows:
  • elasticsearch.yml: This is the main config file for Elasticsearch
  • log4j2.properties: This is the logging config file
• lib: This contains all the libraries required to run Elasticsearch.
• logs: This directory is empty at installation time, but in the future, it will contain the application logs.
• modules: This contains the Elasticsearch default plugin modules.
• plugins: This directory is empty at installation time, but it's the place where custom plugins will be installed.

During Elasticsearch startup, the following events happen:

• A node name is generated automatically (that is, fyBySLM) if it is not provided in elasticsearch.yml. The name is randomly generated, so it's a good idea to set it to a meaningful and memorable name instead.
• A node name hash is generated for this node, for example, fyBySLMcR3uqKiYC32P5Sg.
• The default installed modules are loaded. The most important ones are as follows:
  • aggs-matrix-stats: This provides support for aggregation matrix stats.
  • analysis-common: This is a common analyzer for Elasticsearch, which extends the language processing capabilities of Elasticsearch.
  • ingest-common: These include common functionalities for the ingest module.
  • lang-expression/lang-mustache/lang-painless: These are the default supported scripting languages of Elasticsearch. 
  • mapper-extras: This provides an extra mapper type to be used, such as token_count and scaled_float.
  • parent-join: This provides an extra query, such as has_children and has_parent.
  • percolator: This provides percolator capabilities.
  • rank-eval: This provides support for the experimental rank evaluation APIs. These are used to evaluate hit scoring based on queries.
  • reindex: This provides support for reindex actions (reindex/update by query).
  • x-pack-*: All the xpack modules depend on a subscription for their activation.
• If there are plugins, they are loaded.
• If not configured, Elasticsearch binds the following two ports on the localhost 127.0.0.1 automatically:
  • 9300: This port is used for internal intranode communication.
  • 9200: This port is used for the HTTP REST API.
• After starting, if indices are available, they are restored and ready to be used.

If these port numbers are already bound, Elasticsearch automatically increments the port number and tries to bind on them until a port is available (that is, 9201, 9202, and so on).

There are more events that are fired during Elasticsearch startup. We'll see them in detail in other recipes.

There's more…

During a node's startup, a lot of required services are automatically started. The most important ones are as follows:

• Cluster services: This helps you manage the cluster state and intranode communication and synchronization
• Indexing service: This helps you manage all the index operations, initializing all active indices and shards
• Mapping service: This helps you manage the document types stored in the cluster (we'll discuss mapping in Chapter 2, Managing Mapping)
• Network services: This includes services such as HTTP REST services (default on port 9200), and internal Elasticsearch protocol (port 9300) if the thrift plugin is installed
• Plugin service: This manages loading the plugin 
• Aggregation services: This provides advanced analytics on stored Elasticsearch documents such as statistics, histograms, and document grouping
• Ingesting services: This provides support for document preprocessing before ingestion such as field enrichment, NLP processing, types conversion, and automatic field population
• Language scripting services: This allows you to add new language scripting support to Elasticsearch

See also

The Setting up networking recipe we're going to cover next will help you with the initial network setup. Check the official Elasticsearch download page at https://www.elastic.co/downloads/elasticsearch to get the latest version.