Deploying on Elasticsearch Cloud Enterprise

The Elasticsearch company provides Elasticsearch Cloud Enterprise (ECE), which is the same tool that's used in the Elasticsearch Cloud (https://www.elastic.co/cloud) and is offered for free. This solution, which is available on PAAS on AWS or GCP (Google Cloud Platform), can be installed on-premise to provide an enterprise solution on top of Elasticsearch.

If you need to manage multiple elastic deployments across teams or geographies, you can leverage ECE to centralize deployment management for the following functions:

• Provisioning
• Monitoring
• Scaling
• Replication
• Upgrades
• Backup and restoring

Centralizing the management of deployments with ECE enforces uniform versioning, data governance, backup, and user policies. Increased hardware utilization through better management can also reduce the total cost.

Getting ready

As this solution targets large installations of many servers, the minimum testing requirement is an 8 GB RAM node. The ECE solution lives at the top of Docker and must be installed on the nodes.

ECE supports only some operative systems, such as the following:

• Ubuntu 16.04 with Docker 18.03
• Ubuntu 14.04 with Docker 1.11
• RHEL/CentOS 7+ with Red Hat Docker 1.13

On other configurations, the ECE could work, but it is not supported in case of issues.

How to do it…

Before installing ECE, the following prerequisities are to be checked:

1. Your user must be a Docker enabled one. In the case of an error due to a non-Docker user, add your user with sudo usermod -aG docker $USER.
2. In the case of an error when you try to access /mnt/data, give your user permission to access this directory.
3. You need to add the following line to your /etc/sysctl.conf (a reboot is required): vm.max_map_count = 262144.

4. To be able to use the ECE, it must initially be installed on the first host, as follows:

bash <(curl -fsSL https://download.elastic.co/cloud/elastic-cloud-enterprise.sh) install

The installation process should manage these steps automatically, as shown in the following screenshot:

At the end, the installer should provide your credentials so that you can access your cluster in a similar output, as follows:

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 Elastic Cloud Enterprise installation completed successfully
Ready to copy down some important information and keep it safe?
Now you can access the Cloud UI using the following addresses:
http://192.168.1.244:12400
https://192.168.1.244:12443

Admin username: admin
Password: OCqHHqvF0JazwXPm48wfEHTKN0euEtn9YWyWe1gwbs8
Read-only username: readonly
Password: M27hoE3z3v6x5xyHnNleE5nboCDK43X9KoNJ346MEqO

Roles tokens for adding hosts to this installation:
Basic token (Don't forget to assign roles to new runners in the Cloud UI after installation.)
eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJiZDI3NjZjZi1iNWExLTQ4YTYtYTRlZi1iYzE4NTlkYjQ5ZmEiLCJyb2xlcyI6W10sImlzcyI6ImN1cnJlbnQiLCJwZXJzaXN0ZW50Ijp0cnVlfQ.lbh9oYPiJjpy7gI3I-_yFBz9T0blwNbbwtWF_-c_D3M

Allocator token (Simply need more capacity to run Elasticsearch clusters and Kibana? Use this token.)
eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJjYTk4ZDgyNi1iMWYwLTRkZmYtODBjYS0wYWYwMTM3M2MyOWYiLCJyb2xlcyI6WyJhbGxvY2F0b3IiXSwiaXNzIjoiY3VycmVudCIsInBlcnNpc3RlbnQiOnRydWV9.v9uvTKO3zgaE4nr0SDfg6ePrpperIGtvcGVfZHtmZmY
Emergency token (Lost all of your coordinators? This token will save your installation.)
eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOiI5N2ExMzg5Yi1jZWE4LTQ2MGItODM1ZC00MDMzZDllNjAyMmUiLCJyb2xlcyI6WyJjb29yZGluYXRvciIsInByb3h5IiwiZGlyZWN0b3IiXSwiaXNzIjoiY3VycmVudCIsInBlcnNpc3RlbnQiOnRydWV9._0IvJrBQ7RkqzFyeFGhSAQxyjCbpOO15qZqhzH2crZQ

To add hosts to this Elastic Cloud Enterprise installation, include the following parameters when you install the software
on additional hosts: --coordinator-host 192.168.1.244 --roles-token 'eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJiZDI3NjZjZi1iNWExLTQ4YTYtYTRlZi1iYzE4NTlkYjQ5ZmEiLCJyb2xlcyI6W10sImlzcyI6ImN1cnJlbnQiLCJwZXJzaXN0ZW50Ijp0cnVlfQ.lbh9oYPiJjpy7gI3I-_yFBz9T0blwNbbwtWF_-c_D3M'

These instructions use the basic token, but you can substitute one of the other tokens provided. You can also generate your own tokens. For example:
curl -H 'Content-Type: application/json' -u
admin: OCqHHqvF0JazwXPm48wfEHTKN0euEtn9YWyWe1gwbs8 http://192.168.1.244:12300/api/v1/platform/configuration/security/enrollment-tokens -d '{ "persistent": true, "roles": [ "allocator"] }'

To learn more about generating tokens, see Generate Role Tokens in the documentation.

System secrets have been generated and stored in /mnt/data/elastic/bootstrap-state/bootstrap-secrets.json.
Keep the information in the bootstrap-secrets.json file secure by removing the file and placing it into secure storage, for example.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

5. In my case, I can access the installed interface at http://192.168.1.244:12400.

After logging into the admin interface, you will see your actual cloud state, as follows:

6. You can now press on Create Deployment to fire your first Elasticsearch cluster, as follows:

7. You need to define a name (that is, a book-cluster). Using standard options for this is okay. After pressing Create Deployment, ECE will start to build your cluster, as follows:

8. After a few minutes, the cluster should be up and running, as follows:

How it works…

Elasticsearch Cloud Enterprise allows you to manage a large Elasticsearch cloud service that can create an instance via deployments. By default, the standard deployment will fire an ElasticSearch node with 4 GB RAM, 32 GB disk, and a Kibana instance.

You can define a lot of parameters during the deployments for ElasticSearch, such as the following:

• The RAM used for instances from 1 GB to 64 GB. The storage is proportional to the memory, so you can go from 1 GB RAM and 128 GB storage to 64 GB RAM and 2 TB storage.
• If the node requires ML.
• Master configurations if you have more than six data nodes.
• The plugins that are required to be installed.

For Kibana, you can only configure the memory (from 1 GB to 8 GB) and pass extra parameters (usually used for custom maps).

ECE does all the provisioning and, if you want a monitoring component and other X-Pack features, it's able to autoconfigure your cluster to manage all the required functionalities.

Elasticsearch Cloud Enterprise is very useful if you need to manage several Elasticsearch/Kibana clusters, because it leverages all the infrastructure problems.

A benefit of using a deployed Elasticsearch cluster is that, during deployment, a proxy is installed. This is very handy for managing the debugging of Elasticsearch calls.

See also

You can refer to the following links for further information on the topics that were covered in this recipe:

• Visit https://www.elastic.co/cloud for a PAAS managed cloud provider for ElasticSearch
• The complete documentation of Elasticsearch Cloud Enterprise at https://www.elastic.co/guide/en/cloud-enterprise/current/index.html
• The ECE documentation for monitoring integrating Elastic Beats at https://www.elastic.co/guide/en/cloud-enterprise/2.0/ece-cloud-id.html