Elastic Cloud is the most straightforward way to deploy and manage your Elasticsearch, Kibana, Integrations Server (a combined component for the application performance monitoring server and Fleet Server), and other components of the Elastic Stack. This recipe will guide you through the process of getting started with Elastic Cloud, from signing up for an account to creating your first Elastic deployment.

How to do it…

Before we begin, let’s learn how to create a deployment on Elastic Cloud and verify it using this step-by-step guide:

1. We will create an account on Elastic Cloud:
   1. Visit the Elastic Cloud website at https://cloud.elastic.co/.
   2. Click on the Sign up button (a 14-day trial without needing a credit card is offered by default).
   3. Fill out the registration form with your details, including your name, email address, and desired password.

   Next, we will create a deployment.

2. On the next screen, you’ll be...

ECK is the official Kubernetes operator for automating the deployment and management of Elasticsearch and other Elastic components on Kubernetes. ECK enables the use of Kubernetes-native tools and APIs to manage Elasticsearch clusters, offering capabilities for monitoring and securing them. It supports scaling, rolling upgrades, availability zone awareness, and the implementation of hot-warm-cold storage architectures. ECK allows for the exploitation of Elasticsearch’s power and flexibility on Kubernetes, both on-premises and in the cloud. In this guide, we will first install the ECK operator in a Kubernetes cluster and then use it to deploy an Elasticsearch cluster and Kibana.

Technical requirements

Ensure you have a Kubernetes cluster ready before deploying ECK and the Elastic Stack. For this recipe, you can use either minikube or Google Kubernetes Engine (GKE). Elastic Cloud on Kubernetes also supports other Kubernetes distributions...

In this recipe, you will learn how to install and manage the Elastic Stack on your local machine, focusing primarily on the essential components: Elasticsearch and Kibana.

Getting ready

Before proceeding with the installation, make sure your system meets the minimum requirements for running Elasticsearch, Kibana, and Fleet Server. Check the official documentation for the specific version you want to install to ensure compatibility with your operating system (https://www.elastic.co/support/matrix).

How to do it…

Let’s first look at how to download Elasticsearch:

1. Visit the Elasticsearch download page (https://www.elastic.co/downloads/elasticsearch).
2. By default, the official Elasticsearch download page provides you with the download links for the latest release. Choose the right package for your operating system.
3. Once the download is complete, extract the contents of the package to a working directory of...

A data tier consists of several Elasticsearch nodes that have the same data role and usually run on similar hardware. Often, different hardware is configured for each tier; for example, the hot tier might use the most powerful and expensive hardware, while the cold or frozen tiers could utilize less expensive, storage-oriented hardware. Using data tiers is an efficient strategy for reducing hardware requirements in an Elasticsearch cluster while maintaining access to data and the ability to search through it. To illustrate, a single frozen node can keep up to 100 TB of data compared to 2 TB of data for a hot node.

However, there is a caveat: as data moves to colder tiers, query performance can decrease. This is expected since the data is less frequently queried.

Figure 1.16 – Elasticsearch data tiering

As we can see in Figure 1.16, there are four data tiers provided by Elasticsearch:

• Hot tier: This...

An Elasticsearch cluster can have a variety of node roles, besides data tiers, to function efficiently. Figure 1.18 outlines the several types of nodes available in a cluster:

Figure 1.18 – Elasticsearch node types

Roles such as Master, Machine Learning, or Ingest can be dedicated to specific Elasticsearch instances, and this is often a best practice in a production environment.

In this recipe, we will learn how to configure dedicated nodes for both self-managed deployments and Elastic Cloud.

Getting ready

Ensure that your self-managed cluster from the previous recipe is operational. For simplicity, we will create additional nodes on the same local machine. The nodes will undertake the following roles:

• A dedicated master eligible node
• A machine learning node

The snippets for this recipe are available at https://github.com/PacktPublishing/Elastic-Stack-8.x-Cookbook...

Fleet Server is a key component of the new ingest architecture in the Elastic Stack, which revolves around the Elastic Agent. Before delving into this recipe, let’s review some important concepts about Fleet and the Agent.

Fleet serves as the central management component, providing a UI within Kibana that manages Agents and their configurations at scale. The Elastic Agent is a single, unified binary responsible for data collection tasks – gathering logs, metrics, security events, and more, running on your hosts.

Fleet Server connects the Elastic Agent to Fleet and acts as a control plane for Elastic Agents. It is an essential piece if you intend to use Fleet for centralized management. The schema in Figure 1.19 illustrates the various components and their interactions:

Figure 1.19 – Architecture including Elastic Agent and Fleet Server

In this recipe, we’ll cover the setup of Fleet...

After you’ve set up a functional Elastic cluster, we recommend setting up a snapshot repository according to your deployment method. This allows you to back up your valuable data. Elasticsearch features a native capability for data backup and restoration.

When you create a deployment on Elastic Cloud, it comes also with a default repository called found repository. In this recipe, you’ll learn how to register and manage a snapshot repository for an Amazon S3 bucket with Elastic Cloud, a popular option. The setup concepts can also apply to other cloud repositories, such as Google Cloud Storage or Azure Blob Storage, and self-managed repositories.

Later in the book, we will provide a guide on how to configure and execute snapshot and restore operations.

Getting ready

Make sure that your Elastic Cloud deployment is up and running and that you have sufficient permissions to create and configure S3 buckets on AWS.

How to do it...