Introduction

In the Elasticsearch ecosystem, it's important to monitor nodes and clusters to manage and improve their performance and state. There are several issues that can arise at cluster level, such as:

• Node overheads: Some nodes can have too many shards allocated and become a bottleneck for the entire cluster

• Node shutdown: This can happen for many reasons, for example, full disks, hardware failures, and power problems

• Shard relocation problems or corruptions: Some shards cannot get an online status

• Too large shards: If a shard is too big, the index performance decreases due to massive Lucene segments merging

• Empty indices and shards: They waste memory and resources, but because every shard has a lot of active thread, if there are a huge number of unused indices and shards, the general cluster performance is degraded

Detecting malfunctioning or poor performance can be done via an API or through some frontends, as we will see in Chapter 12, User Interfaces. These allow the readers to have...