Managing time-based indices efficiently using shrink and rollover APIs
Recently, we talked a lot about how to scale Elasticsearch clusters and some general guidelines to follow while going into production. In this section, we are going to talk about two new APIs introduced in Elasticsearch 5.0. The Shrink and Rollover APIs. Both of these APIs are specially designed for managing time series-based indices such as, daily-/weekly-/monthly-created indices for logs, or an index for each week or month of tweets.
We know these basic points related to shards of an index:
We need to define the number of shards in advance at the time of index creation and we can't increase or decrease the number of shards for index once it is created.
The greater the number of shards, the more indexing throughput, the lesser the search speed, and greater number of resources are needed.
Both of these problems may be an overkill for the performance and management of your cluster when the data size grows and scaling is needed...
