The scroll API

Let's imagine that we have an index with several million documents. We already know how to build our query, when to use filters, and so on. But looking at the query logs, we see that a particular kind of query is significantly slower than the others. These queries may be using pagination. The from parameter indicates that the offsets have large values. From the application side, this can mean that users go through an enormous number of results. Often, this doesn't make sense—if a user doesn't find the desirable results on the first few pages, he/she gives up. Because this particular activity can mean something bad (possible data stealing), many applications limit the paging to dozens of pages. In our case, we assume that this is a different scenario, and we have to provide this functionality.

Problem definition

When Elasticsearch generates a response, it must determine the order of the documents that form the result. If we are on the first page, this is not a big problem. Elasticsearch...