Paginate Deep Elasticsearch Results with search_after vs Scroll (2026)

search_after is fundamentally a client-side pagination mechanism, while scroll is a server-side snapshotting mechanism, and understanding this difference is key to choosing the right tool.

Let’s see search_after in action. Imagine you have a massive Elasticsearch index of user activity logs, and you want to display the most recent actions to your users, one page at a time.

// First request (page 1)
GET /user_logs/_search
{
  "size": 10,
  "query": {
    "match_all": {}
  },
  "sort": [
    {"timestamp": "desc"},
    {"_id": "desc"}
  ]
}

// Response for page 1
{
  "took": 5,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": {
      "value": 15000,
      "relation": "eq"
    },
    "max_score": null,
    "hits": [
      {
        "_index": "user_logs",
        "_type": "_doc",
        "_id": "log_12345",
        "_score": null,
        "_source": {
          "user": "alice",
          "action": "login",
          "timestamp": "2023-10-27T10:00:00Z"
        },
        "sort": [
          "2023-10-27T10:00:00Z",
          "log_12345"
        ]
      },
      // ... 9 more hits
    ]
  }
}

// To get the next page, you take the 'sort' values from the LAST hit of the previous page
// and use them in the 'search_after' parameter.
// The 'sort' order MUST be identical to the previous request.
// Request for page 2
GET /user_logs/_search
{
  "size": 10,
  "query": {
    "match_all": {}
  },
  "sort": [
    {"timestamp": "desc"},
    {"_id": "desc"}
  ],
  "search_after": ["2023-10-27T09:59:59Z", "log_12300"] // Values from the last hit of page 1
}

The problem search_after and scroll solve is deep pagination. Standard from and size in Elasticsearch gets inefficient and can even fail for large offsets (e.g., from: 10000, size: 10). This is because Elasticsearch has to fetch from + size documents from each shard, sort them all, and then discard the first from documents. This is a lot of wasted work.

search_after addresses this by using the sort values from the last document of the previous page to tell Elasticsearch where to start for the next page. It’s like saying, "give me the next 10 items that come after this specific item, based on our sorting criteria." This is incredibly efficient for deep pages because Elasticsearch only needs to find the documents that satisfy the search_after condition and then collect the next size documents. It doesn’t need to sort a massive intermediate set. This makes search_after ideal for real-time user interfaces where users are browsing through results sequentially.

The scroll API, on the other hand, is designed for bulk data retrieval. When you initiate a scroll request, Elasticsearch takes a "snapshot" of the index at that moment and keeps it open. You then make subsequent scroll requests, passing a scroll_id that refers to that snapshot. Elasticsearch efficiently retrieves the next batch of documents from its internal snapshot. This is perfect for batch jobs, exporting data, or reindexing because it guarantees you’re processing a consistent view of the data, even if the index is being updated concurrently.

The core difference boils down to state management: search_after is stateless on the server side for each request; it just needs the previous sort values. scroll is stateful; it maintains an open search context on the server.

Here’s a crucial detail about search_after: the sort criteria must be unique. If multiple documents have identical sort values, Elasticsearch doesn’t have a deterministic way to order them for search_after. To ensure uniqueness, you should always include a tie-breaker field in your sort, such as the _id field, as shown in the example. This guarantees that even if timestamps are identical, the document IDs will differentiate them, providing a stable ordering for pagination.

The primary limitation of scroll is that it holds resources on the Elasticsearch cluster. If you don’t clear the scroll context (either by completing all requests or by explicitly calling DELETE /_search/scroll/<scroll_id>), these contexts can consume memory and file descriptors, potentially impacting cluster stability. Always remember to clear your scrolls when you’re done.

When you’ve exhausted all results with search_after, your next request will return an empty hits.hits array. For scroll, you’ll eventually receive a response with hits.hits being empty, and the scrollId will no longer be valid.