Elasticsearch API keys are a secure and granular way to grant access to your Elasticsearch clusters without sharing user credentials.
Let’s see how this works in practice. Imagine you have a Python application that needs to query an Elasticsearch index. Instead of embedding a username and password, you’ll use an API key.
First, you need to generate an API key within Elasticsearch. You can do this via Kibana’s Stack Management section under "API Keys" or using the elasticsearch-create-api-key tool.
bin/elasticsearch-create-api-key \
--name my_app_key \
--role reader \
--expiration 1d \
--id my_app_id
This command creates an API key named my_app_key with the reader role, valid for 1 day, and assigns it a specific ID my_app_id. The output will give you an id and a key. Treat the key like a password; it’s only shown once.
Now, in your Python application, you’ll use the elasticsearch-py client. You configure it with the API key’s ID and key:
from elasticsearch import Elasticsearch
es = Elasticsearch(
"https://localhost:9200",
api_key=("my_app_id", "YOUR_API_KEY_SECRET"),
verify_certs=False, # For development/testing, set to True in production
request_timeout=60
)
# Now you can perform operations
try:
response = es.search(index="my_index", query={"match_all": {}})
print(response)
except Exception as e:
print(f"An error occurred: {e}")
The api_key parameter takes a tuple: (api_key_id, api_key_secret). The elasticsearch-py client automatically formats this into the necessary Authorization: ApiKey base64(id:key) header. The verify_certs=False is a common shortcut for local development, but in production, you’d want to properly configure certificate verification to ensure you’re talking to your actual Elasticsearch cluster.
The real power comes from the granular control you have over these keys. When you create an API key, you can assign it specific roles. These roles define what actions the API key can perform and on which indices. For example, a key might only be allowed to read from logs-* indices, or write to a single index. This is far more secure than sharing a superuser credential.
Consider the underlying mechanism: when an API key is used, Elasticsearch decodes the base64 encoded id:key from the Authorization header. It then looks up the associated API key configuration, retrieves its granted roles, and checks if the requested operation is permitted by those roles. If the key has expired or the requested operation is not allowed, Elasticsearch will return an appropriate error, typically a 401 Unauthorized or 403 Forbidden.
The API key’s expiration is managed by Elasticsearch itself. Once an API key reaches its expiration date, it becomes invalid, and any subsequent requests using it will fail. This automatic invalidation is a crucial security feature, ensuring that compromised keys have a limited lifespan. You can also manually invalidate keys through Kibana or the Elasticsearch API if you suspect a compromise.
One subtlety often overlooked is how API keys interact with the default realm. When you create an API key, it’s tied to the user who created it. However, the API key itself acts as a distinct principal for authentication. This means that role mappings applied to the original user might not directly apply to the API key unless explicitly configured. You manage the permissions of an API key primarily through the roles assigned at the time of key creation or by reassigning roles to the API key itself if the Elasticsearch version supports it.
This allows for a "least privilege" approach. Instead of giving a service account broad permissions, you create API keys with precisely the permissions needed for that service, limited by time and scope.
The next step is to explore how to manage API keys programmatically using the Elasticsearch API itself, rather than relying solely on Kibana or command-line tools.