etcd’s authentication is a lot more flexible than most people realize, often leading them to over-engineer security.
Let’s see how you’d actually interact with a secured etcd cluster from your terminal. Imagine you have an etcd cluster running with TLS enabled and client certificate authentication enforced. Your etcdctl command might look something like this:
ETCDCTL_API=3 etcdctl \
--endpoints="https://etcd-0.example.com:2379,https://etcd-1.example.com:2379" \
--cacert=/etc/etcd/ssl/ca.crt \
--cert=/etc/etcd/ssl/client.crt \
--key=/etc/etcd/ssl/client.key \
get /some/key
This command establishes a TLS connection to the etcd endpoints, verifying the server’s identity using the provided CA certificate (ca.crt). It also presents its own client certificate (client.crt) and private key (client.key) to the server for authentication.
Now, let’s layer token-based authentication on top of that. If your etcd cluster is configured to use an authentication backend (like a JWT issuer or a custom auth plugin), you’ll need to obtain a token first. Assuming you have a mechanism to get a valid token (e.g., from an identity provider or by generating one), your etcdctl command would change to include the token:
ETCDCTL_API=3 etcdctl \
--endpoints="https://etcd-0.example.com:2379,https://etcd-1.example.com:2379" \
--cacert=/etc/etcd/ssl/ca.crt \
--cert=/etc/etcd/ssl/client.crt \
--key=/etc/etcd/ssl/client.key \
--user="myuser:mypassword" \ # Or --user="$(cat /path/to/token)"
get /some/key
Here, the --user flag is used to pass credentials. If you’re using a token, you’d typically pass it as the password part of the user:password format, or if your token itself is the username, you might just provide that. For a raw token, it’s common to see it passed as --user="$(cat /path/to/token)" if the token is meant to be the username or --user="token:$(cat /path/to/token)" if it’s treated as a password. The etcd server then consults its configured auth backend to validate this token.
The underlying mechanism for authentication in etcd involves a layered approach. TLS provides transport-level security, ensuring that communication is encrypted and that you’re talking to the genuine etcd server (and it to you, if client cert auth is enabled). Once the connection is secured, etcd’s authentication subsystem kicks in. If client certificate authentication is enabled, the server verifies the presented client certificate against its trusted CA. If token-based authentication is active, etcd forwards the credentials (username/password or token) to its configured authentication backend. This backend, which could be a simple in-memory store, a distributed identity system, or a custom plugin, performs the actual verification. Upon successful authentication, the backend returns a "grant" to etcd, which etcd then uses to authorize subsequent operations based on predefined roles and permissions.
The most surprising part is how these two authentication methods, TLS client certificates and tokens, can be used independently or in combination. You can have a cluster secured only by TLS client certificates, where the certificate itself is the credential. Or, you can have TLS for transport security and a token-based system for user identity. The --user flag in etcdctl is a bit of a chameleon; it can take a username and password, or it can take a token that the authentication backend understands. This flexibility means you don’t always need a separate username and password; a single, opaque token can suffice for authentication.
When you use --user with a token, etcd doesn’t just blindly trust the token. It sends this token to the configured authentication plugin. The plugin is responsible for validating the token’s format, checking its expiry, and verifying its signature if it’s a JWT or similar signed token. If the token is valid, the plugin issues a set of permissions (grants) back to etcd for that authenticated user. These grants determine what actions the user can perform on which keys within etcd. This separation of concerns means etcd itself doesn’t need to know how to validate complex JWTs or query an external OIDC provider; it delegates that responsibility to the plugin.
The critical takeaway is that etcd’s security model is designed to be extensible. You can plug in different authentication backends and authorization strategies without altering etcd’s core. This allows for integration with existing enterprise identity management systems or custom security policies. The etcdctl client provides a unified interface, abstracting away the underlying authentication mechanisms, making it easier to manage your etcd data securely.
The next step after mastering authentication is understanding how to effectively manage authorization and role-based access control within etcd.