Bootstrapping an etcd cluster with a discovery service means you’re setting up a distributed key-value store where new nodes can automatically find and join the existing cluster without manual configuration.
Let’s see etcd discovery in action. Imagine we’re spinning up a new etcd node, etcd-3, and we want it to join an existing cluster of etcd-1 and etcd-2. Our discovery service is a simple HTTP endpoint that, when queried, returns the current members of the etcd cluster.
Here’s how etcd-3 might be configured to join:
ETCD_DISCOVERY_URL="http://discovery.example.com:8080"
ETCD_NAME="etcd-3"
ETCD_LISTEN_CLIENT_URLS="http://10.0.0.3:2379"
ETCD_ADVERTISE_CLIENT_URLS="http://10.0.0.3:2379"
ETCD_LISTEN_PEER_URLS="http://10.0.0.3:2380"
ETCD_ADVERTISE_PEER_URLS="http://10.0.0.3:2380"
ETCD_INITIAL_CLUSTER_TOKEN="my-etcd-cluster-1"
etcd --name ${ETCD_NAME} \
--discovery ${ETCD_DISCOVERY_URL} \
--listen-client-urls ${ETCD_LISTEN_CLIENT_URLS} \
--advertise-client-urls ${ETCD_ADVERTISE_CLIENT_URLS} \
--listen-peer-urls ${ETCD_LISTEN_PEER_URLS} \
--initial-advertise-peer-urls ${ETCD_ADVERTISE_PEER_URLS} \
--initial-cluster-token ${ETCD_INITIAL_CLUSTER_TOKEN} \
--initial-cluster-state new
When etcd-3 starts, it makes a GET request to http://discovery.example.com:8080. The discovery service, which has been previously configured by etcd-1 and etcd-2 (or by an initial bootstrap process), responds with a list of peer URLs for the existing members, like:
{
"members": [
{
"id": "...",
"name": "etcd-1",
"peerURLs": ["http://10.0.0.1:2380"],
"clientURLs": ["http://10.0.0.1:2379"]
},
{
"id": "...",
"name": "etcd-2",
"peerURLs": ["http://10.0.0.2:2380"],
"clientURLs": ["http://10.0.0.2:2379"]
}
]
}
etcd-3 then uses this information to initiate a handshake with etcd-1 and etcd-2, joining the cluster. The discovery service itself is often implemented using etcd itself (a "discovery cluster" that bootstraps a "production cluster"), or a simple HTTP service that registers etcd members. The key is that it provides a stable, discoverable endpoint for new nodes to find existing ones.
The fundamental problem etcd discovery solves is the "chicken and egg" problem of bootstrapping a distributed system. How do new nodes know about existing nodes to join them, especially in dynamic environments like cloud or container orchestrators? Without discovery, you’d have to manually list every peer URL for every node in its configuration, which is brittle and scales poorly. Discovery abstracts this away by providing a single, well-known point of contact.
Internally, when a new member joins using discovery, it contacts the discovery service to get a list of peer URLs of existing members. It then attempts to connect to these members to join the cluster. Once it successfully joins, it registers itself with the discovery service (if the discovery service is designed to do so, e.g., by being an etcd cluster itself). This ensures that subsequent new members can discover the newly added node. The initial-cluster-state new flag is crucial for the very first node(s) joining; subsequent nodes will typically use existing if they are joining an already running cluster and have their peer information already recorded by the discovery mechanism.
You control the discovery process primarily through the ETCD_DISCOVERY_URL environment variable (or its equivalent command-line flag) and the ETCD_INITIAL_CLUSTER_TOKEN. The token ensures that nodes only join clusters with the same token, preventing accidental mixing of members from different logical clusters. The discovery URL points to the service that will provide the list of initial peers.
Most etcd discovery implementations rely on a mechanism where the discovery service itself is an etcd cluster. In this setup, the discovery etcd cluster is bootstrapped first, and then new etcd members join by querying this discovery cluster. The discovery cluster typically stores information about the members of the target etcd cluster (the one you’re actually trying to build) under a specific key, like /discovery/my-etcd-cluster-1/members. When a new node starts, it queries this key, receives the list of current members, and then attempts to join that cluster. Once it joins the target cluster, it might also register itself in the discovery cluster’s member list. This creates a self-reinforcing system where the discovery service is also managed by etcd.
The next step after successfully bootstrapping your etcd cluster with discovery is to understand how to manage its members and health.