The most surprising thing about Envoy’s xDS APIs is that they aren’t a single API, but a family of dynamic configuration interfaces that allow Envoy to adapt its behavior in real-time without restarts.
Let’s see this in action. Imagine we have a simple Envoy configuration that’s not using xDS. It might look something like this:
static_resources:
listeners:
- name: listener_0
address:
socket_address:
address: 0.0.0.0
port_value: 8080
filter_chains:
- filters:
- name: envoy.filters.network.http_connection_manager
typed_config:
"@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
stat_prefix: ingress_http
route_config:
virtual_hosts:
- name: service_a
domains: ["service-a.example.com"]
routes:
- match:
prefix: "/"
route:
cluster: cluster_a
http_filters:
- name: envoy.filters.http.router
typed_config:
"@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router
clusters:
- name: cluster_a
connect_timeout: 0.25s
type: LOGICAL_DNS
lb_policy: ROUND_ROBIN
dns_lookup_family: V4_ONLY
load_assignment:
cluster_name: cluster_a
endpoints:
- lb_endpoints:
- endpoint:
address:
socket_address:
address: 192.168.1.100
port_value: 80
This is a static configuration. If we wanted to add a new service, change a route, or update an endpoint, we’d have to edit this file, reload Envoy, and potentially restart it. Now, let’s switch to dynamic configuration using xDS. The core idea is that Envoy connects to a control plane (like Istio, Consul Connect, or a custom one) via gRPC and subscribes to updates for different configuration resources.
The xDS APIs are named after the type of resource they manage:
- LDS (Listener Discovery Service): Manages
Listeners. A listener is what Envoy exposes to accept incoming connections. It defines the address and port, and how those connections are processed. - RDS (Route Discovery Service): Manages
RouteConfigurations. A route configuration directs traffic within an HTTP connection manager. It defines virtual hosts, domains, and routes based on request properties (like the path or headers) to specific clusters. - CDS (Cluster Discovery Service): Manages
Clusters. A cluster is a group of upstream endpoints (servers) that Envoy can send traffic to. This is where you define how Envoy connects to your services. - EDS (Endpoint Discovery Service): Manages
Endpoints. Endpoints are the individual IP addresses and ports of the actual service instances within a cluster. EDS is how Envoy gets notified of healthy and unhealthy instances. - ADS (Aggregated Discovery Service): This is a meta-API. Instead of having separate gRPC streams for LDS, RDS, CDS, and EDS, ADS allows Envoy to subscribe to all these resource types over a single, unified stream. This simplifies the gRPC connection management significantly.
The mental model is that Envoy acts as a client to a discoveryservice (the control plane). It tells the service, "Hey, I’m Envoy instance X, listening on port Y, and I’m interested in these LDS resources, which in turn might depend on RDS, CDS, and EDS resources." The discovery service then pushes configuration updates to Envoy as they happen.
For example, if you wanted to add a new route to service-a.example.com that directs traffic for /api/v2 to a new cluster cluster_b, your control plane would:
- Push an update to the
RouteConfigurationforservice-a.example.comto include a new route:# Example snippet of a RouteConfiguration update routes: - match: prefix: "/api/v2" route: cluster: cluster_b - If
cluster_bdoesn’t exist, push a newClusterresource via CDS. - If
cluster_bneeds specific endpoints, pushEndpointresources via EDS forcluster_b.
Envoy receives these updates on its respective xDS streams (or the single ADS stream) and dynamically updates its internal configuration. Listeners, routes, clusters, and endpoints are added, modified, or removed without Envoy needing to be restarted. This is crucial for microservices architectures where services are constantly being deployed, scaled, and updated.
The real power comes from the ability to perform sophisticated traffic management. For instance, you can configure Envoy to send 5% of traffic to a new version of a service (cluster_b) while 95% goes to the old version (cluster_a). This is done by defining multiple endpoints for a cluster and using weighted routing, or by having separate clusters and routing rules.
The actual communication uses Protocol Buffers serialized over gRPC. Envoy establishes gRPC streams to the control plane for each xDS API it uses. For ADS, it’s a single stream. Envoy sends a DiscoveryRequest containing its current state (like the last seen version of a resource) and the control plane responds with a DiscoveryResponse containing the new configuration or an empty response if the configuration hasn’t changed.
What most people don’t realize is how Envoy’s internal state management interacts with xDS. Envoy doesn’t just blindly apply configurations. It maintains versions for each resource (e.g., version_info in DiscoveryResponse). When it receives an update, it checks if the new version is truly different from what it already has. If it receives a DiscoveryResponse with the same version_info as the last one it processed for that resource type, it simply acknowledges it and moves on. This versioning is key to ensuring idempotency and preventing configuration loops.
The next logical step after mastering these core xDS APIs is understanding how Envoy handles certificate management for mTLS via SDS (Secret Discovery Service).