Envoy’s circuit breakers aren’t just about preventing cascading failures; they’re a sophisticated mechanism that actively protects your upstream services from being overwhelmed, often before they even realize they’re in trouble.
Let’s see this in action. Imagine a service, productpage, that calls another service, details. If details starts to slow down, productpage might start accumulating slow requests. Envoy, sitting in front of details, can step in.
Here’s a snippet of an Envoy configuration demonstrating circuit breakers:
static_resources:
listeners:
- name: listener_0
address:
socket_address:
address: 0.0.0.0
port_value: 10000
filter_chains:
- filters:
- name: envoy.filters.network.http_connection_manager
typed_config:
'@type': type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
stat_prefix: ingress_http
route_config:
name: local_route
virtual_hosts:
- name: local_service
domains: ["*"]
routes:
- match:
prefix: "/details"
route:
cluster: details_service
http_filters:
- name: envoy.filters.http.router
typed_config: {}
clusters:
- name: details_service
connect_timeout: 0.25s
type: LOGICAL_DNS
dns_lookup_family: V4_ONLY
lb_policy: ROUND_ROBIN
load_assignment:
cluster_name: details_service
endpoints:
- lb_endpoints:
- endpoint:
address:
socket_address:
address: 127.0.0.1
port_value: 9080
# Circuit Breakers configuration
circuit_breakers:
thresholds:
- priority: HIGH
max_connections: 100
max_pending_requests: 50
max_requests: 200
max_retries: 20
- priority: DEFAULT
max_connections: 1000
max_pending_requests: 100
max_requests: 1000
max_retries: 100
This configuration tells Envoy, for the details_service cluster, to apply specific limits. max_connections is the total number of connections Envoy will establish to the upstream cluster. max_pending_requests is the number of requests that Envoy will queue waiting for an upstream connection to become available. max_requests is the total number of concurrent requests that Envoy will have outstanding to the upstream cluster. max_retries limits how many times Envoy will retry a request.
The core problem circuit breakers solve is cascading failure. When a service becomes slow or unresponsive, the services calling it can start to back up. Each waiting request consumes resources (threads, memory, connection pools) on the calling service. If enough requests back up, the calling service itself can become overloaded and fail, triggering failures in its callers, and so on, creating a domino effect. Envoy’s circuit breakers act as a dam, preventing this overflow.
When an upstream cluster hits a circuit breaker limit, Envoy stops sending new requests to it. Instead of letting the request hang and consume resources indefinitely on the client side, Envoy will immediately return an error (typically a 503 Service Unavailable) to the caller. This instantaneous feedback is crucial. It tells the caller that the downstream service is unavailable right now, allowing the caller to react appropriately – perhaps by returning a cached response, failing fast, or retrying later.
Envoy categorizes circuit breakers by priority: HIGH and DEFAULT. HIGH priority requests are given preference. If the circuit breaker is open for DEFAULT priority, HIGH priority requests can still be sent. This is useful for critical internal services or health checks. You can also configure different limits for different priorities. For instance, you might want to allow fewer concurrent requests for HIGH priority traffic if it’s known to be more resource-intensive.
The exact mechanism for max_connections is Envoy tracking the number of active connections to the upstream host. Once max_connections is reached for a specific host in the cluster, Envoy will not open any new connections to that host. For max_pending_requests, Envoy maintains a queue for requests that have been accepted by Envoy but are waiting for an available connection to the upstream cluster. If this queue fills up, new requests are rejected. max_requests limits the total number of requests that are in flight to the upstream cluster. This includes requests that have been sent and are waiting for a response. Once this limit is hit, Envoy will reject new requests.
A common mistake is to set max_connections too low, causing Envoy to refuse perfectly healthy connections. Conversely, setting it too high can allow too many connections to be opened, starving the upstream service of resources. The max_pending_requests and max_requests limits are often more effective for preventing overload during transient slowdowns.
The next thing you’ll likely want to configure is fine-grained control over which requests get rejected when a breaker is open.