Envoy’s traffic mirroring feature lets you send a copy of live traffic to a separate, identical "shadow" cluster without impacting the primary traffic flow.
Here’s what that looks like in Envoy’s configuration:
static_resources:
clusters:
- name: primary_service
# ... primary service configuration ...
# This is the service receiving the actual traffic
type: STRICT_DNS
dns_lookup_family: V4_ONLY
load_assignment:
cluster_name: primary_service
endpoints:
- lb_endpoints:
- endpoint:
address:
socket_address:
address: 10.0.0.1
port_value: 8080
- name: shadow_service
# ... shadow service configuration ...
# This is the identical cluster receiving mirrored traffic
type: STRICT_DNS
dns_lookup_family: V4_ONLY
load_assignment:
cluster_name: shadow_service
endpoints:
- lb_endpoints:
- endpoint:
address:
socket_address:
address: 10.0.0.2
port_value: 8080
listeners:
- name: listener_0
address:
socket_address:
address: 0.0.0.0
port_value: 80
filter_chains:
- filters:
- name: envoy.filters.http.router
typed_config:
"@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router
# This is the key section for mirroring
traffic_shaping:
shaping_policy:
- policy:
fixed_weight_slos:
# 100% of traffic goes to primary_service
weight: 1.0
# This defines the shadow destination
# When traffic is mirrored, it's sent to this cluster
# The weight here is irrelevant for mirroring, it's about the *other* traffic
# The actual mirroring is defined in the router filter
# envoy.filters.http.router.v3.Router
# This is where the shadowing is configured
# The key here is that the router filter itself is configured to mirror.
# This is not a direct part of the shaping policy itself, but rather
# a separate configuration within the router filter.
# The shaping policy defines how traffic is distributed, not how it's mirrored.
# The mirroring is a separate directive.
# Let's correct this and focus on the actual mirroring configuration.
# The traffic shaping policy is for weighted routing, not mirroring.
# Corrected configuration for mirroring:
listeners:
- name: listener_0
address:
socket_address:
address: 0.0.0.0
port_value: 80
filter_chains:
- filters:
- name: envoy.filters.http.router
typed_config:
"@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router
# This is the crucial part: the 'request_headers_to_add' is NOT for mirroring.
# The mirroring is configured at the ROUTE level, not the filter level directly.
# Let's adjust again to show the correct structure.
# FINAL CORRECTED CONFIGURATION FOR MIRRORING
static_resources:
clusters:
- name: primary_service
type: STRICT_DNS
dns_lookup_family: V4_ONLY
load_assignment:
cluster_name: primary_service
endpoints:
- lb_endpoints:
- endpoint:
address:
socket_address:
address: 10.0.0.1
port_value: 8080
- name: shadow_service
type: STRICT_DNS
dns_lookup_family: V4_ONLY
load_assignment:
cluster_name: shadow_service
endpoints:
- lb_endpoints:
- endpoint:
address:
socket_address:
address: 10.0.0.2
port_value: 8080
listeners:
- name: listener_0
address:
socket_address:
address: 0.0.0.0
port_value: 80
filter_chains:
- filters:
- name: envoy.filters.http.router
typed_config:
"@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router
# This is where the mirroring is configured within the router filter.
# Specifically, within the routes section.
routes:
- match:
prefix: "/" # Match all requests
route:
cluster: primary_service # Primary destination
# This is the key for mirroring
request_mirror_policies:
- cluster: shadow_service # The cluster to mirror traffic to
# You can optionally add a percentage of traffic to mirror
# If omitted, 100% of traffic is mirrored.
# runtime_fraction:
# default_value:
# numerator: 50 # 50% of traffic mirrored
# runtime_key: "envoy.mirroring.shadow_service"
This configuration sets up two clusters, primary_service and shadow_service, both pointing to identical backend deployments. The listener_0 is configured to use the envoy.filters.http.router filter. Crucially, within the routes section of the router filter, for any request matching the prefix / (meaning all requests), we define request_mirror_policies. This policy specifies that traffic should be mirrored to the shadow_service cluster.
The real magic here is that the router filter handles the duplication. When a request comes in, Envoy processes it normally, sends it to primary_service, and then, as a separate, asynchronous operation, it creates an identical copy of that request and sends it to shadow_service. The response from shadow_service is discarded; it doesn’t affect the response sent back to the client from primary_service.
This is incredibly powerful for testing new deployments or configurations. You can point your live, production traffic (or a subset of it) at a new version of your service running in shadow_service and compare the responses and behavior without any risk to your users. You can verify that the new version behaves identically to the old one, or identify regressions before they impact production.
The runtime_fraction within request_mirror_policies is a key lever. It allows you to control the percentage of requests that get mirrored. This is essential for gradual rollouts or for reducing the load on your shadow cluster during initial testing. For example, setting numerator: 50 would mirror 50% of requests.
The most surprising thing is how little overhead this adds to the primary request. Envoy effectively clones the request data and sends it out on a separate connection. The primary request proceeds entirely independently, and its latency is not affected by the mirroring operation itself. The only cost is the additional network egress and the processing of the second request on the shadow cluster.
The next thing you’ll want to figure out is how to compare the responses from the primary and shadow clusters. Envoy itself doesn’t do this comparison; it just sends the traffic. You’ll need an external system, often another Envoy proxy configured to receive responses from both clusters and perform assertions, or a dedicated service that aggregates and compares the results.