Envoy’s memory and CPU usage isn’t just about resource allocation; it’s about how its internal data structures and event loops interact with the underlying OS and network stack, making "tuning" often a misnomer for understanding and guiding its behavior.
Let’s see this in action. Imagine a high-traffic Envoy proxy.
admin:
access_log_path: /dev/null
address:
socket_address:
address: 127.0.0.1
port_value: 9901
static_resources:
listeners:
- name: listener_0
address:
socket_address:
address: 0.0.0.0
port_value: 8080
filter_chains:
- filters:
- name: envoy.filters.network.http_connection_manager
typed_config:
"@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
stat_prefix: ingress_http
route_config:
name: local_route
virtual_hosts:
- name: local_service
domains: ["*"]
routes:
- match:
prefix: "/"
route:
cluster: cluster_0
http_filters:
- name: envoy.filters.http.router
typed_config:
"@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router
clusters:
- name: cluster_0
connect_timeout: 0.25s
type: LOGICAL_DNS
lb_policy: ROUND_ROBIN
load_assignment:
cluster_name: cluster_0
endpoints:
- lb_endpoints:
- endpoint:
address:
socket_address:
address: example.com
port_value: 80
This configuration sets up a basic HTTP listener on port 8080 that proxies requests to example.com:80. Envoy, by default, is very efficient, but in production, especially with TLS termination, a high volume of connections, or complex filter chains, its resource footprint becomes a critical concern.
The core problem Envoy solves is abstracting away the complexities of network communication and service discovery for microservices. It acts as a universal data plane, handling concerns like load balancing, TLS termination, health checking, and observability at the edge of your services. Internally, it’s built around an event-driven, non-blocking architecture. This means a single Envoy process can manage thousands of concurrent connections with relatively low overhead. Its memory usage is primarily driven by connection state, TLS session caches, and internal data structures for routing and statistics. CPU usage spikes typically occur during connection establishment, TLS handshake, request/response processing by filters, and during periods of high network I/O.
When tuning, you’re not just setting arbitrary limits. You’re influencing how Envoy manages its connection pool, how it handles buffering, and how aggressively it reuses resources. For instance, the max_connections setting on a cluster isn’t just a cap; it dictates how many concurrent upstream connections Envoy will maintain, directly impacting its memory usage for connection state and its ability to absorb sudden traffic bursts.
The connection_buffer_limits_bytes setting on a listener or HTTP connection manager is crucial. If you have slow upstream services or a noisy downstream client, Envoy’s buffer can grow. By default, this limit is quite generous. Setting it too low can lead to BUFFER_OVERFLOW errors being sent to the client, while setting it too high can cause Envoy to consume excessive memory and potentially become unresponsive under load. A common starting point for high-throughput scenarios might be 1024Ki or 2048Ki, but this is highly dependent on your specific traffic patterns and upstream service latency.
Consider the per_connection_buffer_limit_bytes on the envoy.extensions.http.connection_manager.v3.HttpConnectionManager configuration. This limits the buffer size for individual connection requests and responses. If you’re seeing ENVOY_STREAM_RESET errors on downstream connections and your upstream services are healthy, this limit might be too restrictive. Conversely, if Envoy’s memory usage is high, and you have many idle connections with data waiting, increasing this limit could exacerbate the problem. A typical value might be 16384 bytes, but again, observe your traffic.
The CPU usage is often tied to the efficiency of your filter chain and the overhead of TLS. For CPU-bound Envoy instances, particularly those handling TLS termination, consider disabling TLS session resumption if your client base doesn’t benefit from it, or tune the session cache size. The session_ticket_keys and session_ticket_cache_size within the envoy.extensions.transport_sockets.tls.v3.TlsContext can be adjusted. A smaller cache reduces memory but increases handshake CPU load per unique client. A larger cache uses more memory but can reduce CPU for frequent re-connections from the same clients.
The max_requests_per_connection on a cluster’s outlier_detection configuration, while primarily for resilience, can indirectly affect resource usage. If outlier detection is aggressively ejecting backends, Envoy might spend more CPU attempting to re-establish connections or re-route traffic. It’s not a direct tuning knob for memory/CPU but part of the overall system behavior that influences resource consumption.
One of the most subtle aspects of Envoy’s performance tuning is understanding how its internal Stream objects are managed. Each active request/response pair, regardless of whether it’s an upstream or downstream connection, is represented by a Stream. The lifecycle and state transitions of these streams, especially under heavy load or during error conditions, can lead to unexpected memory churn or CPU spikes. Envoy’s internal garbage collection for these stream objects is generally efficient, but if you have a scenario with very short-lived, high-volume requests or frequent connection churn, the overhead of stream creation and destruction can become noticeable. This is why observing the envoy_server_streams_active and envoy_server_streams_created statistics from the /stats/prometheus endpoint is crucial. A rapidly increasing streams_created without a corresponding increase in streams_active indicates rapid stream churn, which might point to issues in your application logic or upstream service behavior that Envoy is faithfully reflecting.
The next logical step after optimizing Envoy’s resource usage is to understand how to configure its health checking mechanisms to prevent cascading failures and improve overall service availability.