HTTP/2 is actually slower than HTTP/1.1 for many common workloads, despite its headline-grabbing performance improvements.
Let’s look at how Envoy handles HTTP/2 for upstream connections and how to tune it.
Imagine Envoy acting as a client to a backend service. Envoy needs to decide whether to speak HTTP/1.1 or HTTP/2 to that backend. When configuring an upstream_cluster in Envoy, you specify the protocol.
static_resources:
clusters:
- name: my_backend_service
connect_timeout: 0.25s
type: LOGICAL_DNS
lb_policy: ROUND_ROBIN
# This is the key setting for HTTP/2
typed_extension_protocol_options:
envoy.extensions.upstreams.http.v3.HttpProtocolOptions:
# Enable HTTP/2
explicit_http_config:
use_http2: {}
# Default is HTTP/1.1 if explicit_http_config is not set or use_http2 is not present
# http_protocol_options:
# explicit_http_config:
# use_http1: {}
load_assignment:
cluster_name: my_backend_service
endpoints:
- lb_endpoints:
- endpoint:
address:
socket_address:
address: 10.0.0.1
port_value: 8080
In this configuration, explicit_http_config with use_http2: {} tells Envoy to always attempt an HTTP/2 connection to the my_backend_service cluster. If you omit this block or use use_http1: {}, Envoy will default to HTTP/1.1.
Envoy also supports automatic protocol negotiation. If you don’t specify explicit_http_config, Envoy will try to detect the best protocol. It will first attempt an HTTP/2 connection. If that fails (e.g., the backend doesn’t support it), it will fall back to HTTP/1.1. This is often the most robust default.
static_resources:
clusters:
- name: my_backend_service_auto
connect_timeout: 0.25s
type: LOGICAL_DNS
lb_policy: ROUND_ROBIN
# No explicit_http_config means Envoy will try HTTP/2 first, then fall back to HTTP/1.1
load_assignment:
cluster_name: my_backend_service_auto
endpoints:
- lb_endpoints:
- endpoint:
address:
socket_address:
address: 10.0.0.2
port_value: 9090
When Envoy successfully establishes an HTTP/2 connection, it uses a connection pool. The size and behavior of this pool are critical for performance. You can tune parameters like max_requests_per_connection and max_concurrent_streams.
max_requests_per_connection: This limits how many requests Envoy will send over a single persistent HTTP/2 connection before it’s considered "full" and Envoy will open a new one. For HTTP/2, the ideal is often a very high number, theoretically infinite, as the protocol is designed for multiplexing. However, in practice, setting it too high can lead to a single connection becoming a bottleneck if it encounters issues. A common starting point for high-throughput services is 1000 or 2000.
static_resources:
clusters:
- name: my_backend_service_tuned
connect_timeout: 0.25s
type: LOGICAL_DNS
lb_policy: ROUND_ROBIN
typed_extension_protocol_options:
envoy.extensions.upstreams.http.v3.HttpProtocolOptions:
explicit_http_config:
use_http2: {}
# Tuning HTTP/2 connection pool
http2_protocol_options:
max_requests_per_connection: 2000
load_assignment:
cluster_name: my_backend_service_tuned
endpoints:
- lb_endpoints:
- endpoint:
address:
socket_address:
address: 10.0.0.3
port_value: 7070
max_concurrent_streams: This is a fundamental HTTP/2 concept. It defines how many requests can be "in flight" simultaneously on a single TCP connection. Envoy, as a client, respects the SETTINGS_MAX_CONCURRENT_STREAMS that the server advertises. However, Envoy also has its own limit that it will try to enforce. Setting this too low can limit the parallelism HTTP/2 offers. For services that can handle high concurrency, this can be set to 100 or even higher.
static_resources:
clusters:
- name: my_backend_service_high_concurrency
connect_timeout: 0.25s
type: LOGICAL_DNS
lb_policy: ROUND_ROBIN
typed_extension_protocol_options:
envoy.extensions.upstreams.http.v3.HttpProtocolOptions:
explicit_http_config:
use_http2: {}
http2_protocol_options:
max_requests_per_connection: 2000
# Setting Envoy's internal limit for concurrent streams per connection
max_concurrent_streams: 100
load_assignment:
cluster_name: my_backend_service_high_concurrency
endpoints:
- lb_endpoints:
- endpoint:
address:
socket_address:
address: 10.0.0.4
port_value: 6060
The key insight for tuning is that HTTP/2’s primary benefit is multiplexing over a single TCP connection, reducing head-of-line blocking at the transport layer and lowering connection overhead. However, if your backend service is not designed for high concurrency, or if there are network issues, simply enabling HTTP/2 can sometimes increase latency due to the overhead of managing streams and frames. You might see upstream_cx_protocol_error in your Envoy stats if the backend actively rejects or mishandles HTTP/2.
The actual performance gains from HTTP/2 often come from reducing the number of TCP connections needed, especially in scenarios with many small, frequent requests. If your workload is dominated by a few very large requests, the benefits might be less pronounced or even negative if the overhead of HTTP/2 framing outweighs the multiplexing gains.
When troubleshooting, always check the Envoy access logs for the cx_protocol_error field. A non-zero value here, especially when you’ve explicitly enabled HTTP/2, indicates a problem with the HTTP/2 negotiation or framing between Envoy and the upstream.
The next logical step after optimizing upstream HTTP/2 is to consider how Envoy handles HTTP/2 for downstream connections from clients.