Envoy’s "upstream" and "downstream" are relative terms that define the direction of a request relative to Envoy itself.
Let’s see Envoy in action. Imagine a user making a request to your service.
// User's browser makes a request to Envoy
GET /api/users/123 HTTP/1.1
Host: myapp.example.com
In this scenario, the user’s browser is the downstream client, and Envoy is the upstream server. Envoy receives the request from the downstream client.
// Envoy receives the request
// Envoy then makes a new request to the actual service
GET /users/123 HTTP/1.1
Host: user-service.internal
X-Forwarded-For: <user_ip_address>
Now, Envoy acts as the downstream client, and the actual user service (running on user-service.internal) is the upstream server. Envoy forwards the request to the upstream server. This is the core of how Envoy functions as a proxy and load balancer. It sits between clients and services, managing the flow of traffic.
The problem Envoy solves is the complexity of managing network communication between many distributed services. Instead of each service needing to know how to discover, connect to, and communicate reliably with every other service, they can all talk to Envoy. Envoy handles the heavy lifting: service discovery, load balancing, TLS termination, health checking, routing, and more.
Internally, Envoy is built around a series of filter chains. When a request comes in from a downstream client, it traverses a "downstream filter chain." These filters can inspect, modify, or even terminate the request. Once the request is ready to be sent to an upstream service, it traverses an "upstream filter chain." This is where actions like load balancing and initiating the connection to the upstream host occur.
The exact levers you control are primarily in the configuration. You define:
- Listeners: These are the network endpoints (IP address and port) that Envoy listens on for downstream connections. Here, you configure the filter chains that will process incoming requests.
- Clusters: These represent groups of upstream hosts that Envoy can send traffic to. For each cluster, you define load balancing policies, health checking mechanisms, and how to discover the upstream hosts (e.g., static configuration, DNS, or a service registry).
- Routes: These define how requests arriving at a listener are directed to a specific cluster. You can use sophisticated rules based on host, path, headers, etc., to route traffic.
Consider how Envoy handles TLS. When a downstream client connects to Envoy using HTTPS, Envoy can terminate the TLS connection. This means Envoy decrypts the request, and then it can establish a new TLS connection (or a plain HTTP connection) to the upstream service. This offloads the TLS burden from your application services. The configuration for this involves specifying certificates and private keys on the listener, and then defining whether the connection to the upstream cluster should be TLS-enabled (and how it should be configured, e.g., trust certificates).
The most surprising thing is how Envoy’s internal representation of a request, often called a "request state" or "stream object," evolves as it passes through these filters and across the network boundary. It’s not just a simple copy-paste; data can be transformed, new headers added (like X-Forwarded-For or tracing information), and the underlying connection details can change entirely, all while maintaining the logical flow of the original request.
The next concept to grapple with is how Envoy manages connection pooling and keep-alives to optimize performance when communicating with upstream services.