The most surprising truth about service meshes is that they often make your network more complex, not less, by abstracting away the underlying infrastructure.
Let’s see this in action. Imagine a simple microservice architecture: a frontend service calling an orders service, which in turn calls a payments service. Without a service mesh, if orders needs to talk to payments, it directly establishes a TCP connection to payments’ IP address and port.
Now, let’s introduce Istio. Each of your services (frontend, orders, payments) gets a "sidecar" proxy (Envoy, by default) injected into its Kubernetes pod. When orders wants to talk to payments, it doesn’t talk directly to payments. Instead, it talks to its own sidecar proxy. This sidecar, configured by Istio’s control plane, then intercepts the outbound traffic, looks up the correct payments service endpoint (which it knows through Istio’s service discovery), and forwards the request to the payments sidecar. The payments sidecar receives the request and forwards it to the actual payments application.
Here’s what that looks like in terms of configuration. In Istio, you’d typically have Kubernetes Service and Deployment objects for your applications. The mesh magic comes from Istio’s Custom Resource Definitions (CRDs). For example, to control traffic flow, you might use a VirtualService:
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: orders-route
spec:
hosts:
- orders
http:
- route:
- destination:
host: payments
subset: v1
weight: 100
This VirtualService tells Istio that any traffic destined for the orders service should be routed to the payments service, specifically to its v1 subset. The subset is a concept Istio uses for advanced routing, like canary deployments. Without the VirtualService, traffic would just go to any available payments pod.
The problem this solves is moving critical network concerns out of your application code and into a dedicated infrastructure layer. Instead of your orders service code having to implement retries, timeouts, circuit breakers, mTLS, or detailed metrics collection for its calls to payments, the Istio sidecar handles all of that. Your application code just makes a simple HTTP request to localhost:port (or to the Kubernetes service name, which the sidecar intercepts).
Linkerd takes a slightly different approach. It also uses sidecars, but its philosophy leans towards simplicity and performance. The sidecar (a lightweight binary called proxy) is also injected. Linkerd’s configuration is generally less verbose than Istio’s. For basic service-to-service communication, you might not need much explicit configuration beyond enabling the mesh. Linkerd automatically provides features like TLS encryption between meshed services and basic metrics.
Consul, while often used as a service registry, also has a service mesh component. Consul Connect allows you to mesh services by running its own proxy (also Envoy) alongside your application. Consul’s strength lies in its robust service discovery and health checking capabilities, which are foundational to its mesh. You configure Connect by defining intentions, which are essentially authorization policies between services.
Here’s how the internal flow differs slightly. In Istio and Linkerd, the sidecar acts as a transparent proxy. When orders makes a call, it’s often to localhost:some_port or the Kubernetes service name. The sidecar intercepts this. In Consul Connect, you might explicitly configure your application to talk to a local Consul agent’s proxy, which then forwards the request. Or, like Istio/Linkerd, you can use automatic sidecar injection.
The core mental model for any service mesh is this: your application thinks it’s talking directly to another application, but in reality, it’s talking to its local sidecar. That sidecar then talks to the remote application’s sidecar. All the interesting networking logic – routing, security, observability – happens in these sidecars, managed by a control plane.
What most people don’t realize is that the sidecar proxy is a full-fledged Envoy (in Istio/Consul) or a custom proxy (in Linkerd). It’s a highly configurable network proxy running on every node, intercepting all inbound and outbound traffic for your application. This means it has visibility into everything, but also becomes a potential bottleneck or point of failure. Its configuration is dynamic, pushed by the control plane, and managing that configuration correctly is paramount.
The next concept you’ll likely grapple with is how to observe the traffic flowing through the mesh, which leads to distributed tracing and advanced metric aggregation.