Cilium replaces kube-proxy with eBPF, allowing Kubernetes networking to bypass the kernel’s netfilter hooks entirely for massive performance gains.
Imagine you’ve got a busy Kubernetes cluster. Pods are constantly talking to each other, services are being accessed, and all this traffic has to go somewhere. Traditionally, that "somewhere" involves kube-proxy, which uses iptables rules in the Linux kernel to manage how traffic gets to your services. It’s like a traffic cop directing cars, but this cop is a bit slow and gets overwhelmed easily.
Now, enter Cilium with eBPF. Instead of relying on kube-proxy and iptables, Cilium leverages eBPF (extended Berkeley Packet Filter) directly within the Linux kernel. eBPF programs are tiny, sandboxed applications that can run in the kernel without changing kernel source code or loading kernel modules. Cilium uses eBPF to intercept network packets at various points in the kernel’s networking stack. It can then make intelligent decisions about where to route them, enforce security policies, and even perform load balancing, all before the packet even gets to the traditional iptables path. This bypasses the overhead of iptables lookups, leading to significantly faster packet processing.
Let’s see what this looks like in practice.
First, you need to install Cilium. This typically involves applying a Helm chart or a set of Kubernetes manifests. When you install Cilium, you’ll configure it to run in kube-proxy-replacement mode. This is the key setting that tells Cilium to take over the role of kube-proxy.
Here’s a snippet from a typical Cilium Helm chart values.yaml for enabling this:
# values.yaml for Cilium Helm chart
kubeProxyReplacement: "true"
# Or more specifically:
# kubeProxyReplacement: "strict" # For full replacement, disabling kube-proxy entirely
# kubeProxyReplacement: "probe" # For partial replacement, probing for kube-proxy
# If you are migrating from kube-proxy, you might want to enable this
# to ensure services are still reachable during the transition.
# It will also disable kube-proxy if it's running.
# See https://docs.cilium.io/en/stable/concepts/networking/k8s-services/#kube-proxy-replacement
# If running in an environment where kube-proxy is managed by the cloud provider
# (e.g. GKE, EKS), it might be necessary to disable it manually.
# See https://docs.cilium.io/en/stable/operations/kubernetes/cloud-provider-integration/
Once installed and kube-proxy-replacement is enabled, Cilium’s eBPF programs take over the responsibility of implementing Kubernetes Services. When a pod tries to reach a Service IP (like 10.96.0.10), the eBPF program attached to the pod’s network interface intercepts this. It then looks up the Service definition and selects an appropriate backend pod. Instead of going through iptables chains, the eBPF program directly rewrites the destination IP and port to that of the selected backend pod and sends the packet on its way.
The primary problem Cilium solves with this approach is the performance bottleneck and scalability issues inherent in iptables-based service proxying. As the number of Services and pods grows, the iptables rule table becomes enormous, leading to increased latency and CPU usage on each node. Cilium’s eBPF approach scales much more linearly because it uses efficient data structures and direct kernel path processing.
Here’s a breakdown of the internal mechanics:
- Service Discovery: Cilium watches the Kubernetes API for Service and EndpointSlice objects, just like
kube-proxy. - eBPF Map Population: It translates this information into eBPF maps. These are special key-value stores that eBPF programs can access efficiently. One map might store Service definitions, another might store the current backend IPs for each Service.
- Packet Interception: When a packet destined for a Service IP leaves a pod, an eBPF program attached to the pod’s network stack intercepts it.
- Lookup and Rewrite: The eBPF program consults the Service map to find the Service definition and then the backend map to select a healthy backend pod. It then modifies the packet’s destination IP and port directly in memory.
- Forwarding: The packet is then forwarded to the chosen backend pod, bypassing the entire
iptablesprocessing path.
The result is dramatically reduced latency. For latency-sensitive applications, you might see improvements from microseconds to milliseconds. Throughput also increases because fewer CPU cycles are spent managing and traversing iptables rules.
A common misconception is that eBPF is just a faster iptables. It’s not. It’s a fundamentally different paradigm. While iptables is a rule-based packet filtering and manipulation system, eBPF allows you to run custom logic directly within the kernel’s event processing pipeline. Cilium uses this to implement not just Service load balancing but also advanced features like network policy enforcement, observability, and even L7 protocol awareness, all with minimal overhead.
The one thing most people don’t realize is how much of the networking stack Cilium can influence with eBPF. It’s not just about replacing kube-proxy. Cilium can handle packet routing, load balancing, and policy enforcement at various points in the kernel, from early packet reception (TC_INGRESS and TC_EGRESS) to socket operations. This allows for fine-grained control and optimization that’s impossible with traditional methods. For instance, Cilium can implement load balancing decisions based on L7 request headers before the packet is even handed off to the application, a feat that would require complex iptables rules or sidecar proxies with kube-proxy.
After successfully replacing kube-proxy with Cilium’s eBPF mode, the next major hurdle you’ll likely encounter is configuring and understanding Cilium’s advanced Network Policies, which operate at L3/L4 and potentially L7, offering far more granular security than standard Kubernetes NetworkPolicies.