Cilium doesn’t just replace kube-proxy; it fundamentally rewrites how Kubernetes networking works at the kernel level using eBPF.

Let’s see what that actually looks like. Imagine a simple Service and Pod setup:

apiVersion: v1
kind: Service
metadata:
  name: my-app-service
spec:
  selector:
    app: my-app
  ports:
    - protocol: TCP
      port: 80
      targetPort: 8080
---
apiVersion: v1
kind: Pod
metadata:
  name: my-app-pod-1
  labels:
    app: my-app
spec:
  containers:
  - name: my-app
    image: nginx:latest
    ports:
    - containerPort: 8080

With kube-proxy in iptables mode, when traffic hits my-app-service on port 80, kube-proxy programs iptables rules. These rules, running in the kernel, perform NAT to redirect the service IP:port to one of the backend Pod IPs:targetPort. It’s a series of packet manipulations happening in the kernel’s netfilter hooks.

Now, with Cilium, the story is different. Cilium installs an eBPF program directly into the kernel’s network stack. When a packet arrives destined for my-app-service, the eBPF program intercepts it before iptables even sees it. This eBPF program, compiled from Cilium’s configuration and knowledge of your Kubernetes Service objects, directly looks up the available backend Pod IPs. It then rewrites the packet’s destination IP and port to one of the selected Pods, all within the eBPF program itself. There are no iptables rules for service routing; the logic lives entirely in eBPF.

This means that instead of a kernel module (iptables) doing NAT based on rules, it’s a highly optimized, sandboxed eBPF program executing directly in the kernel. This leads to significantly lower latency and higher throughput because you’re skipping multiple kernel layers and context switches.

Let’s dive into the mental model.

The Problem Cilium Solves

Kubernetes networking, by default, relies on kube-proxy for Service IP routing and DNS resolution. This model, while functional, has inherent limitations:

  • Performance Bottlenecks: kube-proxy often uses iptables or ipvs, which can become performance bottlenecks at scale due to the sheer number of rules and the overhead of kernel packet traversal.
  • Limited Observability: Understanding network traffic flow and debugging issues can be challenging with traditional iptables rules.
  • Security Gaps: Network policy enforcement can be complex and sometimes less granular than desired.

Cilium addresses these by leveraging eBPF, a technology that allows you to run custom, safe, and efficient programs within the Linux kernel.

How Cilium Works Internally

  1. eBPF Data Path: Cilium installs eBPF programs into the kernel’s network stack. These programs intercept network packets at various points (e.g., ingress, egress, L3/L4 hooks).
  2. Beringe (Service Routing): When a packet arrives for a Kubernetes Service, Cilium’s eBPF program performs a direct lookup using a highly efficient eBPF map (a kernel data structure). This map contains the real-time state of your Services and Endpoints. The eBPF program then performs Destination Network Address Translation (DNAT) directly within the kernel, routing the packet to the appropriate Pod. This bypasses kube-proxy and iptables entirely for service routing.
  3. Network Policy Enforcement: Cilium uses eBPF to enforce Kubernetes Network Policies at the packet level. This means policies are evaluated very early in the packet path, providing granular security and preventing unauthorized communication between Pods.
  4. Observability: Cilium can export detailed network flow data (e.g., who talked to whom, what Service was used, policy decisions) directly from the eBPF programs, providing deep visibility into your cluster’s network traffic.
  5. CNI Plugin: Cilium acts as a Container Network Interface (CNI) plugin, responsible for assigning IP addresses to Pods and setting up their network interfaces.

Controlling Cilium

The primary way you interact with and control Cilium is through its Cilium Custom Resource Definition (CRD) and the cilium CLI.

  • Cilium CRD: This is where you configure cluster-wide settings like IPAM (IP Address Management), tunneling modes, encryption, and integration with other services. For example, to enable BGP for advanced routing, you might have a Cilium CRD with a section like this:

    apiVersion: cilium.io/v2alpha1
    kind: Cilium
    metadata:
      name: cilium-cluster-config
    spec:
      ipam:
        mode: "kubernetes"
      bgp:
        enabled: true
        routers:
          - ip: 192.168.1.1
            node: "node-1"
            peers:
              - peerIP: "192.168.1.2"
                myASN: 64512
                peerASN: 64513
    

    This tells Cilium to manage IPs via Kubernetes and configure a BGP router on node-1 to peer with 192.168.1.2.

  • cilium CLI: This command-line tool is invaluable for debugging and introspection. You can use it to:

    • cilium status: Check the health and configuration of the Cilium agent.
    • cilium monitor: Observe network events and policy decisions in real-time.
    • cilium service list: View the services that Cilium is managing.
    • cilium bpf map list: Inspect the eBPF maps used by Cilium.

The one thing most people don’t know is that Cilium’s eBPF-based service routing isn’t just a faster iptables. It fundamentally changes the packet flow. Instead of a sequence of netfilter hooks and NAT operations, Cilium’s eBPF program often performs a direct lookup in an eBPF map to find the real backend Pod IP and then performs a single, optimized DNAT. This map lookup is incredibly fast, often involving a few CPU cycles, and the DNAT is integrated directly into the packet processing pipeline, minimizing context switches and kernel overhead. This is why you see such dramatic performance gains compared to traditional kube-proxy.

The next step after mastering Cilium’s service routing is exploring its advanced observability features, particularly how to leverage Hubble to visualize network flows and policy enforcement.

Want structured learning?

Take the full Cilium course →