Cilium can achieve near-zero packet loss even under extreme load, often outperforming traditional kernel networking stacks.
Let’s see how we can push Cilium to its limits and measure its throughput and latency. We’ll use netperf for raw TCP/UDP performance and iperf3 for more flexible bandwidth testing.
First, ensure you have Cilium installed and running on your Kubernetes cluster. We’ll deploy a simple netperf and iperf3 client/server setup within the cluster.
1. Deploying iperf3 Server
We’ll create a Deployment and Service for an iperf3 server. This server will listen for incoming connections and report the bandwidth.
apiVersion: apps/v1
kind: Deployment
metadata:
name: iperf3-server
spec:
replicas: 1
selector:
matchLabels:
app: iperf3-server
template:
metadata:
labels:
app: iperf3-server
spec:
containers:
- name: iperf3
image: networkstatic/iperf3
command: ["iperf3", "-s"]
ports:
- containerPort: 5201
protocol: TCP
- containerPort: 5201
protocol: UDP
---
apiVersion: v1
kind: Service
metadata:
name: iperf3-server
spec:
selector:
app: iperf3-server
ports:
- name: tcp
port: 5201
protocol: TCP
- name: udp
port: 5201
protocol: UDP
Apply this with kubectl apply -f iperf3-server.yaml.
2. Deploying netperf Server
Similarly, for netperf, we’ll deploy a server. netperf has a daemon mode (netserver).
apiVersion: apps/v1
kind: Deployment
metadata:
name: netperf-server
spec:
replicas: 1
selector:
matchLabels:
app: netperf-server
template:
metadata:
labels:
app: netperf-server
spec:
containers:
- name: netperf
image: jasonacox/netperf
command: ["netserver", "-p", "12865"]
ports:
- containerPort: 12865
protocol: TCP
Apply this with kubectl apply -f netperf-server.yaml.
3. Running iperf3 Tests
Now, let’s run an iperf3 client to connect to the server and measure TCP and UDP throughput. You can run this from another pod, or directly from your kubectl environment if you have iperf3 installed locally and can port-forward.
TCP Test:
First, port-forward to the iperf3 server:
kubectl port-forward service/iperf3-server 5201:5201
Then, in a separate terminal, run the iperf3 client:
iperf3 -c 127.0.0.1
You’ll see output detailing the TCP connection, including bandwidth, jitter, and loss.
UDP Test:
To test UDP, we’ll specify the -u flag and a bitrate. Let’s try to saturate a 10Gbps link (adjust based on your cluster’s capacity).
iperf3 -c 127.0.0.1 -u -b 10G
Pay close attention to the "sender" and "receiver" reports. For UDP, packet loss is critical. Cilium’s eBPF data path is designed to minimize this.
4. Running netperf Tests
netperf provides more granular control and different test types. We’ll use netperf client pods to connect to our netperf-server.
Create a netperf-client deployment:
apiVersion: apps/v1
kind: Deployment
metadata:
name: netperf-client
spec:
replicas: 1
selector:
matchLabels:
app: netperf-client
template:
metadata:
labels:
app: netperf-client
spec:
containers:
- name: netperf
image: jasonacox/netperf
command: ["sleep", "infinity"] # Keep the pod running
Apply this with kubectl apply -f netperf-client.yaml.
Once the client pod is running, exec into it:
kubectl exec -it $(kubectl get pods -l app=netperf-client -o jsonpath='{.items[0].metadata.name}') -- bash
Inside the pod, connect to the netperf-server. You’ll need the cluster-internal IP of the netperf-server service. Find it with kubectl get svc netperf-server -o jsonpath='{.spec.clusterIP}'. Let’s assume it’s 10.100.0.5.
TCP Throughput Test (TCP_STREAM):
netperf -H 10.100.0.5 -t TCP_STREAM
This will measure the maximum TCP send throughput.
TCP Latency Test (TCP_RR):
netperf -H 10.100.0.5 -t TCP_RR -r 1024,0
This measures round-trip time for requests, useful for latency. The -r 1024,0 sets the send and receive buffer sizes.
UDP Throughput Test (UDP_STREAM):
netperf -H 10.100.0.5 -t UDP_STREAM -l 1000000 -u
The -l flag specifies the test data length in bytes. The -u flag indicates UDP. You can also specify a rate with -f (e.g., -f 1000000 for 1M bytes/sec).
Key Metrics to Watch:
- Bandwidth/Throughput: Measured in Mbps or Gbps. Higher is better.
- Latency/RTT: Measured in microseconds or milliseconds. Lower is better.
- Packet Loss: Crucial for UDP. Even 0.1% loss can drastically impact performance. Cilium aims for 0% loss.
- Jitter: Variation in packet arrival time. Lower is better.
Understanding Cilium’s Advantage
Cilium leverages eBPF to bypass much of the traditional kernel networking stack for packet processing. Instead of traversing multiple kernel layers (like iptables, conntrack, kube-proxy rules), eBPF programs attached directly to network interfaces or socket operations handle these tasks at a much lower level. This drastically reduces CPU overhead and context switching, leading to higher throughput and lower latency. For example, when kube-proxy replacement is enabled, Cilium handles service load balancing directly in eBPF, eliminating the iptables hop and its associated performance penalty.
When you run these benchmarks, you’re not just measuring raw network speed; you’re measuring how efficiently Cilium’s eBPF programs can process and forward your network traffic, especially under load. The absence of packet loss, even at high rates, is a testament to its efficient data path.
The next step is often to explore how different network policies or advanced features like load balancing and encryption impact these benchmark results.