eBPF XDP programs don’t just speed up packet processing; they fundamentally change where that processing happens, moving it out of the kernel’s general-purpose networking stack and directly into the network interface card’s driver.
Let’s see XDP in action. Imagine you have a simple packet counter.
package main
import (
"fmt"
"log"
"net"
"os"
"os/signal"
"syscall"
"time"
"github.com/cilium/ebpf"
"github.com/cilium/ebpf/link"
"github.com/cilium/ebpf.examples/bpf" // Assuming this is a path to example BPF programs
)
// $BPF_CLANG $BPF_CFLAGS -O2 -g -target bpf -c xdp_counter.c -o xdp_counter.o
/*
//go:embed xdp_counter.o
var bpfProgramBytes []byte
*/
func main() {
// Load the BPF program
spec, err := ebpf.LoadCollectionSpecFromReader(bpf.XdpCounter) // Assuming bpf.XdpCounter is an embedded byte array of the compiled BPF object
if err != nil {
log.Fatalf("Failed to load BPF collection spec: %v", err)
}
// Load the BPF objects into the kernel
coll, err := ebpf.NewCollectionWithOptions(spec, ebpf.CollectionOptions{
Programs: ebpf.ProgramOptions{
// If you use BPF_PROG_TYPE_XDP, you must specify the kernel version
// and the BPF_PROG_TYPE_XDP.
// If you want to load a BPF program of type BPF_PROG_TYPE_XDP,
// you MUST specify the kernel version.
// If you want to load a BPF program of type BPF_PROG_TYPE_XDP,
// you MUST specify the BPF_PROG_TYPE_XDP.
KernelVersion: ebpf.KernelVersion{Major: 5, Minor: 0}, // Example: Requires kernel 5.0+
ProgType: ebpf.ProctypeXDP,
},
})
if err != nil {
log.Fatalf("Failed to create BPF collection: %v", err)
}
defer coll.Close()
// Get the XDP program
xdpProg, ok := coll.Programs["xdp_progfunc"] // Assuming the program name in the BPF object is "xdp_progfunc"
if !ok {
log.Fatalf("Could not find XDP program 'xdp_progfunc' in BPF object")
}
// Find a network interface to attach to. For simplicity, we'll use the first one.
// In a real application, you'd want a more robust way to select an interface.
ifaces, err := net.Interfaces()
if err != nil {
log.Fatalf("Could not list network interfaces: %v", err)
}
var iface net.Interface
for _, i := range ifaces {
// Skip loopback and down interfaces
if (i.Flags&net.FlagLoopback == 0) && (i.Flags&net.FlagUp != 0) {
ifaces = []net.Interface{i} // Use this interface
break
}
}
if len(ifaces) == 0 {
log.Fatal("No suitable network interface found.")
}
iface = ifaces[0]
log.Printf("Attaching XDP program to interface %s", iface.Name)
// Attach the XDP program to the interface
// XDP_FLAGS_SKB_MODE is a fallback for older kernels or when hardware offload isn't available.
// XDP_FLAGS_DRV_MODE is preferred for performance.
// XDP_FLAGS_HW_MODE attempts hardware offload.
// We'll try DRV_MODE first.
l, err := link.AttachXDP(link.XDPOptions{
Program: xdpProg,
Iface: iface.Name,
Flags: link.XDPFlags(link.XDP_FLAGS_DRV_MODE), // Try driver mode for best performance
})
if err != nil {
// If driver mode fails, try SKB mode as a fallback
log.Printf("Failed to attach XDP in DRV_MODE (%v), trying SKB_MODE...", err)
l, err = link.AttachXDP(link.XDPOptions{
Program: xdpProg,
Iface: iface.Name,
Flags: link.XDP_FLAGS_SKB_MODE,
})
if err != nil {
log.Fatalf("Failed to attach XDP in SKB_MODE: %v", err)
}
}
defer l.Close()
log.Printf("XDP program attached to %s. Press Ctrl+C to detach.", iface.Name)
// Access the BPF map to read counts
// Assuming the map name in the BPF object is "packet_count"
countMap, err := ebpf.MapFromCollection(coll, "packet_count")
if err != nil {
log.Fatalf("Failed to find BPF map 'packet_count': %v", err)
}
// Periodically print the packet counts
ticker := time.NewTicker(2 * time.Second)
defer ticker.Stop()
sig := make(chan os.Signal, 1)
signal.Notify(sig, syscall.SIGINT, syscall.SIGTERM)
go func() {
for range ticker.C {
var count uint64
if err := countMap.Lookup(nil, &count); err != nil { // Lookup with nil key for a single-value map
log.Printf("Error reading count map: %v", err)
continue
}
fmt.Printf("Packets processed: %d\n", count)
}
}()
<-sig
log.Println("Detaching XDP program...")
}
// Minimal BPF C code for xdp_counter.o (assuming this is what bpf.XdpCounter points to)
/*
#include <linux/bpf.h>
#include <bpf/bpf_helpers.h>
struct {
__uint(type, BPF_MAP_TYPE_ARRAY);
__uint(max_entries, 1);
__uint(key_size, sizeof(u32));
__uint(value_size, sizeof(u64));
} packet_count SEC(".maps");
SEC("xdp_progfunc")
int xdp_progfunc(struct xdp_md *ctx) {
u32 key = 0;
u64 *count;
count = bpf_map_lookup_elem(&packet_count, &key);
if (count) {
(*count)++;
}
return XDP_PASS; // Pass the packet to the next stage
}
char _license[] SEC("license") = "GPL";
*/
The core idea is to bypass the kernel’s normal network stack processing. When a packet arrives at the NIC, instead of the driver handing it off to the kernel’s struct sk_buff and then to the IP layer, XDP allows a small eBPF program to intercept it even earlier. This program runs in a highly restricted, safe environment within the kernel. It can inspect the packet, modify it, drop it, or redirect it to userspace.
The xdp_progfunc in the C code is the entry point. It receives an xdp_md (XDP metadata) struct, which provides access to the packet’s data buffer and its length. The program looks up a counter in a BPF map, increments it, and then returns an action. XDP_PASS means "let this packet continue through the normal kernel stack." Other actions like XDP_DROP will discard the packet immediately, and XDP_REDIRECT can send it to another interface or userspace.
The Go code acts as the loader and manager. It compiles the BPF C code (or loads a pre-compiled .o file), loads the eBPF bytecode into the kernel, and attaches the program to a specific network interface. link.AttachXDP is the key function here. It takes the eBPF program and the interface name, and tells the kernel to execute your eBPF program for incoming packets on that interface.
The xdp_counter.o file contains the compiled eBPF bytecode. The Go program uses the ebpf library to load this bytecode. ebpf.LoadCollectionSpecFromReader reads the program and its associated maps. ebpf.NewCollectionWithOptions loads these into the kernel, with ProgType: ebpf.ProctypeXDP being crucial. link.AttachXDP then hooks this loaded program into the specified network interface driver.
The BPF map (packet_count in this example) is how the eBPF program communicates with userspace. It’s a shared memory region. The eBPF program can write to it, and the Go program can read from it. Here, we’re using a simple array map with a single entry to store the packet count.
The real magic happens with link.XDP_FLAGS_DRV_MODE. This flag attempts to run the XDP program directly in the NIC driver, before the packet even reaches the main kernel networking stack. This is where the "line rate" performance comes from – you’re processing packets at the earliest possible point, minimizing CPU overhead and context switches. If DRV_MODE isn’t supported by the hardware or driver, it falls back to SKB_MODE, which runs the eBPF program on a copy of the packet (sk_buff) after it’s been partially processed by the driver, but still before the full kernel stack.
The system in action allows you to see network traffic being counted in near real-time, with minimal overhead, as the Go program polls the BPF map for updates.
The one thing most people don’t realize is that XDP programs can be attached in several modes: XDP_FLAGS_DRV_MODE (native driver mode, highest performance), XDP_FLAGS_SKB_MODE (software fallback mode, lower performance but more compatible), and XDP_FLAGS_HW_MODE (hardware offload, requires NIC support). The link.AttachXDP function and the underlying kernel mechanisms negotiate which mode to use, often preferring DRV_MODE if available.
Next, you’ll likely want to explore more complex filtering and manipulation, or redirecting packets to userspace for custom processing with AF_XDP.