containerd runc vs Kata Containers: Choose the Right Runtime
Kata Containers don’t just run containers in a VM; they run them in a lightweight VM that starts in milliseconds, offering near-native performance with enhanced security.
Here’s a container running a simple web server, served directly by containerd using the default runc runtime, and then the same container running under Kata Containers.
Using runc (default):
First, ensure you have containerd installed and running. You can check its status with:
sudo systemctl status containerd
To run a container with runc, you’ll typically use ctr, containerd’s low-level CLI. Let’s pull and run an nginx image:
# Pull the image
sudo ctr images pull docker.io/library/nginx:latest
# Create a namespace
sudo ctr ns create my-nginx-ns
# Run the container in the namespace
sudo ctr run --rm --user 0 \
docker.io/library/nginx:latest \
my-nginx-container \
ctr-content-oci/my-nginx-container
# You can verify it's running (though `ctr run --rm` will exit quickly)
# For a long-running container, omit `--rm` and check with `sudo ctr c ls -n my-nginx-ns`
If you were to inspect the runtime used by runc (which is the default for containerd), you’d see it’s a standard OCI runtime, creating processes directly on the host kernel.
Using Kata Containers:
Kata Containers require a bit more setup. You’ll need to install the Kata Containers runtime and its dependencies, including QEMU or another hypervisor. The installation process typically involves downloading a release binary and configuring containerd to recognize Kata as a valid runtime.
Once installed, you’ll need to configure containerd’s config.toml to enable the Kata runtime. Find your containerd configuration file (often at /etc/containerd/config.toml or /usr/local/etc/containerd/config.toml). You’ll need to add a section like this under [plugins."io.containerd.grpc.v1.cri".containerd.runtimes]:
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.kata]
privileged_without_host_devices = true
runtime_engine = ""
runtime_root = ""
runtime_type = "io.kata-containers.kata"
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.kata.options]
ConfigPath = "/etc/kata-containers/configuration.toml"
After modifying config.toml, restart containerd:
sudo systemctl restart containerd
Now, to run the same nginx container using Kata, you specify the runtime in your Kubernetes pod definition or, if using ctr directly, with a specific flag:
# For demonstration, let's simulate a pod creation that would use Kata.
# In a real Kubernetes setup, you'd use pod annotations to select the runtime.
# With ctr, you'd typically configure it via a shim that specifies Kata.
# The exact command to *directly* launch a container with Kata via `ctr`
# can be complex as it often involves a specific shim.
# A more common scenario is via Kubernetes.
# If you were using Kubernetes, your pod spec would look something like this:
# apiVersion: v1
# kind: Pod
# metadata:
# name: nginx-kata
# annotations:
# io.kubernetes.container.runtime.class: kata
# spec:
# containers:
# - name: nginx
# image: nginx:latest
# ports:
# - containerPort: 80
The key difference you’d observe is that the container, while appearing to run as a normal container from a user perspective, is actually instantiated within a dedicated, minimal VM managed by Kata Containers. This VM is provisioned and destroyed rapidly, providing isolation without the significant overhead of traditional VMs.
The Problem They Solve
runc, as the de facto standard OCI runtime, excels at speed and efficiency. It creates processes directly on the host OS’s kernel, making container startup nearly instantaneous and resource consumption minimal. This is perfect for stateless applications, microservices, and CI/CD pipelines where rapid deployment and scaling are paramount. However, its isolation model relies on kernel namespaces and cgroups, which, while robust, share the host kernel. A vulnerability in the kernel could potentially affect all containers running on that host.
Kata Containers, on the other hand, address the security concerns arising from shared kernel isolation. They achieve this by running each container (or a small group of containers) within its own lightweight virtual machine. This VM has its own kernel, completely isolated from the host kernel and other VMs. This means that even if a severe vulnerability were discovered in the container’s workload or the guest kernel, it would be contained within the VM and would not compromise the host or other containers.
How They Work Internally
runc:
runc is a low-level OCI (Open Container Initiative) runtime. When containerd needs to start a container, it generates an OCI-compliant runtime configuration (a config.json file) and then executes runc init. runc then uses Linux’s clone() system call to create new namespaces (PID, network, mount, UTS, IPC, user) and cgroups to isolate the container’s processes from the host. The container’s root filesystem is mounted, and the specified entrypoint command is executed within this isolated environment. It’s a direct, kernel-level isolation mechanism.
Kata Containers: Kata Containers employ a different strategy. They act as a "shim" for containerd, but instead of directly creating processes on the host, they interact with a hypervisor (like QEMU, Firecracker, or even gVisor). When a container is requested, Kata spins up a minimal VM. This VM has its own kernel. A small agent process runs inside this VM, and it’s this agent that actually executes the container’s process. The container’s root filesystem is typically mounted into the VM using virtio-fs or a similar mechanism. This VM-based approach provides stronger security isolation because the container’s workload never directly touches the host kernel. The VM itself is designed to be extremely lightweight, with boot times often in the tens to hundreds of milliseconds, making the performance penalty much lower than traditional VMs.
Levers You Control
With runc (via containerd’s CRI plugin):
- Resource Limits: You control CPU, memory, and I/O limits using cgroups, configured through Kubernetes resource requests/limits or directly in containerd’s CRI configuration.
- Privileged Mode: You can grant containers elevated privileges, allowing them direct access to host devices and kernel capabilities, though this significantly reduces isolation.
- AppArmor/SELinux Profiles: You can apply security profiles to further restrict what a container’s processes can do on the host.
With Kata Containers (also managed by containerd’s CRI plugin, but configured differently):
- Hypervisor Choice: You can select which hypervisor Kata uses (QEMU, Firecracker, etc.), impacting performance and resource usage.
- Guest Kernel Configuration: You can sometimes tune the kernel running inside the Kata VM.
- VM Resources: While not directly managed by Kubernetes resource requests in the same way as
runc, the underlying VM configuration can be tuned for CPU and memory. - Security Features: Kata offers features like TPM attestation and hardware-assisted virtualization for enhanced security.
The One Thing Most People Don’t Know
The perceived "slowness" of VMs is often tied to their boot times and the overhead of full hardware emulation. Kata Containers dramatically mitigate this by using minimal VMs with highly optimized guest kernels and fast boot mechanisms like virtio-fs for storage. For instance, using the Firecracker hypervisor, Kata can achieve VM startup times under 100ms, blurring the lines between traditional container performance and VM isolation. This speed is achieved through a combination of Rust-based hypervisor implementations, lean guest kernels, and efficient I/O paths.
The next concept to explore is how to integrate these runtimes into a Kubernetes cluster using Runtime Classes, allowing you to orchestrate workloads with different isolation and security requirements.