The Network Not Ready: CNI Not Initialized error in containerd means the container runtime couldn’t set up network interfaces for your containers because the Container Network Interface (CNI) plugin failed to load or execute.

Common Causes and Fixes

1. CNI Configuration Files Missing or Incorrectly Formatted: containerd relies on CNI configuration files (usually in /etc/cni/net.d/) to know which network plugins to use and how to configure them. If these files are missing, empty, or malformed (e.g., invalid JSON), the CNI plugin will fail.

  • Diagnosis: Check for files in /etc/cni/net.d/.
    ls -l /etc/cni/net.d/
    
    If the directory is empty or contains non-JSON files, or if existing files are not valid JSON, this is your problem. You can validate JSON with jq:
    jq . /etc/cni/net.d/your-cni-config.conf
    
  • Fix: Ensure you have at least one valid CNI configuration file. For a basic setup using bridge and host-local, a common configuration looks like this: /etc/cni/net.d/10-my-cni.conf
    {
        "cniVersion": "0.3.1",
        "name": "mynet",
        "type": "bridge",
        "bridge": "cnibr0",
        "isGateway": true,
        "ipMasq": true,
        "ipam": {
            "type": "host-local",
            "subnet": "10.244.0.0/16",
            "routes": [
                { "dst": "0.0.0.0/0" }
            ]
        }
    }
    
    This configuration tells the CNI plugin to create a bridge network named cnibr0, assign IPs from the 10.244.0.0/16 subnet, and set up IP masquerading.
  • Why it works: This provides containerd with the necessary instructions to load and use the appropriate CNI plugins for network setup.

2. CNI Binaries Not Found or Executable: The CNI plugins themselves are executables (e.g., bridge, host-local, loopback). These need to be present in a directory listed in the PATH environment variable for the CNI process, and they must have execute permissions.

  • Diagnosis: Check if CNI binaries are in /opt/cni/bin/ and are executable.
    ls -l /opt/cni/bin/
    
    Look for files like bridge, host-local, loopback, etc. Ensure they have x in their permissions (e.g., -rwxr-xr-x).
  • Fix: Install the CNI plugins package for your distribution or ensure they are correctly placed and executable. For example, on systems using apt:
    sudo apt-get update
    sudo apt-get install containerd.io cni-plugins
    
    If they are already installed but not executable:
    sudo chmod +x /opt/cni/bin/*
    
  • Why it works: This ensures that when containerd’s CNI plugin invokes a network setup command (like bridge), the executable binary is found and can be run.

3. Incorrect containerd Configuration for CNI: The containerd configuration file (/etc/containerd/config.toml) specifies how containerd should interact with CNI. If the disabled_plugins list incorrectly includes CNI or if the root directory for CNI is misconfigured, it can cause this error.

  • Diagnosis: Examine /etc/containerd/config.toml.
    cat /etc/containerd/config.toml
    
    Look for a [plugins] section and specifically [plugins."io.containerd.grpc.v1.cri"]. Ensure cni is not listed under disabled_plugins. Also, verify the root path for CNI is correctly set if it’s explicitly defined.
  • Fix: If cni is in disabled_plugins, remove it. A typical configuration relevant to CNI would look something like this within config.toml:
    [plugins."io.containerd.grpc.v1.cri"]
      sandbox_image = "k8s.gcr.io/pause:3.5"
      [plugins."io.containerd.grpc.v1.cri".cni]
        bin_dir = "/opt/cni/bin"
        conf_dir = "/etc/cni/net.d"
    
    After editing, restart containerd:
    sudo systemctl restart containerd
    
  • Why it works: This ensures containerd is configured to use the CNI plugins and knows where to find their binaries and configuration files.

4. Insufficient File Descriptors or System Limits: When many containers are started, or if there are issues with resource limits, the CNI plugin might fail due to running out of available file descriptors or hitting other system-level constraints.

  • Diagnosis: Check system logs for messages related to file descriptor limits or ulimit.
    sudo journalctl -u containerd -f
    
    Also, check the system’s current limits:
    ulimit -n
    
  • Fix: Increase the file descriptor limit for the containerd process. This is often done by editing /etc/systemd/system/containerd.service.d/override.conf (or similar override file) and adding:
    [Service]
    LimitNOFILE=1048576
    
    Then reload systemd and restart containerd:
    sudo systemctl daemon-reload
    sudo systemctl restart containerd
    
  • Why it works: Providing more file descriptors allows the CNI plugin to manage the network interfaces and associated kernel resources for a larger number of containers without hitting OS limits.

5. Corrupted CNI State or IPAM Data: The host-local IPAM plugin (often used by default) stores its IP allocation data in /var/lib/cni/networks/. If this directory or its contents become corrupted, the IPAM plugin can fail, leading to CNI initialization errors.

  • Diagnosis: Inspect the contents of /var/lib/cni/networks/.
    ls -lR /var/lib/cni/networks/
    
    Look for malformed files or directories that seem out of place.
  • Fix: If you suspect corruption and can afford to reset IP allocations (e.g., in a development environment or if you know no critical containers are running), you can remove the stale state. Caution: This will cause IP conflicts if containers are still running and relying on these allocations.
    sudo rm -rf /var/lib/cni/networks/*
    
    Then restart containerd.
  • Why it works: This clears out any potentially corrupted IP address allocation records, allowing the host-local IPAM plugin to start fresh and re-allocate addresses.

6. Kernel Module Issues (e.g., overlay or bridge): CNI plugins often rely on specific kernel modules being loaded, such as overlay for storage drivers or bridge for network bridging. If these modules are not loaded or are malfunctioning, CNI operations can fail.

  • Diagnosis: Check if necessary kernel modules are loaded.
    lsmod | grep overlay
    lsmod | grep bridge
    
  • Fix: Load the required modules manually.
    sudo modprobe overlay
    sudo modprobe bridge
    
    To ensure they load on boot, you might need to add them to /etc/modules-load.d/containerd.conf:
    overlay
    bridge
    
  • Why it works: Ensures the underlying kernel functionality required by CNI plugins (like network bridging or specific storage drivers) is available.

After resolving these issues, you might encounter a failed to load Kubelet configuration error if you are using containerd with Kubernetes and the Kubelet configuration itself was dependent on the CNI being initialized correctly.

Want structured learning?

Take the full Containerd course →