The Docker daemon failed to execute a command within a container because the underlying OCI runtime (like runc) couldn’t start the container process.
1. Container Image Corrupted or Incomplete
- Diagnosis:
If the pull completes without errors but the problem persists, the issue might be with a locally cached layer.docker pull <image_name>:<tag>docker image prune -a docker pull <image_name>:<tag> - Fix: Re-pulling the image and pruning old layers ensures you have a clean, complete copy of the container image.
- Why it works: A corrupted image layer prevents the OCI runtime from accessing the necessary files and binaries to start the container process.
2. Insufficient Resources (Disk Space, Memory, Inodes)
- Diagnosis:
- Disk Space:
df -h(Check/var/lib/dockeror wherever your Docker root is configured). - Memory:
free -hand monitordocker stats <container_id>. - Inodes:
df -i(Check/var/lib/docker).
- Disk Space:
- Fix:
- Disk Space: Free up space by removing unused Docker images, containers, and volumes:
docker system prune -a --volumes. If a specific disk is full, clear it manually. - Memory: Increase system RAM, reduce memory limits on other containers, or tune the container’s memory limit (
--memory 512m). - Inodes: This is less common but can happen with many small files. Increase filesystem size or delete many small files.
- Disk Space: Free up space by removing unused Docker images, containers, and volumes:
- Why it works: The OCI runtime needs disk space for the container’s filesystem, memory to load the process, and inodes to represent files. Exhaustion of any of these prevents process creation.
3. Incorrect Permissions on Docker Socket or Runtime Files
- Diagnosis:
The user runningls -l /var/run/docker.sockdockercommands needs to be in thedockergroup, or the socket needs to be world-writable (less secure). Check permissions on/run/containerdor/run/runcif using containerd/runc directly. - Fix:
- Add your user to the
dockergroup:sudo usermod -aG docker $USER(requires re-login). - Ensure the Docker daemon has appropriate permissions to manage its runtime files.
- Add your user to the
- Why it works: The Docker daemon and the OCI runtime need to communicate. If the user running the
dockercommand cannot access the Docker socket, or if the runtime itself lacks permissions to its state directories, it can fail to launch processes.
4. Seccomp or AppArmor/SELinux Profile Blocking System Calls
- Diagnosis:
- Check system logs for
auditmessages ordeniedentries related to Docker or the container process:sudo ausearch -m avc -ts recentorsudo journalctl -f. - Temporarily disable seccomp for a container:
docker run --security-opt seccomp=unconfined ... - Temporarily disable AppArmor/SELinux for a container:
docker run --security-opt apparmor=unconfined ...ordocker run --security-opt label=disable ...(use with extreme caution).
- Check system logs for
- Fix:
- Adjust the seccomp profile to allow the necessary system calls. You can generate a profile from a working container.
- Modify your AppArmor or SELinux policies to permit the actions being blocked.
- Why it works: Seccomp, AppArmor, and SELinux are security mechanisms that restrict what system calls a process can make. If a profile incorrectly blocks a call required by the container’s entrypoint or command, the OCI runtime will fail to execute it.
5. Corrupted OCI Runtime Binary or Configuration
- Diagnosis:
Check the installation of the identified runtime (e.g.,docker info | grep "Runtimes"runc).
Check configuration files forsudo runc --versionrunc(e.g.,/etc/containerd/config.tomlif using containerd). - Fix: Reinstall the OCI runtime (e.g.,
sudo apt-get install --reinstall runcorsudo yum reinstall runc). If using a container runtime like containerd, ensure its configuration is correct. - Why it works: The OCI runtime is the actual executable that creates and runs the container process. If its binary is corrupted or its configuration is incorrect, it cannot perform its job.
6. Kernel Issues or Missing Kernel Modules
- Diagnosis: Check
dmesgfor kernel errors related to namespaces, cgroups, or file operations around the time the error occurs. Ensure necessary kernel modules for containerization (likeoverlayfs,veth) are loaded. - Fix: Update your kernel to a stable, supported version. Ensure the kernel configuration enables necessary features for containerization. Load required modules manually if missing (e.g.,
sudo modprobe overlay). - Why it works: Docker and OCI runtimes rely heavily on specific Linux kernel features (namespaces, cgroups, storage drivers). Kernel bugs or missing modules can prevent these features from working correctly, leading to execution failures.
The next error you’ll likely see if you’ve fixed the exec failure is a containerd-shim panic or a failed to start container message with a different underlying cause.