containerd is failing to connect to your private registry over TLS because it doesn’t trust the registry’s certificate.

Common Causes and Fixes

  1. Registry Certificate Not Trusted by containerd Host:

    • Diagnosis: Check if the registry’s certificate (or its issuing CA certificate) is present in containerd’s trusted certificate store. On Linux, this is typically /etc/containerd/certs.d/<registry_host>:<port>/ca.crt.
    • Fix:
      • Copy the registry’s CA certificate (or the end-entity certificate if it’s self-signed) to the appropriate directory. For example, if your registry is myregistry.local:5000 and the CA cert is ca.crt on your local machine:
        sudo mkdir -p /etc/containerd/certs.d/myregistry.local:5000
        sudo cp ca.crt /etc/containerd/certs.d/myregistry.local:5000/ca.crt
        
      • Restart containerd:
        sudo systemctl restart containerd
        
    • Why it works: containerd checks the TLS certificate presented by the registry against a list of trusted Certificate Authorities. By adding the registry’s CA (or the certificate itself for self-signed cases) to this specific directory structure, you’re telling containerd to trust it.
  2. Incorrect Certificate File Permissions:

    • Diagnosis: The containerd process might not have read permissions for the certificate file.
    • Fix: Ensure the certificate file is readable by the user containerd runs as (often root):
      sudo chmod 644 /etc/containerd/certs.d/myregistry.local:5000/ca.crt
      
    • Why it works: Standard file permissions prevent containerd from accessing the certificate, even if it’s in the right place. Correcting permissions allows containerd to read the file.
  3. Certificate Not Present in System-Wide Trust Store (Less Common for containerd Specific):

    • Diagnosis: While containerd uses its own certs.d directory, some environments might rely on the system’s CA bundle.
    • Fix: Add the CA certificate to the system’s trust store. On Debian/Ubuntu:
      sudo cp ca.crt /usr/local/share/ca-certificates/myregistry.crt
      sudo update-ca-certificates
      
      On RHEL/CentOS/Fedora:
      sudo cp ca.crt /etc/pki/ca-trust/source/anchors/myregistry.crt
      sudo update-ca-trust extract
      
      Then restart containerd.
    • Why it works: This makes the certificate trusted by the operating system, and containerd might pick it up as a fallback or if its configuration is set to use system CAs.
  4. Incorrect Registry Hostname/Port in certs.d Path:

    • Diagnosis: The directory name must exactly match the registry hostname and port as configured in containerd’s config.toml or as used in docker pull commands.
    • Fix: Verify config.toml (usually /etc/containerd/config.toml) and your pull commands. If your registry is registry.example.com:5443 and you configured it as registry.example.com or registry.example.com:5000, the path will be wrong. Correct the directory name:
      # Example: If registry is registry.example.com:5443
      sudo mv /etc/containerd/certs.d/registry.example.com:5000 /etc/containerd/certs.d/registry.example.com:5443
      sudo cp ca.crt /etc/containerd/certs.d/registry.example.com:5443/ca.crt
      sudo systemctl restart containerd
      
    • Why it works: containerd uses the hostname and port from the registry address to look up the corresponding CA certificate in its certs.d directory. A mismatch means it won’t find the correct trust anchor.
  5. Self-Signed Certificate Issues:

    • Diagnosis: If the registry uses a self-signed certificate (not issued by a public or private CA), the certificate itself must be placed in the ca.crt file, not just the CA.
    • Fix: For self-signed certificates, copy the actual certificate file (e.g., registry.pem) into the directory:
      sudo mkdir -p /etc/containerd/certs.d/myregistry.local:5000
      sudo cp registry.pem /etc/containerd/certs.d/myregistry.local:5000/ca.crt
      sudo systemctl restart containerd
      
    • Why it works: When a certificate is self-signed, it acts as its own root of trust. containerd needs to be explicitly told to trust this specific certificate.
  6. Expired Certificate:

    • Diagnosis: The registry’s TLS certificate has expired.
    • Fix: Obtain a new certificate for your registry that is not expired. Update the certificate files on the registry server and, if necessary, update the ca.crt on the containerd hosts if the new certificate is signed by a different CA.
    • Why it works: TLS relies on valid, non-expired certificates for security. An expired certificate is invalid and will be rejected by any TLS client.
  7. Intermediate CA Certificate Missing:

    • Diagnosis: The registry’s certificate is signed by an intermediate CA, and that intermediate CA certificate is not included in the certificate chain presented by the registry, nor is it trusted by containerd.
    • Fix: Ensure the registry server is configured to send the full certificate chain. If that’s not possible, you may need to bundle the intermediate CA certificate with the end-entity certificate on the registry, or add the intermediate CA’s certificate to the ca.crt file in containerd’s certs.d directory (concatenating multiple CA certs if needed).
      # Concatenate your primary CA and the intermediate CA
      cat my_primary_ca.crt my_intermediate_ca.crt > bundled_ca.crt
      sudo cp bundled_ca.crt /etc/containerd/certs.d/myregistry.local:5000/ca.crt
      sudo systemctl restart containerd
      
    • Why it works: TLS clients verify a certificate by tracing its signature back to a trusted root CA. If an intermediate CA is missing from the chain presented by the server, the client cannot complete this validation chain unless it already trusts the intermediate CA directly.

After fixing certificate trust issues, you might encounter unauthorized errors if the registry’s authentication configuration is incorrect or missing credentials.

Want structured learning?

Take the full Containerd course →