containerd’s content store is where all image layers and metadata live.
Let’s watch it in action.
# First, find a running containerd container
CONTAINER_ID=$(sudo crictl ps -q | head -n 1)
# Now, get the image ID for that container
IMAGE_ID=$(sudo crictl inspect --output json $CONTAINER_ID | jq -r '.image | .id')
# With the image ID, we can inspect the image's layers.
# Note: This is a simplified view. containerd uses content hashes internally.
# We're looking at the manifest, which points to layer blobs.
IMAGE_MANIFEST=$(sudo ctr content get docker.io/library/nginx:latest | jq '.config')
# The manifest contains a digest for the image config, which in turn
# has digests for the layers. These digests are the actual content hashes.
CONFIG_DIGEST=$(echo $IMAGE_MANIFEST | jq -r '.digest')
# To see the actual layer blobs, we'd typically query the content store directly
# using these digests. For example, to fetch the config blob itself:
sudo ctr content get $CONFIG_DIGEST
This ctr content command is your gateway. It lets you interact with the blob store that underpins containerd’s image management. Every image layer, every image configuration, every manifest you pull is stored as an immutable blob identified by its cryptographic hash (a digest). When you pull nginx:latest, containerd doesn’t just download files; it downloads blobs and stores them based on their content. If another image uses the exact same layer, containerd just references the existing blob. This deduplication is a huge win for storage efficiency.
The core of it is the content subcommand of ctr.
ctr content ls: Lists all blobs (layers, manifests, configs) currently in the store. You’ll see digests and sizes.ctr content get <digest>: Fetches the content (the actual bytes of the layer or manifest) associated with a given digest.ctr content rm <digest>: Removes a specific blob from the store. This is dangerous if other images or containers still need it.ctr content pin <digest>: Marks a blob as "pinned," preventing it from being garbage collected. This is crucial for keeping images and layers that are actively in use.ctr content unpin <digest>: Removes the pin, making the blob eligible for garbage collection.
The command ctr content ls will show you a list of digests. These are SHA256 hashes of the content. For example, you might see something like:
sha256:a3ed95ca8bb1f77c00390c9c7a01f9f5b4e3f9e6b7445783505c3e4134195551
This digest represents a specific piece of data – a layer, a manifest, or a configuration file. When you pull an image, containerd resolves the image’s manifest, which then lists the digests of all the layer blobs required for that image. It checks its content store for these digests. If a blob is already present, it’s reused. If not, it’s downloaded.
The real magic, and the source of most confusion, is how containerd manages what stays and what goes. This is garbage collection. By default, containerd garbage collects unreferenced blobs. However, images and containers create references. ctr content pin is how you manually assert that a blob is still needed, overriding the automatic garbage collection for that specific digest.
When you remove an image using docker rmi or crictl rmi, containerd doesn’t immediately delete the underlying blobs. It decrements reference counts. Only when a blob is no longer referenced by any image or any running container and is not explicitly pinned, will it become eligible for garbage collection. You can trigger garbage collection manually:
sudo ctr content gc
This command walks through the content store, identifies blobs that are not pinned and not referenced by any active image or container, and removes them. It’s the system’s way of cleaning up old, unused layers to free up disk space.
The most surprising thing is that the content store doesn’t inherently know about "images" or "layers" in the way you think of them. It just stores blobs keyed by their content hash. The concepts of "image" and "layer" are built on top of this blob store by the image service, which uses manifests and configuration files (also stored as blobs) to define how these individual content blobs assemble into a coherent image.
If you try to ctr content rm a digest that’s still part of an image that might be pulled again, or is even referenced by a container that’s just stopped but not yet removed, you’ll get an error. The system is designed to prevent accidental data loss. You’ll then see an error related to the image service not being able to find a required blob, even though you thought you deleted it.