containerd’s image annotation feature lets you attach arbitrary metadata to container images, which can be incredibly useful for things like build provenance, security policies, or custom labels for orchestration.
Let’s see this in action. Imagine we’ve built an image and want to add some provenance information to it.
# First, build a simple image (or use an existing one)
echo "FROM alpine:latest\nRUN echo 'hello world' > /app/hello.txt" | ctr images import -
IMAGE_ID=$(ctr images list --format '{{.Name}}' | grep alpine)
# Add an annotation
ctr images commit --annotation "build.example.com/git-sha=abcdef123456" $IMAGE_ID annotated-alpine:latest
# Now, let's try to read that annotation
ctr images inspect annotated-alpine:latest --format '{{.Annotations}}'
This will output something like:
map[build.example.com/git-sha:abcdef123456]
The core idea behind annotations is simple: they’re key-value pairs associated with an image’s manifest. When you push or pull an image, these annotations travel with it. containerd stores these annotations within its internal content-addressable store, linked to the specific image manifest digest. This means the annotations are immutable once set for a given manifest version. If you want to change an annotation, you’re effectively creating a new image manifest with the updated metadata.
You can add annotations during the image build process using tools like Docker or Buildah, which then push these annotations to the registry. containerd, when pulling these images, preserves them. Alternatively, as shown above, you can add or modify annotations on existing images within containerd using the ctr CLI. The ctr images commit command is particularly handy here, as it allows you to create a new image reference with added or modified annotations without re-uploading the image layers.
The ctr images inspect command is your primary tool for retrieving these annotations. You can use Go template formatting to selectively pull out specific annotations or all of them. This is crucial for automated systems that need to programmatically access this metadata. For example, a CI/CD pipeline might check for specific security annotations before deploying an image.
Internally, containerd’s image store is built around content-addressable storage. Image manifests, which include the list of layers and, importantly, the annotations, are stored as blobs identified by their content hash. When you add an annotation, you’re essentially creating a new manifest blob that references the same layer blobs but with the added metadata. This ensures that the image content itself remains unchanged, and only the metadata associated with its representation is updated.
The most surprising thing about annotations is how they interact with image immutability. While the content of an image’s layers is immutable (tied to its content hash), the manifest itself can be updated with new annotations. This means you can, in a sense, "re-tag" or "re-annotate" an image without changing its underlying layers. This is achieved by creating a new manifest that points to the existing layers but includes the updated annotations. Tools like ctr images commit abstract this process, making it seem like a simple modification, but under the hood, a new manifest is generated and stored.
The next step after mastering image annotations is understanding how to integrate them with image signing and verification to ensure the integrity and authenticity of your container images.