BuildKit’s registry cache is a powerful way to speed up builds by sharing intermediate build layers across different machines.
Here’s how it works in practice. Imagine you’re building a Docker image on your local machine, and then you need to build the same image on a CI server. Without registry caching, the CI server would have to re-download and rebuild every single layer from scratch, even if those layers haven’t changed. This can take a significant amount of time, especially for large images or complex build processes.
With registry cache, BuildKit uploads these intermediate layers to a Docker registry (like Docker Hub, AWS ECR, or your own private registry) as it builds. When another machine needs to build the same image, BuildKit first checks the registry for existing layers that match the current build’s requirements. If it finds them, it downloads those layers instead of rebuilding them, dramatically reducing build times.
Let’s look at a typical workflow.
First, you need to configure BuildKit to use a registry for caching. This is often done via environment variables or build arguments. A common setup involves specifying the registry hostname and potentially a specific cache image name.
# Example environment variables for BuildKit
export BUILDKIT_HOST="tcp://docker.your-registry.com:2376"
export DOCKER_CONFIG=$(pwd)/.dockerconfig # if using custom auth
export DOCKER_USERNAME="your-registry-user"
export DOCKER_PASSWORD="your-registry-password"
# Or using build arguments when running buildx
docker buildx build --cache-from docker.your-registry.com/my-org/my-app:cache \
--tag docker.your-registry.com/my-org/my-app:latest .
In this example, docker.your-registry.com/my-org/my-app:cache is the image that BuildKit will use to store and retrieve cache layers. It’s important to note that this isn’t a traditional image you’d pull and run; it’s a special image managed by BuildKit to store its cache blobs.
Here’s a Dockerfile that benefits from this setup:
# syntax=docker/dockerfile:1
FROM alpine:3.18 as builder
RUN apk add --no-cache git build-base
RUN git clone https://github.com/someuser/someproject.git /app
WORKDIR /app
RUN make build # This step can be cached
FROM alpine:3.18
COPY --from=builder /app/myapp /usr/local/bin/myapp
CMD ["myapp"]
When you build this with the registry cache enabled, BuildKit will:
- Build the
builderstage. - During the
RUN git clonestep, if the layer with the cloned repository already exists in the registry cache and thegit clonecommand is identical, BuildKit will reuse that layer. - During the
RUN make buildstep, if the source code hasn’t changed and themake buildcommand is the same, the result from the cache will be used. - Finally, it copies the built
myappbinary to the final image.
The key is that the intermediate layers, even large ones like the cloned repository or the compiled binary, are pushed to and pulled from the registry. This means if another build job starts on a different machine with the same cache reference, it can download these pre-built layers and skip the git clone and make build steps entirely, provided the inputs to those steps haven’t changed.
The cache-from flag is crucial here. It tells BuildKit where to look for existing cache layers. If you’re building an image tagged my-app:latest, you might use my-app:cache as the --cache-from reference. BuildKit intelligently maps these.
The underlying mechanism involves BuildKit storing cache manifests and blobs as image layers in the specified registry. When a build starts, BuildKit queries the registry for layers matching the current build’s instructions and inputs. If a match is found, it downloads the layer’s content and rehydrates the build cache on the local machine. Conversely, as the build progresses, new intermediate layers are pushed to the registry, making them available for future builds.
One aspect that often surprises people is how granularly BuildKit can cache. It doesn’t just cache entire stages; it can cache individual commands within a stage. If you have a RUN command that takes a long time but its inputs haven’t changed, BuildKit can potentially skip executing it by reusing a cached result from the registry, even if other commands in the same stage have changed. This is why the order of commands in your Dockerfile and the immutability of your build inputs are so important for effective caching.
The next step in optimizing your build process might involve exploring BuildKit’s advanced cache policies, such as using different cache backends or configuring cache expiration.