It’s surprisingly easy to expose your BuildKit daemon to the network, but the security implications are often overlooked.
Here’s a BuildKit daemon running locally, serving builds over a Unix socket:
docker run -d --name buildkitd -v /path/to/buildkit/data:/var/lib/buildkit moby/buildkitd:latest
To connect to this, you’d typically use DOCKER_BUILDKIT=1 docker build ... with DOCKER_HOST=unix:///var/run/docker.sock if you were using the Docker daemon, or more directly with BUILDKIT_HOST=unix:///path/to/buildkit.sock.
Now, let’s expose it over TCP. The key is the --addr flag for the buildkitd command. We’ll bind it to 0.0.0.0:1234, making it accessible on all network interfaces on port 1234.
docker run -d --name buildkitd-tcp \
-p 1234:1234 \
-v /path/to/buildkit/data:/var/lib/buildkit \
moby/buildkitd:latest \
buildkitd --addr tcp://0.0.0.0:1234 --oci-snapshotter overlayfs
Notice -p 1234:1234 maps the container’s port 1234 to the host’s port 1234. The --oci-snapshotter overlayfs is a common and performant option for the snapshotter, essential for efficient layer management.
To connect to this remote daemon, your client needs to know its address. You’ll set the BUILDKIT_HOST environment variable. If your remote BuildKit daemon is accessible at 192.168.1.100, you’d run your build command like this:
BUILDKIT_HOST=tcp://192.168.1.100:1234 \
DOCKER_BUILDKIT=1 \
docker build --frontend dockerfile.v1 --local context=. --output type=local,dest=./out .
This command tells your Docker client (with BuildKit enabled) to direct all build operations to the specified TCP address instead of the default Docker daemon or a local BuildKit socket. The DOCKER_BUILDKIT=1 ensures that the docker build command actually uses BuildKit. The --frontend dockerfile.v1 specifies the build system, --local context=. points to the build context on your client, and --output type=local,dest=./out directs the build artifacts to a local directory.
The primary problem this solves is decoupling the build process from the local Docker daemon or a specific machine. You can have a powerful build machine running BuildKit and have multiple developer machines or CI agents connect to it over the network, sharing the build infrastructure. This is particularly useful for large projects where building locally is slow or resource-intensive.
BuildKit’s internal architecture leverages a distributed, declarative, and parallel execution engine. When you connect via TCP, your client sends build requests (including the build definition, context, and any necessary secrets) to the daemon. The daemon then orchestrates the build, pulling base images, executing build steps, caching intermediate results, and ultimately sending the final build artifact back to the client. The TCP connection is the communication channel for this entire process.
The daemon itself manages image layers, build cache, and worker processes. The --oci-snapshotter flag tells BuildKit which storage driver to use for managing image layers on the daemon’s host; overlayfs is a common choice on Linux. Other options include native (which might use btrfs or zfs depending on the host) or stargz. The choice impacts performance and disk usage.
You can also configure BuildKit with more advanced options, such as authentication, by passing arguments to buildkitd. For instance, to enable TLS, you might run:
docker run -d --name buildkitd-tls \
-p 1234:1234 \
-v /path/to/buildkit/data:/var/lib/buildkit \
-v /path/to/certs:/certs \
moby/buildkitd:latest \
buildkitd --addr tcp://0.0.0.0:1234 \
--tls-cert-file /certs/cert.pem \
--tls-key-file /certs/key.pem \
--oci-snapshotter overlayfs
Then, your client would connect using the tcp:// prefix and potentially additional TLS options, though the client-side configuration for TLS with BuildKit can be complex and often involves environment variables or specific client tool flags depending on the client.
The most surprising thing about exposing BuildKit over TCP is how little configuration is needed on the daemon side to make it accessible, but how much is required to make it secure. By default, it’s an open invitation. Anyone who can reach that TCP port can potentially trigger builds, consume resources, and access cached artifacts.
When you connect using BUILDKIT_HOST=tcp://..., the client establishes a gRPC connection to the BuildKit daemon. This connection is used to send build requests, receive build progress updates, and transfer build outputs. The daemon processes these requests, utilizing its local resources (CPU, memory, disk for cache and layers) to perform the build. If the daemon is running on a separate, more powerful machine, this offloads the build workload from your local machine.
The context in the docker build command is sent over the network to the BuildKit daemon. This means that if your build context is very large, you’ll be transferring that entire context over the network to the daemon for each build, which can be a significant bottleneck. BuildKit has features like buildkit-remote, which can help mitigate this by using remote sources for the context, but the basic TCP connection doesn’t inherently optimize for large context transfers.
If you’re seeing connection refused errors when trying to connect to your remote BuildKit daemon, double-check that the port mapping in your docker run command for buildkitd is correct (e.g., -p 1234:1234) and that there are no firewall rules on the host running the daemon blocking incoming connections on port 1234. You can test basic network reachability with nc -vz 192.168.1.100 1234 from your client machine.
The next step after successfully connecting to a remote BuildKit daemon is often integrating it into a CI/CD pipeline, where managing credentials and ensuring secure communication become paramount.