BuildKit can run as a non-root user, but it requires specific configurations to ensure security and proper operation.

Here’s a breakdown of how to get BuildKit running without root privileges, covering the essential setup and potential pitfalls.

Understanding the Core Problem

BuildKit, by default, often relies on root privileges for several reasons:

  • Container Runtime Access: It needs to interact with the Docker daemon or a compatible container runtime, which typically runs as root.
  • Filesystem Operations: Creating and managing build contexts, caching layers, and output artifacts can involve operations that require elevated permissions.
  • Network Access: Sometimes, building images requires privileged network access for downloading dependencies or communicating with registries.

Running BuildKit as a non-root user means you need to delegate or replicate these necessary permissions in a secure, user-specific manner.

The Non-Root BuildKit Setup

The primary way to run BuildKit without root is by using its client-server architecture and configuring the server to run under a specific user.

1. Install BuildKit

First, ensure BuildKit is installed on your system. This usually involves installing the buildkitd binary.

# Example installation for Debian/Ubuntu
apt-get update && apt-get install -y buildkit-daemon

2. Configure the BuildKit Daemon (buildkitd)

The buildkitd daemon needs to be configured to run as a non-root user. This is typically done via a configuration file, often located at /etc/buildkit/buildkitd.toml.

Here’s a sample buildkitd.toml for non-root operation:

# /etc/buildkit/buildkitd.toml

# Disable rootless mode if you're explicitly running as a user,
# but this is more for when buildkitd itself is launched by a rootless daemon.
# For manual non-root daemon startup, this isn't strictly necessary,
# but good to be aware of.
# rootless = true

# Define a specific user/group for buildkitd to run as.
# This requires that the user and group exist and have appropriate permissions.
# user = "buildkit:buildkit" # Example: if you create a 'buildkit' user/group

# If you're running buildkitd directly as a non-root user (e.g., via systemd user service),
# you might not need the 'user' directive here, as the process will inherit the user's ID.

# Configure the cache directory to be owned by the non-root user.
# This is crucial. The directory must be writable by the user running buildkitd.
[worker.oci]
  # Example: pointing to a user-owned directory
  cache-dir = "/home/youruser/.cache/buildkit/worker"

[worker.containerd]
  # If using containerd as a backend, ensure its paths are accessible.
  # This is less common for simple non-root setups.

# Configure storage driver if needed. Default is usually fine.

# Network configuration: buildkitd needs to listen on a TCP port.
# Ensure this port is accessible and not already in use by other services.
# By default, it might try to listen on 127.0.0.1:8080 or similar.
# You can explicitly set it:
# [grpc]
#   address = ["tcp://127.0.0.1:8080"]

# Logging configuration
[debug]
  # Log level can be set to 'debug', 'info', 'warn', 'error'
  level = "info"

# If you want to use rootless Docker daemon as the backend, you need to point to its socket.
# This is a common scenario for true rootless builds.
# [rootless]
#   daemon-mode = "docker"
#   docker-socket = "/run/user/1000/docker.sock" # Example path for rootless Docker

# For Podman as a backend:
# [rootless]
#   daemon-mode = "podman"
#   podman-endpoint = "unix:///run/user/1000/podman/podman.sock" # Example path

Key Configuration Points:

  • cache-dir: This is arguably the most critical setting. The directory specified for caching must be owned by the user running buildkitd. If buildkitd is run as youruser, then /home/youruser/.cache/buildkit/worker must be owned by youruser:youruser.
  • user directive: If you are starting buildkitd as root and want it to drop privileges to a specific user (e.g., via a systemd service file), use this. If you are starting buildkitd directly as a non-root user (e.g., sudo -u youruser buildkitd), then this directive is usually omitted, and the process runs with the privileges of youruser.
  • Backend Configuration ([rootless]): If you want to leverage an existing rootless container runtime (like rootless Docker or Podman), you need to specify its socket path. This allows BuildKit to delegate container creation and execution to that runtime.

3. Create Necessary Directories and Set Permissions

Before starting buildkitd, ensure the directories it needs are set up correctly.

# As the user who will run buildkitd (e.g., 'youruser')
mkdir -p /home/youruser/.cache/buildkit/worker
chown -R youruser:youruser /home/youruser/.cache/buildkit

4. Start the BuildKit Daemon

You can start buildkitd in several ways:

  • Manually (for testing):

    # As your non-root user
    buildkitd --config /etc/buildkit/buildkitd.toml
    

    This is good for initial testing. You’ll see output directly in your terminal.

  • Using systemd (user service): This is the recommended approach for persistent, background operation. Create a service file, e.g., ~/.config/systemd/user/buildkit.service:

    [Unit]
    Description=BuildKit Daemon
    Documentation=https://github.com/moby/buildkit
    After=network.target
    
    [Service]
    Type=simple
    ExecStart=/usr/bin/buildkitd --config /etc/buildkit/buildkitd.toml
    Restart=on-failure
    RestartSec=5
    # Optional: Set environment variables if buildkitd needs them
    # Environment="PATH=/usr/local/bin:/usr/bin:/bin"
    
    [Install]
    WantedBy=default.target
    

    Then enable and start it:

    systemctl --user enable buildkit.service
    systemctl --user start buildkit.service
    systemctl --user status buildkit.service
    

5. Configure the BuildKit Client (buildctl)

Your build clients (e.g., buildctl, Docker CLI configured to use BuildKit) need to connect to the running buildkitd instance.

By default, buildkitd might expose a gRPC API on localhost:8080. You can configure your client to use this.

  • Using buildctl:

    # Set the BuildKit endpoint as an environment variable
    export BUILDKIT_HOST="tcp://localhost:8080"
    
    # Now run your build commands
    buildctl build --frontend dockerfile.v1 --local context=. --local dockerfile=. --output type=image,name=my-nonroot-image:latest
    
  • Using Docker CLI: If you want the docker build command to use your non-root BuildKit, you need to configure the Docker CLI. Edit ~/.docker/config.json:

    {
      "features": {
        "buildkit": true
      },
      "buildkit-daemon.address": "tcp://localhost:8080"
    }
    

    With this, docker build will attempt to connect to your specified buildkitd instance.

Common Causes for Failure and How to Fix Them

  1. Permission Denied for Cache Directory:

    • Diagnosis: Check buildkitd logs (e.g., journalctl --user -u buildkit.service -f or terminal output) for errors like permission denied when accessing /path/to/cache.
    • Fix: Ensure the cache directory specified in buildkitd.toml is owned by the user running buildkitd.
      # Example: If buildkitd runs as 'youruser'
      chown -R youruser:youruser /home/youruser/.cache/buildkit
      
    • Why it works: BuildKit needs to write build cache layers and intermediate data. If the user running the daemon doesn’t own the cache directory, it cannot perform these essential write operations.
  2. Port Binding Issues:

    • Diagnosis: buildkitd logs might show address already in use or errors related to binding to 0.0.0.0:8080.
    • Fix:
      • If running manually, ensure no other process is using port 8080.
      • If using systemd and it fails, check sudo netstat -tulnp | grep 8080 or sudo lsof -i :8080.
      • Change the grpc.address in buildkitd.toml to an unused port, e.g., address = ["tcp://127.0.0.1:8081"].
    • Why it works: The gRPC server needs a unique, available network port to listen for client connections.
  3. Backend Runtime Not Found or Inaccessible (Rootless Docker/Podman):

    • Diagnosis: Build errors indicating the container runtime (Docker, Podman) couldn’t be contacted or failed to create containers. Logs might mention docker.sock or podman.sock not found or permission denied.
    • Fix:
      • Ensure your rootless Docker or Podman daemon is running.
      • Verify the socket path in buildkitd.toml ([rootless] docker-socket or podman-endpoint) is correct for your user’s session.
      • Ensure your user has permissions to access that socket file (e.g., chmod 666 /run/user/1000/docker.sock if necessary, though often group membership is better).
    • Why it works: BuildKit, when configured for rootless backends, acts as a client to the rootless container runtime. It needs to correctly locate and communicate with that runtime’s API socket.
  4. Docker CLI Not Connecting to BuildKit:

    • Diagnosis: docker build commands still behave like the old builder, or you get connection refused errors when trying to build.
    • Fix: Double-check your ~/.docker/config.json for the correct buildkit-daemon.address and ensure features.buildkit is true. Restart your Docker client or terminal session.
    • Why it works: The Docker CLI needs to be explicitly told to use BuildKit and where to find its API.
  5. Build Context Permissions:

    • Diagnosis: Build errors like file not found or permission denied for files within your build context when buildkitd is trying to read them.
    • Fix: Ensure the user running buildkitd (or the user whose systemd user service is running) has read permissions on the build context directory and its contents.
      # Example: If build context is in your home directory
      chmod -R u+rX /home/youruser/my-project/
      
    • Why it works: The build process needs access to the source files, Dockerfile, and any other assets included in the build context.

The Next Hurdle: Multi-Stage Builds and Network Access

Once you have BuildKit running successfully without root, the next common challenge is managing multi-stage builds where intermediate stages might require specific network access or filesystem operations that are more restricted for non-root users. You might encounter issues with downloading dependencies in a secure way, or with cross-stage file copying if permissions aren’t handled meticulously.

Want structured learning?

Take the full Buildkit course →