BuildKit can actually make your Docker builds slower if you don’t understand its fundamental performance levers.

Let’s see BuildKit in action. Imagine this simple Dockerfile:

FROM alpine:latest
RUN echo "Hello, world!" > /hello.txt
COPY app.py /app/app.py
RUN pip install -r requirements.txt
CMD ["python", "/app/app.py"]

When BuildKit runs this, it doesn’t just execute these commands sequentially. It analyzes the dependencies between them. If requirements.txt hasn’t changed, it won’t re-run pip install. If app.py hasn’t changed, it won’t copy it again. This is called caching, and it’s BuildKit’s superpower.

The core problem BuildKit solves is the inefficiency of traditional Docker builds. Before BuildKit, every RUN command created a new layer, and if anything in the Dockerfile changed before that command, the entire build from that point down would be re-executed, even if the command’s inputs (like files or dependencies) hadn’t changed. BuildKit introduces a directed acyclic graph (DAG) for build steps, allowing for parallel execution and smarter caching.

Here are 10 tips to make your BuildKit builds faster:

  1. Enable BuildKit: This sounds obvious, but many environments still run with the legacy builder. Ensure DOCKER_BUILDKIT=1 is set in your environment or via DOCKER_OPTS in daemon.json.

    • Diagnosis: Run docker build --progress=plain . and observe the output. If you don’t see the [+] syntax or detailed build steps, you’re likely not using BuildKit.
    • Fix: For Docker Desktop, it’s usually enabled by default. For server setups, edit /etc/docker/daemon.json and add:
      {
        "features": {
          "buildkit": true
        }
      }
      
      Then restart the Docker daemon: sudo systemctl restart docker.
    • Why it works: BuildKit’s advanced caching and parallel execution are only available when the builder is active.
  2. Leverage Multi-Stage Builds: Separate your build environment from your runtime environment. This drastically reduces the final image size and build time by not carrying over build tools and intermediate artifacts.

    • Diagnosis: Look for large base images in your Dockerfile that include development dependencies (like compilers, SDKs) that aren’t needed at runtime.
    • Fix:
      # Builder stage
      FROM golang:1.20 AS builder
      WORKDIR /app
      COPY . .
      RUN go build -o myapp
      
      # Runtime stage
      FROM alpine:latest
      COPY --from=builder /app/myapp /myapp
      CMD ["/myapp"]
      
    • Why it works: The final image only contains what’s explicitly copied from the builder stage, discarding all build-time dependencies and intermediate layers.
  3. Order Your Dockerfile Wisely: Place instructions that change less frequently (e.g., installing base packages) earlier in the Dockerfile, and instructions that change often (e.g., copying application code) later.

    • Diagnosis: Observe build times after making small code changes. If the build takes a long time and rebuilds many steps, your order might be suboptimal.
    • Fix:
      FROM ubuntu:latest
      # Base dependencies, change infrequently
      RUN apt-get update && apt-get install -y --no-install-recommends \
          python3 \
          python3-pip \
          # ... other packages
          && rm -rf /var/lib/apt/lists/*
      
      # Application code, changes frequently
      WORKDIR /app
      COPY requirements.txt .
      RUN pip install --no-cache-dir -r requirements.txt
      COPY . .
      CMD ["python", "app.py"]
      
    • Why it works: BuildKit caches layers. If a layer hasn’t changed, it’s reused. Placing stable instructions first ensures those layers are cached and reused across builds, even if later instructions change.
  4. Use Specific Image Tags: Avoid latest. Pin your base images to specific versions (e.g., ubuntu:22.04, python:3.10-slim).

    • Diagnosis: Builds are intermittently slow or break unexpectedly due to upstream base image changes.
    • Fix: Change FROM ubuntu:latest to FROM ubuntu:22.04.
    • Why it works: Using latest means BuildKit might pull a new base image even if your Dockerfile hasn’t changed, invalidating caches for subsequent layers. Specific tags guarantee consistent inputs.
  5. Optimize RUN Commands: Combine multiple RUN commands where logical, especially for package installations, to reduce the number of layers and improve cache hit rates.

    • Diagnosis: You have many small RUN commands in a row, like RUN apt-get update, RUN apt-get install foo, RUN apt-get install bar.
    • Fix:
      # Before
      # RUN apt-get update
      # RUN apt-get install -y foo
      # RUN apt-get install -y bar
      
      # After
      RUN apt-get update && apt-get install -y --no-install-recommends \
          foo \
          bar \
          && rm -rf /var/lib/apt/lists/*
      
    • Why it works: Each RUN command creates a new layer. Combining them reduces layer count. More importantly, if any part of the combined command changes (e.g., a package list), the entire layer is invalidated. Grouping related, stable installations together minimizes this.
  6. Use COPY Instead of ADD for Local Files: ADD has special behavior for remote URLs and tarball extraction, which can be less predictable and harder to cache effectively. COPY is simpler and more direct.

    • Diagnosis: Builds are slow or unpredictable when ADD is used, especially with compressed files.
    • Fix: Replace ADD archive.tar.gz /destination/ with COPY archive.tar.gz . followed by RUN tar -xzf archive.tar.gz -C /destination/.
    • Why it works: COPY has a more straightforward caching mechanism based on file checksums. ADD’s automatic extraction and URL fetching can lead to unexpected cache invalidations.
  7. Clean Up Build Artifacts: Remove temporary files, package manager caches, and unneeded build tools within the same RUN instruction that created them.

    • Diagnosis: Images are larger than necessary, or build steps that should be cached are being re-run because intermediate files are left behind.
    • Fix:
      # Before
      # RUN apt-get install -y some-package
      # RUN rm -rf /var/lib/apt/lists/*
      
      # After
      RUN apt-get update && apt-get install -y --no-install-recommends some-package \
          && rm -rf /var/lib/apt/lists/*
      
    • Why it works: If you install a package in one RUN and clean up in a separate RUN, the cleanup layer doesn’t affect the caching of the installation layer. However, if the installation layer changes, the cleanup layer is also re-executed, potentially re-downloading and then removing things unnecessarily. Combining them ensures that if the installation layer is cached, the cleanup is also implicitly done for that cached state.
  8. Optimize COPY Instructions: Copy only what you need. Avoid COPY . . if only a few files are required for a specific build step.

    • Diagnosis: A small change to one file in your project invalidates a COPY instruction that copies your entire project, causing a lengthy rebuild of subsequent steps.
    • Fix:
      # Instead of COPY . .
      COPY requirements.txt .
      RUN pip install --no-cache-dir -r requirements.txt
      COPY app.py .
      
    • Why it works: BuildKit caches based on the content of what is copied. Copying only the necessary files means that if unrelated files change, the COPY instruction’s cache is not invalidated, preventing unnecessary rebuilds.
  9. Use BuildKit’s Cache Mounts (--mount=type=cache): For package managers or dependency directories that are expensive to rebuild but whose contents can be shared across builds, use cache mounts.

    • Diagnosis: Repeatedly installing dependencies for languages like Node.js (npm install, yarn install) or Python (pip install) takes a long time on every build, even though the dependencies themselves rarely change.
    • Fix:
      docker build \
        --mount=type=cache,target=/root/.cache/pip \
        .
      
      In your Dockerfile:
      FROM python:3.10-slim
      WORKDIR /app
      COPY requirements.txt .
      RUN pip install --no-cache-dir -r requirements.txt
      # ... rest of your Dockerfile
      
    • Why it works: This tells BuildKit to persist the contents of /root/.cache/pip (or other specified targets) between builds, dramatically speeding up dependency installation by avoiding re-downloading and re-compiling.
  10. Parallelize Independent Build Steps: If you have multiple independent build tasks that don’t depend on each other, BuildKit can run them concurrently.

    • Diagnosis: Your Dockerfile has sections that could logically run at the same time but are executed sequentially.
    • Fix: This is more of a structural Dockerfile approach. If you have multiple services to build, consider separate Dockerfiles and a meta-build system, or structure your Dockerfile to have distinct, parallelizable stages. For example, building frontend assets and backend code can often happen in parallel.
      # Stage 1: Build backend
      FROM golang:1.20 AS backend-builder
      # ... build backend ...
      
      # Stage 2: Build frontend
      FROM node:18 AS frontend-builder
      # ... build frontend assets ...
      
      # Final stage combining results
      FROM alpine:latest
      COPY --from=backend-builder /app/backend /backend
      COPY --from=frontend-builder /app/dist /static
      CMD ["/backend"]
      
    • Why it works: BuildKit’s scheduler can identify independent execution graphs within a build and run them concurrently on available CPU resources, reducing overall wall-clock build time.

The next common problem you’ll encounter is cache invalidation due to subtle changes in base images or build arguments.

Want structured learning?

Take the full Buildkit course →