Your CI build is dragging, and you suspect cargo build is the culprit. The target/ directory, full of compiled artifacts, is growing and being rebuilt from scratch every single time. This is where optimizing the Cargo build cache comes in, and the most surprising thing about it is that the default behavior is often the least efficient for CI environments.
Let’s see what a typical Cargo build looks like in action. Imagine a simple Rust project with a few dependencies.
# First build - everything compiles
cargo build
# ... lots of output ...
# Compiling some-crate v1.2.3
# ...
# Finished dev [unoptimized + debuginfo] target(s) in 15.20s
# Second build, no code changes - Cargo is smart!
cargo build
# ... very little output ...
# Finished dev [unoptimized + debuginfo] target(s) in 0.15s
This is great for local development. But in CI, you don’t get the benefit of a persistent target/ directory between runs unless you explicitly manage it. The real problem is that CI runners are ephemeral. Each job starts with a clean slate, and that massive target/ directory, along with its compiled dependencies, has to be regenerated. This is the primary bottleneck.
The core idea is to make cargo build’s output persistent across CI runs. Cargo caches build artifacts in the target/ directory. If this directory can be restored from a previous run, subsequent builds will be lightning fast.
Here’s how to tackle it, starting with the most common and effective strategies:
1. Caching the target/ Directory Directly
This is the most straightforward approach. Most CI platforms offer caching mechanisms. You tell the CI system to save the target/ directory after a successful build and restore it before the next build.
Diagnosis:
Check your CI configuration file (e.g., .gitlab-ci.yml, .github/workflows/ci.yml, circle.yml). Look for sections related to caching. If you see a cache entry for target/, but the build times are still long, it might not be configured correctly or the cache is being invalidated.
Fix: In GitLab CI:
cache:
key: "$CI_COMMIT_REF_SLUG" # Cache per branch
paths:
- target/
In GitHub Actions:
jobs:
build:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v3
- name: Cache Cargo target
uses: actions/cache@v3
with:
path: target
key: ${{ runner.os }}-cargo-${{ hashFiles('**/Cargo.lock') }}
restore-keys: |
${{ runner.os }}-cargo-
Why it works:
By preserving the target/ directory, Cargo finds pre-compiled dependencies and your own crate’s intermediate build artifacts, skipping recompilation for unchanged code. The key ensures that the cache is specific to the branch and, crucially, the hashFiles('**/Cargo.lock') in GitHub Actions ties the cache to the exact dependency versions, preventing stale caches when dependencies change.
2. Caching Specific Cargo Directories (More Granular)
While caching target/ is common, Cargo also caches downloaded dependencies in ~/.cargo/registry/ and ~/.cargo/git/. Caching these can offer additional speedups, especially if you have many dependencies that don’t change often.
Diagnosis:
If caching target/ alone isn’t enough, or if you see cargo spending a lot of time downloading or updating registries, this is the next step.
Fix:
In GitLab CI, add these to your cache:paths:
cache:
key: "$CI_COMMIT_REF_SLUG"
paths:
- target/
- .cargo/registry/
- .cargo/git/
In GitHub Actions, you can add these to the path in the cache action:
- name: Cache Cargo registry and git
uses: actions/cache@v3
with:
path: |
~/.cargo/registry
~/.cargo/git
target
key: ${{ runner.os }}-cargo-${{ hashFiles('**/Cargo.lock') }}
restore-keys: |
${{ runner.os }}-cargo-
Why it works:
This caches the raw source code of your dependencies (in registry/) and any crates fetched from Git repositories (in git/). When cargo build needs a dependency, it first checks target/. If it’s not there or incomplete, it looks in the local cache (~/.cargo/registry and ~/.cargo/git) to find the source, then compiles it into target/. Caching these significantly reduces download times for dependencies.
3. Using sccache for Distributed Caching and Remote Storage
sccache is a compiler cache that works with Rust (and other languages) and can be configured to use distributed caching or remote storage like S3. This is more advanced but can be very effective, especially for large projects or when multiple CI runners need to share a cache.
Diagnosis:
If you have a very large project with long compile times even after basic caching, or if your CI involves multiple parallel jobs that could benefit from a shared cache, sccache is worth investigating.
Fix:
- Install
sccache:cargo install sccache - Configure
sccachein CI:- Local Cache (similar to default Cargo cache but potentially faster):
# In your CI script before cargo build export RUSTC_WRAPPER=sccache sccache --start-server cargo build sccache --stop-server - Remote Cache (e.g., S3):
You’ll need to configure
sccacheto point to an S3 bucket. This involves setting environment variables for AWS credentials and the S3 bucket configuration.
The CI job would then need to upload/download the# Example for S3 backend export RUSTC_WRAPPER=sccache export SCCACHE_BUCKET="your-s3-bucket-name" export SCCACHE_REGION="your-s3-region" # Ensure AWS credentials are set via environment variables or IAM roles sccache --start-server cargo build sccache --stop-serversccachecache directory (~/.cache/sccacheby default) if not using direct S3 integration. Some CI platforms have specific integrations for remote object storage.
- Local Cache (similar to default Cargo cache but potentially faster):
Why it works:
sccache intercepts compiler calls. When cargo build invokes rustc, sccache checks if the output for the given inputs (source code, compiler flags, etc.) is already in its cache. If so, it returns the cached object file. If not, it invokes rustc, caches the result, and returns it. Using a remote backend like S3 allows multiple CI runners to share the same cache, drastically reducing build times if one runner has already compiled a piece of code.
4. Selective Caching with cargo-cache (Experimental/Advanced)
Tools like cargo-cache aim to provide more intelligent caching by analyzing build dependencies and only caching what’s necessary. This is often more complex to set up and might be overkill for many projects.
Diagnosis: If you’re hitting very specific cache invalidation issues or want finer-grained control, this might be an option, but it’s usually not the first or second step.
Fix:
This typically involves installing cargo-cache and integrating its commands into your CI pipeline, often replacing or augmenting standard cargo build commands with cargo cache build and managing its cache directory. The exact commands and configuration depend on the tool.
Why it works: These tools often build a dependency graph of build artifacts. When a change occurs, they can precisely invalidate only the affected downstream artifacts, rather than invalidating the entire cache, leading to more targeted recompilation.
5. Using cargo-chef for Dependency-Only Caching
cargo-chef is a tool designed to optimize dependency fetching and compilation separately from your own crate’s code. It can significantly speed up CI builds by caching only the dependency compilation steps.
Diagnosis:
If your project has many dependencies that rarely change, but your own code changes frequently, cargo-chef can be beneficial. It helps isolate the cost of dependency compilation.
Fix:
-
Install
cargo-chef:cargo install cargo-chef -
Configure CI: The typical workflow involves:
- A "dependencies" stage that runs
cargo chef prepare --recipe your-project/Cargo.toml --target <target-triple>. This generates arecipes.jsonfile. - A "build" stage that runs
cargo chef recipe --recipe recipes.json --target <target-triple> --build-target <your-crate-name>. This will compile only your dependencies. - A final "test" or "build-app" stage that runs
cargo build --release(or similar) which will now find the dependencies pre-compiled.
You would cache the output of
cargo chef prepareand the compiled dependencies generated bycargo chef recipe. - A "dependencies" stage that runs
Why it works:
cargo-chef pre-compiles only your dependencies into a separate directory. Your main CI job then only needs to compile your application code, as all dependencies are already ready. This is particularly effective because dependency compilation is often a large portion of the total build time, and these dependencies change much less frequently than your application code.
The next error you’ll hit after optimizing Cargo build cache is likely related to test execution times, or perhaps a specific dependency with a problematic build script that still takes a long time to compile.