Skip to content
  • Rust
  • Cargo
  • sccache

Rust build cache is three caches, not one

I thought Rust build cache meant the target directory. Building the BoringCache CLI taught me why dependency state, Cargo target state, and sccache solve different parts of the build.

Gaurav Tiwari Updated 17 min read

I chose Rust for the BoringCache CLI from the beginning. This was systems-level work, the command had to run on several platforms, and I wanted something fast that I would enjoy maintaining. I had used Rust years before, but this was the first time I had used its build tooling every day. At first the CLI was small enough that a clean build took a minute or two. That was fine. Then the product grew.

I added more protocols, platform support, Docker and BuildKit integration, content addressing, compression and the rest of the things a cache client eventually needs. The clean builds became three or four minutes, sometimes more. Locally I did not always notice because my usual checkout already had a warm target/ directory. Fresh CI jobs, Git worktrees and coding-agent checkouts made the cost visible again. This is the same repeated-build problem that led me to build BoringCache in the first place. I wrote about that in Why I built BoringCache. This time I wanted to share the cached build state for BoringCache’s own CLI between machines.

Cargo home avoids downloads, Cargo target lets Cargo skip fresh units, and sccache reuses cacheable compiler results.
Dependency state, Cargo target state and compiler results remove different work from the same Rust build.

I thought the target directory was the cache

I initially thought: if I have the target/ directory, I have the Rust build cache. That is mostly how it feels in one checkout. With Cargo's default directory layout, final and intermediate build artifacts go there, along with fingerprints and build-script output. Keep the directory around and a second build can be very quick. Lose it on a fresh runner and Cargo has to reconstruct much more of the build.

Copying the target directory between machines adds transfer time. A large target directory can take long enough to copy that rebuilding starts to look reasonable. Sharing one directory between worktrees also introduces locking, so one agent can end up waiting for another. Howard John describes both problems in his worktree cache experiment. I was seeing the same problems locally: the state was useful, but it belonged to one path on one machine.

There are three caches

The mistake was treating Rust build cache as one thing. I ended up working with three useful kinds of state, and each one removes a different piece of work. These are the three layers in this setup; incremental compilation is another reuse mechanism within Cargo's build state.

Cargo home
The registry index, crate archives and Git source data. Restoring these avoids dependency downloads, but the dependencies are not compiled yet. BoringCache's default entries select registry archives, registry metadata and bare Git databases; credentials and installed tools elsewhere in Cargo home are outside those entries.
Cargo target
Cargo's build artifacts, fingerprints and build-script output. This lets Cargo decide that a complete unit is already fresh.
sccache
Individual rustc invocation results. This helps after Cargo decides that compilation is still required.

Target and sccache are complementary. Target is not a bigger version of sccache, and sccache is not a replacement for target. Target state lets Cargo skip compilation for fresh units. sccache can return cached results for some compiler invocations that Cargo still runs. If Cargo sees an exact target hit and never calls the compiler, sccache has nothing to do.

sccache requires incremental compilation to be disabled and cannot cache crates that invoke the system linker, including binaries and procedural macros. Its Rust support notes also require emitted link output, which excludes metadata-only invocations from cargo check. Build scripts can still execute, so a high compiler-cache hit rate does not mean the whole build has been reused. If you only want to try sccache with ordinary Cargo, install it on your path and put these settings in .cargo/config.toml. This configures the compiler wrapper; remote storage is a separate sccache setting.

.cargo/config.toml
[build]
rustc-wrapper = "sccache"
incremental = false

One Cargo command

I could have configured three separate cache steps in every workflow. There are good Rust cache actions, and sccache already supports remote storage. What I wanted to know was whether I could use all three through one Cargo command, locally and in CI.

Terminal
$ boringcache cargo build --release --locked

boringcache cargo restores configured dependency state and the target snapshot, starts the sccache session when it is enabled, runs Cargo with the arguments untouched, then publishes successful work from a trusted checkout. Dependency resolution, fingerprints and compilation stay with Cargo. BoringCache transfers the reusable state between runs.

The configuration belongs in the repository so the same cache choices are available locally and in CI. After installing BoringCache, Cargo and sccache, run boringcache onboard from the Cargo project and commit .boringcache.toml. Use sccache 0.16.0 or newer for the compiler-cache setup below. The relevant sections look like this, with the workspace selected during onboarding. Credentials are configured separately. Cargo still reads its dependencies and build profiles from Cargo.toml, and its own configuration from .cargo/config.toml.

.boringcache.toml
workspace = "your-org/your-project"

[profiles.cargo-deps]
entries = ["cargo-registry-cache", "cargo-registry-index", "cargo-git-db"]

[profiles.cargo]
entries = ["cargo-registry-cache", "cargo-registry-index", "cargo-git-db", "cargo-target"]

[adapters.cargo]
profiles = ["cargo"]
compiler-cache = "sccache"

[adapters.sccache]
tag = "cargo"

Choosing the layers also means deciding when to transfer the target directory. For repeated local checks, boringcache cargo --profile cargo-deps --skip-save check --locked restores dependency state and keeps the worktree's target local. The skip flag disables archive publication; compiler-cache writes still depend on credentials and trust, and only eligible compilations can hit. For a fresh runner, boringcache cargo build --release --locked uses the full profile. Choose that path when restoring target state costs less than rebuilding it. The --profile before the Cargo command selects BoringCache entries; Cargo's own --profile after its subcommand selects a build profile. The Cargo adapter guide covers authentication and the CI setup.

Moving target between machines

Preserving a target/ archive does not necessarily preserve its freshness relationship with a newly checked-out source tree. Cargo's decision is based on more than whether a file with the right name exists. The restored artifacts and the source tree have to keep the freshness relationship Cargo expects. BoringCache records exact source and directory modification times beside a target snapshot. On restore, unchanged paths get their recorded times back. A changed or newly created source stays newer than the restored artifacts, so Cargo can rebuild the affected units. Directory times matter too because build scripts can watch directories rather than individual files. The target keeps its configured tag with the existing platform and Git suffixes. Compiler and project changes do not add a hash; Cargo decides which restored outputs need rebuilding. This still depends on the inputs Cargo and build scripts track; timestamp handling cannot account for an undeclared external input.

A dirty worktree can restore a target cache, which is useful for normal local development and agent work. It cannot publish one. If a checkout has local changes, BoringCache restores a target snapshot produced from a clean checkout, preserves the local source edits, keeps changed paths newer, runs Cargo, and refuses to move the shared target tag afterwards. That prevents other checkouts from restoring a shared target snapshot produced from those local changes. BoringCache defaults CARGO_INCREMENTAL to 0 and excludes Cargo profile incremental/ directories from saves. Set CARGO_INCREMENTAL=1 with compiler-cache = "none" under [adapters.cargo] to enable incremental compilation and include those directories. Cargo's compiler-information cache, .rustc_info.json, remains excluded. Cargo's own development profile normally enables incremental compilation, so using sccache changes that local edit-and-build tradeoff. Compare complete build times for the way you work.

The local test

For this post I ran a small exact-source cohort against the BoringCache CLI. Every lane used BoringCache 1.19.2, Rust 1.97.1, the same source commit and a different fresh worktree path. Cargo dependency state was already warm in all four lanes, so the test isolated the target and compiler layers. I ran each lane three times, sequentially, on the same Apple M5 Pro. These measurements describe that earlier release. The target-tag reuse across manifest and lockfile changes and the incremental-directory setting described above require BoringCache 1.30.2 or newer.

Fresh worktrees · macOS ARM64
plain Cargo
  runs     32.89 · 40.91 · 45.35s
  median   40.91s · full compile

sccache only
  runs     35.22 · 36.47 · 38.20s
  median   36.47s · 208 Rust hits · 40 misses

target only
  runs     30.55 · 35.62 · 59.99s
  median   35.62s · 10,472 sources reused · no compile

target + sccache
  runs     32.17 · 32.85 · 41.10s
  median   32.85s · exact target · no compiler work

target materialized     1.86 GB · about 3,300 files
sccache Rust hit rate   83.87% in fresh different-path worktrees

This is a fast development machine, the plain build is under a minute, and target restore varied from about 26 to 54 seconds as the archive moved over the network. With three sequential runs per lane and overlapping ranges, the roughly three-second difference between target-only and the combined lane does not establish that adding sccache made an exact target restore faster. The cache layers removed different work. With only sccache, Cargo still ran the build and 208 of 248 cacheable Rust compilations hit remotely. Forty missed; the counts alone do not identify which compiler inputs differed. With the target restored, all 10,472 tracked source files and 2,239 directories matched the producer. Cargo did four to six seconds of checking and invoked no compilation. sccache reported no hits because there was nothing for it to serve.

My first target-only lane rebuilt after I removed RUSTC_WRAPPER=sccache. I discarded it and produced a separate no-wrapper snapshot, but that observation alone does not establish which input caused the rebuild. Cargo documents separate artifact hashing for RUSTC_WORKSPACE_WRAPPER, which is a different setting. Assigning a cause to the original rebuild would require its fingerprint diagnostics.

A changed CI build

The controlled tests explain the cache layers, but using them for daily CLI builds tests the workflow I wanted to improve. In one recorded CLI build, the shared target had moved on since the source checkout. BoringCache restored it and reported eight source files that were new or changed. Cargo decided which units needed rebuilding, and sccache served eight cacheable Rust compilations.

BoringCache CLI · changed CI build
target restore       55.6s   · 6.67 GB · 11,300 files
source freshness     10,464 unchanged · 8 changed/new
Cargo                2m17s
sccache Rust         8 hits · 0 misses
target publication    8.3s   · 6.84 GB · 11,400 files
archive reuse        1,052 / 1,114 verified chunks · 4 blobs missing

This is what I wanted. Target state restored the previous build state. Cargo still made the correctness decision for the changed checkout. The eight cacheable recompilations it requested were already available through sccache. Then the next target generation reused most of the archive chunks instead of treating the entire 6.84 GB materialised directory as a new upload. On an adjacent run with 28 changed or new sources, sccache missed eight Rust compilations and Cargo took 4m07s. On another with eight changed sources, all eight hit and Cargo took 2m17s. A cache hit rate is not a substitute for looking at what changed: these were different source changes, and the timings do not isolate the benefit of compiler hits. Source-file counts and compiler-invocation counts also measure different things. The remaining Cargo time includes work outside those cacheable invocations; these measurements do not break it down further.

Dirty worktrees

I repeated the restore with one tracked source edited in a fresh worktree. BoringCache restored the target for reading, marked that source and its parent directories newer, and printed the publication warning before the build:

Dirty worktree · restore allowed, publication refused
[boringcache] Cargo target publication disabled:
the source checkout has local changes

sources       10,471 unchanged · 1 changed/new
directories    2,236 unchanged · 3 changed/new
publication   skipped

I want an agent or a branch with local changes to benefit from shared state. I do not want those local changes to be published as the shared target snapshot. For worktrees on the same machine, BoringCache 1.32.0 adds optional local target snapshots: set enabled = true and, for example, max-size = "20GiB" under [adapters.cargo.local-target-cache]. In that release, the selected profile must include cargo-target. BoringCache can restore independent files or filesystem clones into an empty target, so each worktree keeps its own writable target without a remote transfer. Existing populated targets are kept. The local worktree setup covers capture, restore and disk limits.

Large target directories

Rust targets grow. My own debug target is now several gigabytes, and larger projects can be much bigger. Uploading the whole archive after a small rebuild would leave little time saved by caching. BoringCache writes a canonical tar stream and splits it with FastCDC content-defined boundaries. When bytes shift in one part of the target, the chunker can realign after the changed region and reuse later content. In the no-source-change publication from the local cohort, the target still materialised to 1.86 GB, but 286 of 289 verified archive chunks were reused. The save phase took 3.6 seconds and only four of 237 remote blobs were missing. Chunk-reuse counts and remote-blob counts describe different stages; neither is a measurement of uploaded bytes. The client still has to walk the filesystem and produce the stream, and a fresh worktree still has to restore the files it needs. Content-defined chunking can reuse unchanged content after a changed region, but the amount reused depends on the resulting stream.

Rust cache inside Docker

These caches also became useful inside a Docker build. A Docker layer is the outer cache. If the layer hits, Cargo does not run. If it misses, Cargo can reuse the restored target state and sccache can return cached compiler results inside the rebuilt layer. The command below assumes a configured Docker build with the Cargo target mount and compiler-cache wiring described in the adapter guide.

Terminal
$ boringcache docker --mount-cache --tool-cache sccache

The mount cache can save a selected Cargo target directory and restore it on fresh builders, while the sccache sidecar serves compiler results during the build. They stay opt-in because restoring a large mount only makes sense when it is faster than rebuilding the directory. The Docker adapter guide explains the current setup, and the Rust build cache page keeps the local and CI path together.

What I built

I started with the idea that Rust cache meant target/. I ended up putting three cache layers around Cargo's own decisions: dependency state so a new machine does not download everything again, target state so Cargo can skip complete units that are still fresh, and sccache for the compiler work that remains. The target path needed the most care. Exact source and directory times, configured platform and Git scope, dirty-read but clean-publish behaviour, and the explicit incremental setting are what make a transported target useful while preserving Cargo’s ability to detect changed source and rebuild it.

I have been using this cache setup to build BoringCache’s own CLI. The public Deno benchmark shows the compiler-cache side: in the first build of the adjacent-source warm job, sccache reported 1,333 compiler-cache hits and 58 misses, a 95.8% hit rate among those hits and misses across languages. Its separately reported Rust hit rate was 98.56%. These runs show the target and compiler layers doing different jobs. If you have a Rust project that is fast in one old checkout but cold again in CI, another worktree or a Docker builder, try boringcache cargo build. Compare dependency setup, target restore, Cargo execution and publication times. Those measurements show which work caching removed and whether transferring the state saved time overall.

Use all three caches in your next Cargo build.

Connect dependency downloads, Cargo target state, and eligible compiler results to the same workspace. Keep the Cargo command your project already uses.

  • Docker
  • BuildKit
  • CI caching

A Docker layer miss does not mean a cold build

When BuildKit has to run a step again, portable cache mounts and native tool caches can preserve cached directories and tool results for reuse inside that step.

11 min read
  • Rails
  • Deployment

How I deploy Rails to plain Linux

I build Rails with Dagger, reuse the slow work with BoringCache, then send one filesystem artifact to four Ubuntu servers.

8 min read