I keep benchmarking BoringCache because I want to know whether I'm building the right thing. I care about how it feels to use: how you set it up, what happens when you run it, and whether you understand what it did. But I also need to know whether the build finishes sooner after I've added all that caching. A successful restore doesn't answer that.
The first commit in the PostHog benchmark repository was on March 17, 2026. I'd been running benchmarks internally before that, but this gave me a separate place to keep building a real application as its source changed. I've spent a lot of time watching those builds since. I enjoy it, though trying to explain a result can be frustrating.
The first version saved directories
BoringCache started with the repeated work I was seeing in Ruby installations. I wrote about that in Why I built BoringCache. The first version was straightforward: save an archive, restore it on another machine. I liked that model. GitHub's cache had made save and restore familiar, and I wanted reusable work to be available wherever I was building.
Then I tried doing more with Docker. I could save its local cache, move it to another machine and let Docker import it. That worked, but preparing, moving and importing the cache could take longer than the build itself. I remember looking at builds where a few minutes of useful work came with much more time spent handling the cache.
I chose PostHog for its large Docker cache graph and regular source changes. I wanted to see what happened when the next commit arrived, invalidated part of the build and left the rest potentially reusable.
Those experiments led me into BuildKit: how cache bodies are prepared, when they are exported, which bodies already exist remotely, and how much CPU and disk work the runner does before an upload even starts. I worked on preparing missing content while the build was still running and avoiding work for content already stored. I also built a whole-state backend and deleted it after the changed-build measurements showed it was slower. The backend I kept is described in Docker build done. Still exporting cache….
BoringCache stores build results remotely, but it also handles the preparation, transfer and reconstruction needed to use them. I want those operations to cost less than repeating the build work.
Cargo needed different work
I moved on to Rust because I use it for the BoringCache CLI. I pay for that CI, and I wait for those builds myself. Faster builds reduce both the wait and the CI bill. I initially thought I could put the Cargo target directory in the cache, restore it and carry on. Then I had to understand why Cargo sometimes rebuilt work that appeared to be sitting right there. Paths, environment variables, compiler settings and source freshness all mattered. Making a download faster didn't explain why the tool chose to compile something again.
A fresh checkout can contain exactly the same source bytes with newer modification times. Cargo checks those times against its recorded build state, alongside fingerprints for the compiler, features, configuration and dependencies. A source file newer than the recorded reference can make an output stale even when the restored target is intact. Build scripts can also watch files, directories and environment variables. The archive arrived successfully, but Cargo has reasons to build again. Its freshness documentation explains those checks.
That changed what I needed from the Cargo adapter. It records source and directory modification times alongside the target snapshot, then restores those times only for paths whose identities still match. Changed and new source must remain newer than the restored outputs, so Cargo can rebuild what needs rebuilding. Directory times matter too when a build script watches a directory. I don't want to make every file look old just to get a fast benchmark; an unchanged checkout should retain the freshness information from its previous build, while Cargo still decides which outputs are valid.
I wrote about the three different Rust caches as I worked through this: downloaded dependencies, Cargo's target state and compiler results. They preserve different work, so I compare configurations and other caches to understand what each keeps, what it discards, and what the next build has to repeat.
The Zed release bundle lets me compare two configurations that both restore the target directory on Depot's 8 vCPU runners. In this 30-pair sample, BoringCache finished sooner in 29 pairs. 28 pairs recorded matching CPU models: 27 EPYC 9R45 pairs and one Xeon 6975P-C pair. BoringCache finished sooner in all 28. Its median target restore was 31.8 seconds, with a range of 20.7–80.9 seconds. The total time below includes more than that restore, so I can't attribute the whole saving to transfer speed.
| Sample | Pairs | BoringCache | Depot runner cache | BoringCache faster |
|---|---|---|---|---|
| All pairs | 30 | 13.1 min | 19.6 min | 29 of 30 |
| Matching CPU model | 28 | 12.9 min | 19.4 min | 28 of 28 |
30 successful first-attempt pairs, October 9–13, 2026. Both configurations retain Cargo downloads and target state. Depot runner cache is Depot's runner archive cache, separate from the Depot Cache compiler-cache service compared below. Source: Zed release-bundle records.
AI helps me compare environments, find differences between logs and follow configuration through the code. That saves investigation time, but I still check proposed explanations against the measurements.
What the runner label leaves out
After watching enough builds, I've started to recognise patterns. A build is taking longer than I expect, and I wonder whether it missed the cache, picked up a different processor or changed a compiler option. The first guess is sometimes right, but I can also spend much longer finding out that it isn't. A 4 vCPU GitHub runner label doesn't identify one CPU model. Even two AMD runners can be different processor variants. That matters when I'm comparing caches: the source and command can match while the machines doing the remaining compilation differ. I now look at the recorded CPU model as well as the runner class.
In the 20 successful Zed compiler-cache pairs I reviewed from October 12–14, BoringCache was slower in all seven where it received an EPYC 7763 and Depot Cache received an EPYC 9V45. In the one pair with that assignment reversed, BoringCache was faster. It was also faster in four of the five pairs with matching CPU models. Both caches were running on GitHub's 4 vCPU runners; Depot Cache here is the cache service, not a Depot runner.
20 successful matched-commit pairs, October 12–14, 2026, BoringCache CLI 1.41.0. Each group contains different commits, so the bars show observed timings, not an isolated CPU ranking. The matching-model group includes three CPU models. Lower is faster.
Build times, CPU models and source runs for all 20 pairs
| Commit / run | BoringCache CPU | Depot Cache CPU | BoringCache | Depot Cache |
|---|---|---|---|---|
| 4f6d97b | EPYC 7763 | EPYC 9V45 | 39.9 min | 30.9 min |
| 679ad7c | EPYC 7763 | Xeon 8573C | 40.1 min | 40.6 min |
| 9f94d13 | EPYC 9V74 | Xeon 6973P-C | 32.4 min | 31.0 min |
| 1c9dd3e | EPYC 7763 | EPYC 9V45 | 40.0 min | 31.1 min |
| 089abd6 | EPYC 9V74 | EPYC 7763 | 20.9 min | 26.9 min |
| 129c68f | Xeon 6973P-C | EPYC 9V45 | 30.1 min | 33.9 min |
| 5f08939 | EPYC 7763 | EPYC 9V74 | 24.9 min | 25.6 min |
| 687d662 | EPYC 7763 | EPYC 9V45 | 37.8 min | 29.4 min |
| 9698422 | EPYC 7763 | EPYC 9V74 | 24.4 min | 21.0 min |
| 51a3a0e | EPYC 7763 | EPYC 7763 | 39.5 min | 40.6 min |
| 4905e67 | EPYC 7763 | EPYC 9V45 | 37.4 min | 30.0 min |
| c66312b | EPYC 7763 | EPYC 7763 | 39.2 min | 41.5 min |
| 735bc10 | EPYC 9V45 | EPYC 7763 | 19.5 min | 25.9 min |
| 8d541bc | EPYC 7763 | EPYC 9V45 | 26.1 min | 20.2 min |
| 1773013 | EPYC 9V74 | EPYC 7763 | 35.0 min | 38.0 min |
| b35ff6b | EPYC 7763 | EPYC 9V45 | 25.6 min | 19.9 min |
| 05d5e10 | EPYC 9V74 | EPYC 9V74 | 32.5 min | 35.6 min |
| 3668254 | EPYC 7763 | EPYC 7763 | 26.1 min | 27.5 min |
| 61c8c19 | EPYC 7763 | EPYC 9V45 | 24.1 min | 19.3 min |
| 7dc397b | Xeon 8573C | Xeon 8573C | 25.5 min | 23.1 min |
Same source commit within each row. Latest 20 successful pairs through step 86; step 82 failed and is excluded. Source: each commit links to its workflow run; result files are in the Zed result records.
The last pair in that sample is also why I can't explain everything by CPU assignment. Both jobs recorded an Intel Xeon Platinum 8573C and 4 vCPUs, yet BoringCache finished about 145 seconds later. The cache-hit counts were nearly identical, and BoringCache's reported hit reads were faster, with hits served locally after prefetch. The difference was in the final build interval. The logs didn't separate the remaining compilation, linking, disk waits and host contention well enough to tell me why.
| Measurement | BoringCache | Depot Cache |
|---|---|---|
| Restore, build and save | 25 min 30 s | 23 min 5 s |
| Cache hits | 1,989 | 1,988 |
| Non-cacheable calls | 264 | 264 |
| Average cache-hit read | 8 ms | 231 ms |
Hit-read latency excludes the separate prefetch work and is not an end-to-end restore measurement. Source: October 14 Zed run.
A total duration tells me there is a difference, and cache counters and build timestamps help me narrow down where it happened. Matching the CPU model is useful, but it doesn't make two jobs the same machine.
Deciding what is worth caching
There is also a temptation, when you're building a cache product, to cache everything. I have looked at Dockerfiles and wanted to preserve every directory: package downloads, installed dependencies, compiler outputs, all of it. Then I run the build and find that some of those directories weren't worth moving. A dependency installation can be quick enough that packing its files, uploading them and restoring them next time adds more work than it avoids. Linear described a similar result in its CI performance write-up: restoring node_modules took about 28 seconds, while a filtered install took roughly 7.5 seconds. That was their particular setup, but I recognised the problem from my own experiments.
Inside Docker, I have two ways to preserve work beyond the layer cache. A cache mount retains a directory that a build step uses. A native tool cache lets a compiler or build tool request reusable results through its own cache interface. That can avoid transferring a complete directory snapshot, although the tool still has local work to do. I explained the distinction in A Docker layer miss does not mean a cold build. For tools with a suitable native cache, that's often where I want to start. But a directory can also be worth retaining when it contains expensive work. The choice depends on what survives the next source change and how much the save and restore cost.
One llama.cpp CUDA build shows why I need to separate those mechanisms. The plus configuration used ccache's native remote cache and a cache mount for its small local state. It finished about ten minutes sooner in this pair, with substantial compilation still left to do. Both configurations passed their output checks and recorded the same 4 vCPU runner class and AMD EPYC 7763 model, but used different Dockerfiles and compiler setup. Restoring the mount took about two seconds, and publishing it took about 2.4 seconds; those timings exclude the native compiler-cache transfers. This pair doesn't establish how much time either caching mechanism saved.
| Configuration | Time |
|---|---|
| BoringCache base | 90 min 8 s |
| BoringCache plus, native ccache and mount | 79 min 59 s |
Source: October 14 llama.cpp run.
I don't want saving and restoring a directory to cost almost as much as recreating it. The cost includes CPU, disk, network and storage as well as elapsed time. If retaining a directory saves very little time and adds work on every run, I'd rather leave it out. If it avoids minutes of compilation, I want to understand how to keep that benefit as the source changes.
What a smaller runner can do
I still find it interesting when a smaller runner finishes close to a much larger builder. I wrote about my preference for reuse in Cache, not bigger runners. Across 10 matched PostHog commits, BoringCache's base configuration on a Depot 8 vCPU runner finished in similar time to Depot's 16 vCPU remote builder. The same BoringCache configuration on GitHub's 4 vCPU runners took much longer.
| Cache | Runner | Mean time |
|---|---|---|
| BoringCache base | Depot 8 vCPU runner | 5 min 42 s |
| Depot cache | Depot 16 vCPU remote builder | 5 min 37 s |
| BoringCache base | GitHub 4 vCPU runner | 14 min 29 s |
These are complete configurations, including different cache and builder arrangements. The comparison does not isolate core count or measure the total bill. Source: PostHog result snapshot.
The Zed target-cache comparisons use BoringCache plus on the same commits, with a different runner. Namespace's 8 vCPU runner finished sooner in all 20 pairs in this sample, and Depot's 4 vCPU runner finished sooner in all 30 of its pairs against GitHub's 4 vCPU runner. The two comparisons have different commit samples, so their averages aren't a direct Namespace-versus-Depot comparison.
| Runner | Pairs | Runner mean | GitHub 4 vCPU mean | Saved per build |
|---|---|---|---|---|
| Namespace 8 vCPU | 20 | 14.9 min | 25.7 min | 10.8 min |
| Depot 4 vCPU | 30 | 18.8 min | 31.3 min | 12.5 min |
Historical first-attempt samples: Namespace October 8–9, Depot October 11–13. Four Namespace pairs overlapping the October 8–9 upload incidents are excluded from both sides. Namespace runs span CLI 1.40.1–1.41.0; this is separate from the single-release benchmark-page sample. Its target restore had a median of 31.8 seconds, ranging from 18.5 to 47.1 seconds. Source: Zed result records.
I've found the GitHub results harder to keep consistent, partly because of the changing CPU models. These paired averages show how much the runner can matter even with the same cache; they don't measure variability or identify whether CPU, disk, memory or host contention caused the difference. I want BoringCache to make useful work available quickly on whichever runner someone chooses. A persistent builder avoids restoring state it already has locally, but a fast restore gives me another option: start a fresh machine and reuse the previous build. The less time and work that restore takes, the less reason I have to keep a particular builder running just to retain its cache.
Serving thousands of cache objects
I also tried Cachely in the Bazel gRPC benchmark and removed that configuration on October 8. In the run before removal, its warm build took 28 minutes 22 seconds and its cold build took 53 minutes 58 seconds. The warm build did reuse work: Bazel reported 3,241 remote cache hits. It also ran 548 sandboxed processes, so this wasn't a fully cached build spending all its time downloading. The other remote-cache configurations in that run finished the warm build in roughly two to ten minutes.
| Remote cache | Timed build | CPU |
|---|---|---|
| BoringCache | 2 min 15 s | EPYC 7763 |
| BuildBuddy | 3 min 20 s | Xeon 6973P-C |
| Depot Cache | 5 min 50 s | EPYC 7763 |
| NativeLink | 10 min 16 s | EPYC 7763 |
| Cachely | 28 min 22 s | EPYC 7763 |
One historical run, with different CPU assignments and cache behaviour, not a current provider ranking. Sources: gRPC benchmark run and the removal record.
I can't attribute Cachely's time to its storage backend from those logs. There was local work left to do, and I don't have measurements that separate its storage and network costs. But serving this kind of workload efficiently is a problem I have to solve too. Bazel caches action results and output objects separately, so a build can need thousands of objects rather than one archive. If each request has to wait on a remote object store, a small delay per request can become expensive. Low-cost storage needs a delivery system that suits the tool: fetch work concurrently, make reusable data available locally, and avoid unnecessary round trips.
Moving less cache
Speed is the first thing I look at, but storage, bandwidth and the runner's own work are part of getting there. A large target directory can contain mostly the same content after a small source change. I don't want to compress and upload all of it again. BoringCache uses content-defined chunking to find reusable portions even when data shifts within the archive. It can keep references to chunks already stored remotely and upload the missing ones. In the large Rust build experiments, changing the chunking approach made a substantial difference to how much data had to be uploaded after a source edit.
Avoiding an upload still leaves work if I have to read the whole target to discover that nothing changed. Selective reads let the archive save reuse verified unchanged ranges without reading those contents again. That's a save optimisation; it doesn't predict which files the next commit will need or make a fresh runner's restore disappear. Keeping the target smaller addresses a different cost. The Cargo adapter uses observed build outputs to prune older unused artifacts, while protecting active outputs and avoiding pruning when the usage information is incomplete. I want to keep expensive reusable work without carrying every old output into every new job. Saved, read and restored bytes are related, but each needs its own measurement.
Some comparisons are close, and some favour another cache. In the Nix Zed sample on Depot 4 vCPU runners, all 24 pairs recorded the same CPU model. BoringCache averaged 35.8 minutes against Cachix's 37.0, but finished sooner in only 12 of the 24 pairs. One slow Cachix build accounts for the mean advantage; without that pair, BoringCache's mean is slightly slower. I still look into those results, but I don't assume every small difference has an explanation I can establish, or that every workload has minutes left to remove. The benchmarks have led me to remove a working backend, change how I preserve Cargo state and stop caching directories that cost too much to move. They also keep giving me results I don't understand yet, and finding out can take longer than I'd like.
I've been benchmarking on Depot and Namespace, and I'm paying for Depot. My budget limits how many runner configurations I can keep testing. If you run a CI or runner service and can lend me access, I'd be happy to benchmark it. I'd like to see how the same workloads behave across more CPU models and runner configurations, with enough detail to explain the differences. You can also send me a build command, cache setup and a result you're trying to understand.
What I want to offer with BoringCache is reusable build work on low-cost storage, available quickly on the runner you choose, with a setup you don't have to keep fiddling with. You should be able to enable caching for your project and keep using the build tools you already have, without operating a cache server or working out how each tool should talk to it. I keep benchmarking because I want to know that BoringCache is saving you both build time and effort.