When I started building BoringCache, most of the products I saw for slow CI were runner products: more cores, faster disks or storage close to the machine. The pitch was easy to understand. Give the same build more hardware and it finishes sooner. That works, and it's easy to sell, because you can compare runner sizes before changing anything about the build.
I didn't think larger runners were a bad product, but I thought they were often the wrong first purchase. The slow builds I watched were downloading the same dependencies, compiling the same source and preparing the same Docker cache on fresh machines. More compute made that repeated work finish sooner, but it didn't remove it. I thought a shared cache was usually the better place to start: restore the work a previous machine had done, then pay the runner to do only what changed.
What four times the cores bought
I measured the runner-size effect while testing the BuildKit state backend I later deleted. The test used PostHog's exact upstream Dockerfile on 4, 8 and 16-core AMD64 runners:
| Runner | Build | Total job | State overhead |
|---|---|---|---|
| AMD64 · 4 cores | 816s | 848s | 109.9s |
| AMD64 · 8 cores | 530s | 574s | 64.2s |
| AMD64 · 16 cores | 339s | 378s | 35.6s |
Four times the cores cut the build from 816 to 339 seconds, about 2.4 times faster, not four. The larger machines also cut the cache-state overhead, but state processing still took between 35.6 and 109.9 seconds. This was one Dockerfile, so it only shows what extra hardware did for this workload: each step got shorter, but every step still ran. The BuildKit backend we deleted after it worked has the complete experiment.
Why I chose object storage
I chose object storage because it's the cheaper durable storage that many infrastructure providers offer, and it keeps the cache separate from the compute. A team can change runner fleets, use another CI provider or restore the same work on a developer machine without moving a disk or keeping one builder alive. BoringCache doesn't need to sell you the machine that runs the build to make completed work available to it.
I came to this from Rails and full-stack web development. I already understood HTTP, object storage and the application around the cache, but I hadn't built a systems client that moved tens of gigabytes through a CI runner. Starting with one CLI gave me a problem with clear edges: make the build tool's reusable state cheap enough to save, fast enough to restore and safe to share. I could keep improving that client without asking anyone to move their build onto compute BoringCache controlled.
The trade-off is network latency and throughput. Object storage won't behave like a local NVMe disk, so the client on the runner has to make fewer requests, avoid repeating local work, transfer several objects at once and verify the result. That put most of the engineering into one CLI:
- Content-defined chunks let a changed archive find matching boundaries again, so one insertion doesn't make every later chunk look new.
- The client reuses a stored compressed chunk without recompressing it or writing another temporary copy when the previous verified descriptor still points to available content.
- It can skip reading an unchanged directory when it can verify the previous snapshot and prove that the selected directory didn't change.
- Restore downloads and verifies chunks concurrently, changing the download width when measured goodput or transport failures show that the current width is too low or too high.
- The Docker backend prepares cache bodies during the solve when CPU and I/O are available, which reduces the cache work left after the Dockerfile steps finish.
None of this gives object storage local-disk latency. It reduces the data and repeated work on the storage path, then uses the network capacity the runner has. The build tool still decides what's reusable: Cargo checks its artifacts, BuildKit owns its cache graph, and BoringCache moves the selected data between machines.
Forty gigabytes on a stock runner
The Zed benchmark is my long-running test of a large target directory. Every rolling build restores Zed's Cargo target, between 15 and 43 GB uncompressed, onto a fresh runner before Cargo starts. On GitHub's 4 vCPU runners the restore took a median of 87 s, and the restores of more than 40 GB took about two to four minutes, depending on which CPU model the runner got. On a Depot 4 vCPU runner, where the CPU is consistent, the same restores took a median of 47 s over 66 builds, and a 43 GB target restored in 49 s.
Those are restore times only, taken from the job logs. The complete Zed build times, with the compared caches, are in Seven months of benchmarking my CI build cache on real projects and on the benchmark page.
Where a bigger runner still wins
A larger runner is the direct answer for a compute-bound build with little reusable work. It also helps after a cache hit when substantial compilation remains. Cache has its own fixed costs, so a small directory that can be recreated in a few seconds may be slower to download and publish than to rebuild. I found that with a small npm cache, and kept such mounts off by default. The Docker cache-mount measurements show why the complete build matters more than whether the cache technically produced a hit. I use cache when restore removes more work than it adds; I use more compute when the remaining work can only run faster. The same build can need both.
Fast storage beside the runner
Another approach puts fast storage beside the runner. It's fast because the storage and network are built for that machine. My objection is about cost, not the technology: the vendor pays for that speed on every stored gigabyte and has to recover it from the customer, which makes a 40 GB target harder to keep on a small budget. For the large targets in these measurements, sustained throughput decided most of the transfer time, and shaving a little off the time to first byte would only affect a small part of the restore. I'd rather use cheaper object storage and put the engineering into the client, so more teams can keep large reusable state without also buying a particular runner.
Reuse before more compute
It's the same idea behind why I built BoringCache: reuse completed build work before paying a larger machine to repeat it.
Give it a try
Try BoringCache on the build step you spend the most time waiting for. Run boringcache onboard, use the quick start to configure the repository, and compare the complete job before and after. If you try it, tell me which work the cache removed and where restore still costs too much.