BuildKit: Docker's hidden gem that can build almost anything

22 points by tuananh

dvogel

This article does the same basic thing the docs do:

BuildKit gives you a content-addressable, parallelized, cached build engine for free.

Perhaps you didn't have to build those parts yourself. However they aren't free. The content addressable part isn't just a benefit. It is also a cost. It is the cost of providing this as a general solution and it is what leads to slow bits like the docker context hashing.

I still think it is good. I just dislike overzealous sales pitches.

white-star

I read this but didn't get why I'd want to use it instead of a make file and shell scripts.

mdaniel

Please don't forget to add a license to your repo

gthm

So could I use this as a kind of ccache at work? We have a number of large C++ projects. Could I write a frontend that reads cmake (or its make/ninja output perhaps) and outputs this IR format and have buildkit deal with the rest? Or am I missing something?

payne

That's a very interesting thought and think it might actually work.
mologie
We have both BuildKit and ccache at $work and they are complementary:
- BuildKit caches docker layers (including an option for remote caches) -- if any input or previous layers change (e.g. your COPY that copies all source files), the cache becomes invalid
- BuildKit can provide a shared cache mount for ccache via RUN --mount=type=cache,...
- ccache caches preprocessor outputs => compiled objects, will be used when rebuilding layers
The IR is essentially a lower-level representation of Dockerfiles, but doesn't solve the issue that caches work at layer-level, and you can't have a layer per source file (they are ordered and creating them is slow).

The same applies to other caching module/package managers, e.g. Go: BuildKit can provide the cache mount at /go/mod/pkg, changes there won't count for the layer hash and are always available from previous runs, hence Go needs to redownload and recompile packages less often.

Edit: And indeed the apk builder provided with the article does nothing but run make in a container: https://github.com/tuananh/apkbuild/blob/main/pkg/apk/build.go -- ccache is still needed here to make this fast(er). The article suggests that one can model the entire build, down to source level, in LLB, but I find that hardly practical. (Also lots of LLM tells in the article, mhm.)

southclaws

oh this is really neat I had no idea! I might use this for the plugin packaging tool i'm building for my OSS project

jorsk

FWIW I'm pretty sure you can use buildkit directly without involving the docker cli.

Halkcyon

I read the article and attached repo—why is this being flagged as spam with the vibecoding tag added? I thought it was rather clever to build a custom frontend for buildkit and not something I've run across before. Are the em-dashes and misaligned ASCII charts that big a tell that it must be LLM-written/-edited?