Debian must ship reproducible packages
104 points by fanf
104 points by fanf
This is a good move for security. It might be annoying to transition, but the end result will be higher assurance for Debian Linux users the world over.
Can someone please explain the main benefit to a project like Debian? Is it so that we can all have proof that the binaries have not been backdoored? I.e., it reduces the trust required on the maintainers, and reduces the risk of malicious maintainers?
I'm not sceptical, I'm just not 100% sure I understand why Debian is spending so much time on this, I assume making builds reproducible is quite tricky, and fiddly?
Is it so that we can all have proof that the binaries have not been backdoored?
or tampered with or modified in general. this provides benefits both for security and for project development, as a package that can be built reproducibly (and-ish hermetically) is somewhat less likely to break weirdly in another developer's environment.
the idea is to be able to provide an artifact of proof that the source code used to build the output package is indeed that specific source code, and isn't a second secret set. the reproducible-builds org has a bit of a 'why' which puts it better than i can.
I assume making builds reproducible is quite tricky, and fiddly?
extremely. for instance timestamps, even in compilation pipelines cause variance, so does metadata in created archives. all of this needs to be stamped out, and in some cases needs active patching of upstream projects (not the package itself, but perhaps cmake, or go's compiler). in many cases this simply wasn't ever on the radar before, so work has to go in across the entire build stack for a lot of things. this work has been ongoing for a long time now though, so quite a bit of that end of the plumbing is either done or deeply in progress.
I also think this shouldn't be the top priority for Debian, but Debian doesn't work that way. People work on what they want to work on, and personal priorities rarely match what would be rational for project priority.
Is Debian's idea of reproducibility the same as that of, say, NixOS?
No. NixOS is not reproducible is the standard reference.
Did you answer my question or a different one?
Your question. The linked post explains why NixOS is not reproducible in Debian sense.
I understood that NixOS does hermetic, (almost) deterministic builds. And it's their way of going towards reproducible builds.
My question though was about the goals, whether both are trying to achieve the same idea of "reproducible". NixOS, in my view, implicitly has a goal of finding a model of builds/packages/services that can make reproducibility (together with other good things) easier. Not just retrofit it on top of something from 1995. Not only make packages reproducible, but also make a language/model where managing them is cheaper, that matches the nature of these things more closely.
Back in 2017 when I did Debian packaging at my job, working with it was an absolute pain. It's just a collection of shell/perl scripts for extracting tarballs into your rootfs. It's made usable by putting in a tremendous amount of man-hours, testing, limiting packages to a hand-picked subset that's been tested together. Each new thing that's missing is retrofitted on top of existing shell scripts. No doubt that it works when a user does apt-get, but maintaining a package is a lot of pain.
I believe one can retrofit more on top. For reproducibility, you can wrap existing shell scripts in the right amount of more shell scripts and containers to make it more deterministic. Then see what's missing and add more if-then-else clauses on it to make it work. You vercome the issue with system-level tools, with no improvement on language- or model-level.
My question though was about the goals, whether both are trying to achieve the same idea of "reproducible".
The issue is that "reproducible" in this context means two things. This is what my blog post attempts to hilight and how "reproducible" in the nixos sense is different to "reproducible" in the Debian and Reproducible Build's project sense.
So the answer to your question is "no".
NixOS wants reproduction of system behaviour.
Debian wants bit-for-bit identical packages.
My blog post goes through these nuances.
all of the distros tracking reproducible builds as a project generally fall along the same targets. reproducible-builds.org tracks projects that are actively working on it in their packaging pipelines. nixos is working on reproducible builds (and the minimal iso is 100% build- and runtime-reproducible if i recall right), but reproducibility as applies to nixos in the colloquial sense isn't the "reproducible builds" being discussed here (check the foxboron post linked in a sibling to this by ~sanxiyn). that's more environment reproducibility, or "deterministic builds", which is still valuable, but it isn't this.
For the moment this sounds like a ratchet, I assume existing stuff that's never previously built reproducibly is still allowed?
If the package doesn't build reproducibly it won't be part of the next Debian release. This means "never built reproducibly" packages can stay in Debian unstable, but keeping packages in Debian unstable which never reach stable is frowned upon (I don't think there is a mechanical rule though).