Debian’s git transition
65 points by fanf
65 points by fanf
This is such great news to hear. Last year I spent a lonnnnng time trying to get Fennel packaged for Debian, and the difference between my Debian experience as a user (everything works perfectly, 100% of the time) and my Debian experience as a packager (documentation describes five different ways to do a common task, assumes you're a fervent autotools fan, and leaves out important details) was like night and day. The insistence that everything be centered around tarballs was particularly bizarre; if your upstream didn't publish tarballs no one could really agree on the right way to proceed. Note that the package I was trying to publish was nearly trivial; nothing architecture-specific, used a fairly standard Makefile, and only had one runtime dependency. It still took two years!
Many of Debian’s processes assume everyone is an insider. It’s okay that there are Debian insiders and that people feel part of something that they worked hard to become involved with. But lack of perspective can lead to software which fails to uphold our values.
It is so refreshing to hear someone else say this; someone who will actually be listened to and has the ability to change things. I am convinced that in the state I encountered things last year, it would be impossible for someone to learn on their own how to produce a compliant Debian package without a great deal of help from folks like the #debian-mentors channel.
Git forges like Gitlab can be very convenient. But Gitlab is not sufficiently secure, and too full of bugs, to be the principal and only archive of all our source code.
It's also really encouraging to hear a frank discussion of the shortcomings of Gitlab. I understand that it was probably the best choice available at the time, but these days Gitlab feels extremely clunky compared to the other options. I'm glad they're focusing on moving away from tarballs as the first priority, but I hope the future will contain a non-gitlab option.
This is very exciting to me even as a long time Debian developer. I understand the old school pre-git way of managing Debian packages, but I’ve never got on with the current git tooling for Debian packaging. I find it opinionated in the wrong ways and over complex. What this team are working on seems fantastic.
At $previous_job we found it necessary to maintain an internal fork of Debian's kernel and, as this article implies, the actual git repo the maintainers use is completely disconnected from the upstream repo, and you're meant to use tools like git-buildpackage to pull down the tarball and perform an actual build. My team was full of Debian experts so this wasn't a huge problem.
If you maintain a fork, though, someone will want to send you patches, and some kernel devs from another part of the company wanted to try out their new whizbang approach to monitoring. After spending a few minutes writing a "how to debian kernel for non-debian-dorks" guide I quickly realized writing a tool to materialize the Debian repo into something that looked like a real upstream kernel checkout was way easier.
If I'd known about dgit at the time it definitely would have helped, and in either case this seems like a great move towards making The Debian Way look a lot more like the way the outside world does development.
AFAIU, nothing is actually happening.
Ian Jackson has created tag2upload but as far as I'm aware, it's not getting that much traction. The tag2upload team has a plan, but it's up to other people to carry it out.
In that regard, it's similar to previous related attempts. Just like with any other system, until everything is transitioned to the new format, the backend needs to handle the current format. This typically means the interface is going to remain the same. For debian, this is tarballs and debian/ and terrible developer experience. Of course, teams have higher-level workflows which hide this. Of course, all these workflows are different and won't benefit much, if at all, from being built on top of tag2upload.
The current Debian situation is terrible but I don't see it change unless a switch is mandated and developers have only two options: switch or leave. The "leave" part would be unfortunate but the switch does not have to be costly: the matter is just to change the storage and to give up debian/changelog being a terrible implementation of VCS history management where commit contents are ftp-uploaded tarballs.
I think it's a bit better than that: yes, tag2upload adoption is still low, but it's also very new. Designing it is one thing, but getting it accepted into the Debian infrastructure (which was not straightforward) is where this effort differs from things that have been tried before.
I wasn't aware that it was now possible to upload through it. It's good progress.
I really hope adoption gets high (or total?), I'm just not very hopeful. My point is also that the title is misleading as you cannot call the current events a "transition", at least not from this article. There are no statistics or integration progress in other tools. Again, I hope it's happening, but the article isn't backing its claims.
Ian Jackson has created tag2upload but as far as I'm aware, it's not getting that much traction.
tag2upload has been in open beta since the 19th of June 2025 (186 days ago). In this span of time it has been used to perform 1607 uploads of all kinds (including of new releases of crucial pieces of software like openssh).
I'd say that tag2upload is getting enough traction.
I wouldn't say that less than 9 uploads per day across the Debian archive is a lot. I've counted 21 uploads just for dgit since then (it is maintained by Ian Jackson).
Of course, it's the beginning, it can get higher, but there is no indication that it will. An optimistic adoption rate is maybe 80% and the remaining 20% will probably remain forever.
I'm not against tag2upload. I'm not seeing proofs that a transition is actually happening.
Just like with any other system, until everything is transitioned to the new format, the backend needs to handle the current format. This typically means the interface is going to remain the same. For debian, this is tarballs and debian/ and terrible developer experience.
AIUI tag2upload is the new interface, running in parallel with the old tarball-based upload service, so the interface is not “going to remain the same” and has in fact already changed. The article talks a lot about bidirectional conversion between the git format and the tarball format to make this possible. This git/tarball compatibility also makes it possible for people other than the official maintainers of a package to work on the package using git, even if the package uses an older tarball/patches maintenance setup. It should be an incrementally-adoptable improvement on the current DX that doesn’t need any forced switchover.
I expect adoption will be slow (at least in these early months) because package maintainers will not be particularly keen to rework their setups. It’ll be faster if that rework can be made easier, hence the call for help in this article.
Yes, it's meant to run in parallel, at least at first. I don't remember exactly where it plugs in but if it reconstructs a typical tarball, this should make it a proxy (for now?) and that's why I'm saying the old/current stuff isn't going away. My issue with that is that there are people who are very very reluctant to change their workflows and this may mean that the tarball format cannot go away. In the Debian world, there is probably a very very long tail of such maintainers and packages.
Ian has a lot of relevant experience and he's been very active and involved. I tend to trust he's making good technical decisions. My major concern is the human side of other people sadly.
My issue with that is that there are people who are very very reluctant to change their workflows and this may mean that the tarball format cannot go away.
OK, but the new way should be able to succeed on its own merits, without abolishing the old ways. Your suggestion to force the migration would cause unnecessary disruption and alienation. By supporting both old and new in parallel the migration can go at its own relaxed pace: some can benefit from the advantages of the git-based workflow, and others can benefit from their existing setup continuing to work.
I mostly agree with this. My pessimistic take is that they may still both be running in parallel in 10 or 15 years unfortunately. This has been the case for every attempt to move away from the tarball workflow so far sadly.
This has been the case for every attempt to move away from the tarball workflow so far sadly.
AFAIK tag2upload is the first time Debian has supported uploads to its archive in a non-tarball format. It’s the biggest change since source-only uploads.
I don’t use Debian. I’m just wary of engineer initiated migrations. Typically migrations don’t make things better. They just make them different. And in the meantime bug fixes and new features languish.
What does the Debian migration achieve?
I’m just wary of engineer initiated migrations.
This maybe makes sense for corporations, but Debian is open source; I imagine most of what they do is engineer initiated. Who else would initiate the migration in this context?
What does the Debian migration achieve?
Answering this question is half of what the blog post is about, but in summary they want to move away from arcane sets of tarballs with patches applied in inconsistent ways to using git as far as possible, which seems to me like a very good thing — especially after the xz vulnerability demonstrated how bad actors can leverage this sort of confusion for nefarious use.
This git migration doesn’t affect most users of Debian: it’s about how packages are developed and modified.
One of Ian’s main goals is to make it easier for users to customise the software they are running and keep the local changes working across upgrades. Git does that much better than the old tarballs and patches systems. This is the main purpose of dgit.
Another goal, as discussed in the article, is to make Debian source packages closer to the way upstream software is developed. The GPL says that source code is “the preferred form of the work for making modifications to it” which nowadays is a git clone rather than a tarball, especially if you want to get your modifications accepted upstream.
Hopefully it’ll be a lot more convenient than tarballs and patches to use git for upstream development and debian packaging and local customisation. Especially if the downstream git repos are the same as the upstream repo but with some extra branches.