A plea not to use uv-build
67 points by lim
67 points by lim
(I am one of uv’s maintainers.)
The nice thing about Python packaging is that it’s standardized: if uv’s build backend doesn’t fit into the hole you want, then you can use any number of other backends that do an admirable job of conforming to the packaging standards.
With that said, I don’t understand why Gentoo needs to rebuild every single release of uv-build. It doesn’t change as much as uv itself, and it’s intentionally compatible across uv versions. Why not set a policy of only updating once weekly, or even less frequently, with exceptions for important breaking changes or bug fixes?
(For context: the only reason uv-build releases as often as it does is because it’s easier to keep its internals in lockstep with the rest of uv. But you can do a normal version bound in your Python build backend metadata and uv will happily use whatever version of uv-build is already present.)
With that said, I don’t understand why Gentoo needs to rebuild every single release of uv-build.
Good question - I'm not sure what Gentoo's policy is about fast-moving dependencies like this. It had also made me wonder if uv-build was bundled with uv (which would make reducing its update frequency less of an option), but I see they do have a separate package for it.
Given there is a separate uv-build release per uv build, it may just not be clear to them how frequently they need to update it. Official guidance for external packagers telling them "it's okay to only update this every N days" seems like it could help out in this situation.
Yeah, uv comes with uv-build baked in, so the majority of builds shouldn’t even require a separate build backend fetch (assuming they’re using a semver-compatible version of uv-build).
That’s a good point about documentation! I’ll see what we currently have there; I believe we already document the semver range recommendation but we might also be able to expand it to downstream distribution advice. I suppose the tricky thing there will be finding “one size fits most” language since AFAICT other from-source distributions don’t have exactly the same problem.
Personally, I'm against integrating anything backed by corporate interest as a central dependency for an ecosystem. It should be self-supporting, as the interests of the company can and will change at any time, rather than being organised specifically to support the ecosystem as a whole.
Doesn't that massively limit your options for software dependencies though?
There are open source projects that have firm, non-single-corporate governance - Python, Django, PostgreSQL, Node.js are good examples of those. But that's a pretty thin group.
It does. I also recognise the reality that many projects are primarily corporate-backed, and that there's little I can do about that. I'm stating a preference, not a dietary restriction.
Its not limiting when you experience a rug pull, its a liability. Limitations can be empowering and spark creative solutions. I’m all for it
I survived a corporate rug-pull once - when Mapzen shut down I was building a system at work that depended on their WhosOnFirst polygon dataset. That dataset was released under an open source license, so I was able to replace the Mapzen API with a MySQL polygon thing I hacked together.
WhosOnFirst has gone on to have a healthy post-Mapzen life, thankfully.
It seems bad to advocate that we shouldn’t have nice thing across a variety of mainsteam and not-so-mainstream systems in order to support legacy CPUs that are now of retrocomputing hobby interest or that 32-bit targets should remain self-hosting when pretty much all software development takes place of 64-bit platforms.
One thing not mentioned is the config APIs for those other build backends can be lacking/frustrating depending on what you're doing. I remember a while back, I had the worst time trying to get hatchling to include files in the package that were generated, but gitignore'd. Otherwise a solid post, I didn't really think about uv_build from a distro package maintainer's POV. I'm not really sold that Python packages should be in those repos, but that's their prerogative.
You can run Python pretty much anywhere where you can get a working C compiler. Some minor features may not work, but the core is reasonably portable.
I wonder how long that will hold, with Python 3.16 planning to start using Rust. It'll probably remain optional for a good few versions, but the writing is on the wall.
Gentoo is well aware, they've already expressed worry about this on the pre-PEP thread (https://discuss.python.org/t/pre-pep-rust-for-cpython/104906/70).
It’s a backend and the whole point of PEP 517 is that you can use any backend with any tool.
Then surely Gentoo can apply patches pointing to another build backend?
That refers to the fact that a developer running the uv build frontend could be picking any build-backend and it would work, regardless of if it's flit-core or uv-build or whichever. But the backends themselves can have different implicit behaviour (and different configurations), so someone building the project later usually can't just one-line patch the build-backend to another one that they would prefer.
Ah, I see. Thanks. Well, I guess I don't think this is really anybody else's problem but Gentoo's.
It's a problem for Gentoo's users, and it is also a (perhaps lesser) problem for any Linux distribution that offers support for architectures that don't have Rust or don't have a Rust implementation capable of building uv_build. (As is described in the blog post.)
I don't like the author's idea that build performance doesn't matter because it only has a positive effect on developers, and affects negatively Gentoo users
I don't read it as "build performance doesn't matter". Rather, as "the additional performance provided by uv is not worth the other costs it comes with". And to be honest, the pure-Python build backends are already fast enough without needing uv/a rewrite in Rust.
For example, I maintain (among other things) this package, a Python interface to the Akismet spam-filtering service. I'm working on the Python 3.15 update right now. And I use PDM as my packaging interface, including its build backend which is pure Python. Here's pdm build running on the current tree, on my laptop:
$ time pdm build
Building sdist...
Built sdist at ../dist/akismet-26.10.0a0.tar.gz
Building wheel from sdist...
Built wheel at ../dist/akismet-26.10.0a0-py3-none-any.whl
pdm build 0.59s user 0.15s system 97% cpu 0.755 total
Running a few more times just to avoid picking a fluke outlier, I consistently get times between 0.75 and 0.79 seconds. If it were 75 seconds, or even 7.5 seconds, I'd see an argument for switching to something faster. But this is a task I don't need to do very often, and it's already under a second. So why would I need to bring in uv and a Rust rewrite of the package build step?
It would be nice if there was a pure python fallback for uv-build. There are definitely some situations where it's not possible to use the uv build backend today (micropip for instance).
rustix::fs and its dependencies skipping SPARC, for no apparent reason
The reason may be that testing it requires visiting a museum.
If the biggest problem is that it doesn't run on SPARC... I mean, that sucks for the hobbyists running Python on Gentoo on their 20 year old Sun server, but I don't think it's fair to expect open source devs to support those platforms either.
I'm curious, what are some examples of architectures that are supported by Linux but "not at all" by Rust? DEC Alpha?
I have encountered a similar issue with termux sometimes since the wheels are not built for android (usually), though there is an extra pypi index you can add for the same.
What would be really nice is to start shipping webassembly as a standard pre built wheel architecture and then the host can use a runtime for the same. This obviously doesn't address the Gentoo we want everything as source problem, but addresses architecture inconveniences and not needing to use an unsanboxed binary.
What would be really nice is to start shipping webassembly as a standard pre built wheel architecture and then the host can use a runtime for the same.
PEP 783 formalized this, and support in PyPI landed in April of this year. There are already packages publishing WASM wheels.
That's nice, a cool slower than native but much better compatible and almost run everywhere whl.