Pre-PEP: Rust for CPython
54 points by njha
54 points by njha
I'm really not looking forward to another week long round of ✨Rust discourse✨ from people rubbernecking on the proposal.
To get the first few points I expect to be raised on every thread that links to the proposal:
Sorry Esteban! 😅
Thank you for posting these points (here and elsewhere) 💜
Thank you for putting in the work on this analysis and for your work in Python! I'm sorry that you'll likely have to deal with low effort and repetitive drive-by comments in that thread and elsewhere. I hope the team will be able to have a focused discussion regardless.
After reading the proposal in full and the first few replies, it seems an exhausting position to be in to have what appear to be uninformed bystanders all chiming in with their opinions on the culture of programming instead of the topic.
Making Rust a dependency of CPython would introduce a bootstrapping problem: Rust depends on Python to bootstrap its compiler
Emma proposes solutions, but they all seem brittle. I think it's really irresponsible to make such a fundamental dependency depend on a language with such a long and difficult bootstrap path. The most recent rust buildable by mrustc is only 1.74, and currently only x86_64 linux and windows(msvc) are supported targets.
Recently someone leveraged some changes to the bootstrap machinery to try and build current rust with older stable rust with minimal patches to the source code to account for the changes to the language that rustc has used.
https://github.com/rust-lang/compiler-team/issues/925
That patchset allowed to build 1.91 with 1.81.
https://github.com/rust-lang/rust/compare/main...bjorn3:rust:bootstrap_from_older_rustc
I hope that patchset is supported in some way, precisely to allow for shorter bootstraps from mrustc.
I really hope so too, because we're talking about adding a really tough bootstrap dependency to a language required to build glibc.
I think it is very interesting that Ruby wasn‘t mentioned under examples: https://shopify.engineering/porting-yjit-ruby-compiler-to-rust
Yeah, we probably should have mentioned it. Will make a note to do so in the PEP, thank you!
I guess Thompson hack is a very niche topic, even among programming language developers working on CPython. But Chris and Steven are correct it is real, and Chris specifically mentions having multiple compilers is the best defense. (I don't think Chris is explaining why well, in fact it is pretty subtle. For details see Diverse Double-Compiling.) Emma's reply "isn't Clang also built with GCC?" shows Emma doesn't understand.
The annoying thing is, Rust has multiple compilers which actually performed DDC so it is a moot, but nobody seems aware of it. (Some) Rust people are serious about Thompson hack. The other compiler is not gccrs, but mrustc. Yes, mrustc is not production ready, but that doesn't matter for DDC. Can someone relay this to discuss.python.org?
I think the Rust team and Rust users care enough about avoiding a trusting-trust attack that I am not concerned that Rust is susceptible to one. That is to say there is enough community interest in verifying that there isn't one that I expect this is tested. I have a lot of respect for the Rust team and community. I also am aware of mrustc and actually almost mentioned it, but felt that Steve's response was sufficient. If the subject comes up again I will mention mrustc.
Okay. My point is your reply in #39 to Chris's point in #37 is incorrect or misleading. Multiple compilers counter Thompson hack because now hacking GCC is not sufficient, but both GCC and Clang need to be hacked and more importantly two hacks need to be synchronized (read DDC for details). C having multiple compilers and Rust not having do make real difference for Thompson hack. But Chris seems to be under misunderstanding that Rust does not have multiple compilers, which is wrong due to mrustc.
And that would be a strong protection, if the only type of attack were one that hits everything all at once. Unfortunately, as Ken Thompson’s hack proved, this sort of attack can be extremely narrowly targeted. And Python is a juicy target. (comment 34)
That reasoning seem flawed to me, but maybe I'm misunderstanding it ?
The benefit of widespread adoption is that you can start trusting that the community is doing the work of verifying compiler binaries by bootstrapping via multiple paths (which is already being done via mrustc, and will eventually have more paths available). Using community-verified compiler binaries seem to cover that attack vector against Python equally well for C and Rust. If you're building your own binaries and want to do your own trusting trust checks, adding Rust obviously increases the workload. But the amortized cost should be similar to gcc/clang.
In general I don't find this a useful critique, and in fact I think it borders on bad faith. Sure, Python is used in a lot of places, but so too is Rust, and Rust also tends to be used in extremely critical places because of its perceived safety and security properties. So if someone has the ability to undetectably backdoor Rust, it's already game over regardless of what Python does. That hypothetical person already has, effectively, root on a big chunk of the internet and probably could just compromise Python and a bunch of other things without even needing the core team to adopt it as an official language for coding in.
Which makes it kind of weird to suddenly argue "I was OK with Rust in that, and that, and that, and over here, and in this thing, but now that it might be in this I have concerns about a trusting-trust attack". It feels much more like a desperate attempt to find an argument against Rust because of personal dislike than a good-faith attempt to point out an actual problem.
I'm skeptical that a team which roundly rejected a Python codebase will embrace a Rust codebase. However, I'm not at all against Rust and will figure out how to integrate it into rpypkgs if it becomes an accepted PEP.
I'm skeptical that a team which roundly rejected a Python codebase will embrace a Rust codebase
I'm confused... what is this referring to? (An attempt to implement the CPython interpreter... in Python?)
About two decades ago, people started thinking about porting CPython to Python as part of a larger goal of partially evaluating Python. The result is PyPy. The main reasons that PyPy was rejected by CPython's core team are analogous to the rejection reasons given for Rust: brute conservatism, sunk costs, and a belief that C is the fastest and most portable substrate.
The world has changed within the last two decades, so adding another language to Python interpreter might be worth to reconsider
tbh these feel like...completely different things to me? PyPy is basically written in a bespoke (and sometimes bizarre) GC'd Python subset. that's very different to a low-level language that's enjoying rather widespread popularity!