How Our Rust-to-Zig Rewrite is Going
117 points by rrampage
117 points by rrampage
As impressive as that improvement is, Zig's 35ms is still way ahead. Not only is it 1/100th the build time of 3.4 seconds, it's also in a different performance category—and that 35ms is on a Zig code base with ~50% more lines of code than the Rust one that got 3.4s.
On paper it's crazy fast but in reality it doesn't seem to me the build improvement is that impressive. 3.4 seconds is quite fast enough and if you're coding a compiler you're not going to be rebuilding every single minute, so that's a 3 second gain at maybe every 10 minutes or so. number come down with no extra effort.
Sure, like they said, the codebase will grow so Rust would be slower in the long run, but I think too much is being made about sub-second compilation like it's a must have. The cold build times got worse as well, those are also relevant. I'd maybe posit that simply upgrading your notebook every 2 years or so, along with their improvements to compilation speed, would make that Rust number flatline over time, so it's not a given that the Rust build would always get slower.
Pressing save and having the compile errors (or even the newly invalidated test results) visible before I can press the next key, let alone move to my next edit location, is a huge boon. I know rust has a check mode and the LSP is faster than a re-compile, but it's not the same.
I dunno, I'm of the opinion that faster-than-light feedback is not a huge concern for highly technical work like a compiler. It's the kind of thing that requires you to think before you implement them, probably sketch it out on a piece of paper, then type it out. A couple seconds feedback is fine here. Most likely the test suite will be several seconds slower too.
It's different if you're doing something highly dynamic like a Web page and sketching it out as you go along and need the visual result quickly.
Maybe in a very large codebase the rust analyzer is quite slow so that matters more, I can't say. In my projects quite often when working with Rust I'd hit "save" out of habit already knowing it would fail because there were already red squiggly lines on my editor from the LSP. So the fact that it took longer than 1 second didn't matter, it was a wasted cycle anyway, the LSP was faster.
I work on the Zig compiler (which is self-hosted) and I can confidently say that the ability to rebuild in milliseconds hugely improves my productivity.
Consider a case where you're working on a large refactor of part of your codebase. After you've made all of the changes you want to make, there are, very broadly speaking, two main tasks ahead of you. (YMMV of course, this is based on my own experience writing Zig code with my preferred workflow.)
First, you're going to fix a bunch of compile errors. Admittedly, if you use LSPs, this may be less of an issue, because you'll have fixed stuff as you go---but I don't use an LSP (just because I don't find it hugely beneficial), so I might spend a while dealing with some dumb typos, incorrect argument orders, etc. To me, the improvement in productivity here is already more than enough to justify the feature.
Second, you're inevitably going to have written some bugs, so you'll hit crashes or incorrect behavior at runtime---so your second task is to find and fix those bugs. Sometimes, you'll have a crash with a really useful stack trace, and can fix it straight away; in this case, speedy compilation is mainly just a QOL feature. But other times, you'll be diagnosing slightly trickier bugs, and maybe you want to do some print debugging, or add some assertions to test a hypothesis, or something like that. In this case, immediate rebuilds are a godsend: you can try a dozen things in the span of a minute without ever feeling bottlenecked by the tool you're using. It's hard for me to express how lovely of an experience it is for me as someone used to the poor performance of most systems language compilers (and yes, I include previous versions of the Zig compiler in that!).
(Also: having ADHD, I find that fast compilation is very effective at stopping me from sabotaging myself! Running 10-15 second builds multiple times only a few minutes apart presents a lot of great opportunities for me to idly tab into Firefox or pull out my phone, and at that point it's game over for productivity for anywhere from 5 minutes to several hours.)
For compiler errors, in my opinion, Rust (cargo check) is plenty fast enough. I edit, hit Ctrl-S, the code gets rebuilt, compiler errors pop up in the editor -- the delay is measurable but does not get in the way. It'd be the same if I would build things on a terminal. (But then, I never worked in a language with millisecond compile times, so maybe I just don't know how much better that is than the ~0.2-1s I would guess that I currently have.)
But when it comes to actually running tests and debugging -- yeah, I can totally see how amazing it would be to have to wait less there. :)
Full disclosure, I don't know Rust and haven't really used its toolchain before. My experience prior to getting into Zig is pretty much entirely with C and C++, from which I am... somewhat scarred :^) but it sounds like cargo check is pretty quick. I agree that once you're in the 1sec-ish range you're definitely reaching the point of diminishing returns on the benefits of a faster compiler. It's definitely a bigger deal to actually get a working binary this quickly than it is just to get compile errors!
I think you're downplaying the difference it makes.
I don't think you can assume that someone is recompiling every 10 minutes. The --watch -fincremental flag combination allows you to recompile on save, which means you're getting feedback very fast and very frequently.
Having done the Rust to Zig transition myself for some personal projects, and having used Rust professionally for several years, Zig's compilation speed is such a breath of fresh air.
Wow, 35ms is super fast. What is it that the compiler is doing there? I would have assumed that relinking alone would have taken longer than that.
The key thing is incremental linking and in-place binary patching (what the Roc compiler calls surgical linking iirc?). If you change the implementation of a function it just has to insert the new assembly into the existing binary (with plenty of analysis for in-lining, signature changes / dependencies, relocations if it doesn't fit...).
(Zig team member here, I've done quite a lot of work on incremental compilation.)
I need to write up a blog post on a bunch of this stuff at some point, but regarding linking, our new ELF linker (currently quite incomplete, but getting more complete by the day!) is designed to be incremental at the granularity of individual functions and global variables.
Very basically speaking, if you modify a function or add a new one, the linker will first find a sufficiently-large unused region in the .text section of the output executable file. (If there is insufficient unused space it might need to expand the .text section, which may in turn mean it needs to move some other stuff around, but this cost is amortized simply by using exponential growth on the sections.) It'll then write the new machine code into that region; possibly write in a new symbol table entry and some new relocation entries (again, we try to keep unused space around for this); and then it's done, we close the file handle!
I'd need to check to be sure, but based on when I last looked at the Zig compiler's Tracy output, I would guess that of those 35ms, only around 1ms is actually spent in the linker.
Memory-unsafe code wasn't rare for us, though (more on this later) and we ended up with about 1,200 uses of unsafe (out of our 300K lines of Rust code
That's more unsafe code than I was expecting to be honest. For example, for Inko there are only 162 unsafe { ... } expressions and 87 unsafe functions (these are exposed through the C ABI to generated code, hence the need for unsafe), though the entire LLVM backend is essentially one giant unsafe because of how Inkwell works. The total number of lines for all the Rust code is about 88 000 lines.
It's difficult to draw any conclusions from this though. For example, I generally don't flag Rust functions as unsafe unless they actually do something that is unsafe memory wise (e.g. fiddling with pointers), while others may flag a function as unsafe just as a "hint" of sorts to ensure XYZ is done first to prevent a panic.
Very nice article! Always great to get some nice comparison of what works well and doesn't work well for two languages in this particular context.
However, this part here isn't quite right:
Memory-unsafe code wasn't rare for us, though (more on this later) and we ended up with about 1,200 uses of unsafe (out of our 300K lines of Rust code; compare to about 40,000 uses of unsafe in rust's 3.5M lines,
This 40,000 number is completely bogus. This is the number of times the term unsafe occurs in the Rust standard library and compiler, including in tests and comments. Almost all of this is in the standard library, the compiler has less than 2000 occurrences of unsafe and many of those are comments and tests.
Unsafe code is extremely rare in the Rust compiler; in all my time working on it, I am fairly sure less than 1% of my PRs involved unsafe code in the compiler.
Having lots of unsafe in the standard library is entirely expected -- it establishes the basic runtime that all of Rust rests on, that kind of code is (explicitly or implicitly) unsafe in any language.
So it sounds like their compiler is around 10x smaller than rustc and has a similar amount of unsafe. I still wouldn't call that "pervasive" but you would encounter unsafe a lot then, yeah. I wonder why they need unsafe so much more than we do in rustc.
That's a totally fair distinction! Our code base also mixes compiler and stdlib code (e.g. parts of the stdlib are implemented as compiler intrinsics), so I think I was doing an apples-to-apples comparison in that metric - but it's a fair point that the stlib is separate from the compiler.
I didn't analyze the breakdown in either of our code bases of which uses of unsafe are stdlib-related or not, but I would expect a Rust version of our new compiler to have more uses of unsafe than rustc just because of some of the caching and indices-over-pointers things we're doing.
By the way, thanks for all your great contributions to Rust! I hope I didn't come across as ungrateful for all the great work I think the Rust team is doing; it was my goal to explain our decision and talk about our current progress without disparaging another project that I also think does great work. :)
That's a totally fair distinction! Our code base also mixes compiler and stdlib code (e.g. parts of the stdlib are implemented as compiler intrinsics), so I think I was doing an apples-to-apples comparison in that metric - but it's a fair point that the stlib is separate from the compiler.
Part of the stdlib for Roc is written in Rust? Interesting. I would not expect that to require a lot of unsafe though since it's mostly going to be generating code. In Rust we also have intrinsics obviously and their implementations do not require any unsafe code. (Of course, as you explain in the post, their implementations can still cause memory safety issues in the compiled program, that's not something the type system of the host language can do much about -- for most host languages, anyway. So it's "implicitly" unsafe, in the words of my previous post.)
I would expect a Rust version of our new compiler to have more uses of unsafe than rustc just because of some of the caching and indices-over-pointers things we're doing.
Elsewhere here someone also mentioned your hot reloading tech, which I assume isn't exactly safe either. Maybe that explains the bulk of the difference?
By the way, thanks for all your great contributions to Rust! I hope I didn't come across as ungrateful for all the great work I think the Rust team is doing; it was my goal to explain our decision and talk about our current progress without disparaging another project that I also think does great work. :)
No worries at all, your post was very informative and balanced. :) Thank you for the Miri shoutout :D
I fully agree that unsafe Rust is hard to get right (at least once raw pointers are involved), and it's something I am quite worried about. I am very interested in ideas for how we could do better. Some of that is ergonomics, I hope some of the ongoing work on field projections and smart pointer reborrowing will let us define more ergonomic raw pointers. Some of it is fundamental -- unless we want to entirely give up on optimizing Rust references, there will be a tax on unsafe code. (Tree Borrows is better here than Stacked Borrows IMO, but there's still a tax.) Beyond that, I am not entirely sure what we can do. Sure, pointer types indexed by their aliasing requirements are neat, we should copy that from Zig (should be possible as a library type once we have enough smart pointer ergonomics). But I don't think that's going to be common enough to make a huge dent, so I wonder if there are other things we are missing.
Part of the stdlib for Roc is written in Rust? Interesting.
Zig, actually - it was written in Zig even in the Rust version of the compiler. (We tried writing it in Rust, but we ran into some problems related to emitting the output bytes in the way we wanted. I wish we'd written down what they were, because I've been asked about them a few times over the years and I never have a really detailed answer, just that we tried Rust first, it didn't go well, we thought about doing it in C instead, and ended up trying Zig as a "better C" alternative.)
An example: we do the small string optimization, so our Str type is basically all implemented in Zig: https://github.com/roc-lang/roc/blob/main/src/builtins/str.zig
I fully agree that unsafe Rust is hard to get right (at least once raw pointers are involved), and it's something I am quite worried about. I am very interested in ideas for how we could do better.
For what it's worth, I do think some of the stuff Zig does could make sense to enable by default in #cfg(debug_assertions) in things like pointer dereferences, global alloc/dealloc, and test blocks. Concretely, in Zig if I have a UAF, double-free, or pointer dereference into undefined memory, I'll get a panic in debug builds (assuming Zig is involved in that code, of course; if I'm using FFI into a C library that just reads uninitialized memory right away, Zig can't help with that). If I allocate memory in a test and then don't deallocate it explicitly by the end of the test, the test will fail saying I've leaked memory.
I don't know the exact techniques Zig does for all this, but the general theme is "runtime bookkeeping in debug builds only, then the compiler emits extra instructions to go check against those and panic if things aren't as expected." It's caught a lot of stuff for us!
Thanks, that is good feedback. We've started doing this (debug builds check for alignment on raw pointer derefs), but nothing fancy like UAF or so. That'd be basically like a built-in address sanitizer... or more likely, making asan easier to use so it can maybe even become the default.
Pointer deref into undefined memory is sometimes entirely legal in Rust (MaybeUninit is a thing) so that's more tricky.
That's awesome to hear! "Built-in ASan and UBSan except going further with compiler integration" feels to me like a reasonable way to describe what Zig does.
What a nice level-headed assessment of their ongoing efforts. I appreciated reading some thoughts about Rust vs Zig that wasn't completely afraid to discuss shortcomings of each. I especially appreciated the honesty evident here:
After 18 months of development, hundreds of total bug reports, and hundreds of thousands of lines of code, my main takeaway from retrospecting on this table is that picking a different row would have made no appreciable difference to the project. So far our choice has gotten us the outcome we'd hoped for.
One thing that wasn't totally clear for me, was the reasoning for the following statement:
for compilers which emit machine code, like roc and rustc, doing memory-unsafe things is a big part of the job
I'm not sure why that would be the case? Plenty of compilers have been written in OCaml or Haskell which doesn't let you do memory unsafe things at all. After all, emitting machine code is just a matter of spitting out bytes to a file. You can construct the bytes in a vector first and then write it, right? Now, if you were to do some kind of interpretation or jitting, I would understand, but not specifically for compilation. What am I missing?
Yeah I definitely could have made this part clearer! I agree - emitting machine code is not unsafe, since it's just writing bytes down - it's only once you execute that machine code that there's potentially unsafety. The reason I said "a big part of the job" is that in practice a lot of compilers both emit machine code and execute it - but you're totally right that it's not a requirement that a compiler do both.
In addition to the examples you gave (interpretation/jitting/etc.), others would be things like evaluating userspace code at compile time (e.g. const fn in Rust, or in Roc any expression that could be hoisted to the top level), running tests and inspecting their output to decide what to display to the user, etc.
Those are the types of things I had in mind when I wrote that.
Interpretation would be the safest option here, right? Since e.g. a Roc to WASM compiler could be entirely in safe code and you can make a WASM interpreter without dynamically generating executable pages.
You can, but then of course it runs slower. We used to use an interpreter for tests and compile-time evaluation of constants, but now everything is done by compiling to machine code and executing it - specifically in response to people encountering things taking unacceptably long to run in practice.
The point, I think, is that they want a very fast incremental compiler, just like the Zig compiler. For that you need a low level language and to do some pretty unsafe stuff.
What am I missing?
the post counts miscompilations that cause memory safety bugs as memory safety bugs in the compiler (which is pretty fair I guess):
Compilers emit machine instructions. When a machine executes those instructions, they can cause memory corruption, resulting in memory corruption bug reports from the people who experienced them. Regardless of which process had the bug—the compiler or compiled program—in both cases the processor only did the bad thing because the compiler told it to. And in both cases the fix is the same: the compiler's code must change, since that code was what caused the memory corruption.
Plenty of compilers have been written in OCaml or Haskell which doesn't let you do memory unsafe things at all.
One thing, which is the component that Roc was using Zig even whilst they were using Rust for the compiler, is something like the runtime / standard library. These are lots of bits of code that are extern and linked in or implementing garbage collection, both of which would use a lot of unsafe rust and obviate many of the advantages of safe rust.
Additionally, if you're just talking about the compiler, you can do that, the question is performance. The Zig / Roc compiler use a lot of struct-of-arrays representation and use indices into arrays instead of pointers which, if done in Rust, side-steps the borrow checker and result in you losing the related safety benefits.
The lifetimes within a compiler often aren't that complex either, at least in the Zig one you basically allocate / generate / mutate a bunch of things in an arena in one stage, pass that read-only to the next stage, which then destroys the whole arena when it's read the data.
The more complex things come when you get to incremental compilation / linking. You're reading a bunch of state from disk and modifying a binary in place. There's a whole bunch of corruption / bugs / mis-compilation / memory problems that you can introduce there, but none of them are with your program.
Safety and correctness are so much more than the internal memory safety of your program. Especially if you want to safely and correctly do something in a way that is hard to encode in Rust.
if done in Rust, side-steps the borrow checker and result in you losing the related safety benefits
It doesn't result in losing memory safety though. Using the wrong index will cause bugs but will still read an initialized region of memory using the correct type. And the borrow-checker still prevents you from doing things like pushing to a vec while holding a reference to one of the elements, which is the main way I get UAF in the compiler I wrote in zig.
Those compile times are nice though.
My point is that these bugs:
Using the wrong index will cause bugs but will still read an initialized region of memory using the correct type.
are the same in Zig, so rust isn't providing additional memory safety there. And that these bugs:
And the borrow-checker still prevents you from doing things like pushing to a vec while holding a reference to one of the elements
Don't happen if you're using indices (and don't delete individual nodes) because they're offsets rather than pointers. Even for the deletion case, Rust doesn't catch that with indices.
As the author of the mentioned compact_arena crate, I am glad to get a shout out, but also a bit unhappy they are misrepresenting my use of unsafe here. The whole reason of compact_arena is to make the indexing operation safe by ensuring that new would not be possible to misuse by safe Rust. And just for the record, the mk_arena macro can be safely called from safe Rust code even in a loop (where you don't know how many arenas you need).
Hey, thanks for making compact_arena!
Maybe I'm missing something, but regarding "make the indexing operation safe by ensuring that new would not be possible to misuse by safe Rust" - I think I might be missing something. The "Safety" section of https://docs.rs/compact_arena/0.5.0/compact_arena/struct.SmallArena.html#method.new says:
The whole tagged indexing trick relies on the 'tag you give to this constructor. You must never use this value in another arena, lest you might be able to mix up the indices of the two, which could lead to out of bounds access and thus Undefined Behavior!
My point was that there's no potential unsafety when you call new(), but rather when you index into something created with new() - e.g. the "You must never use this value in another arena" part of the Safety note. To me, the part that needs auditing (and therefore marking as unsafe) are the call sites where the indexing occurs.
But maybe I'm misunderstanding something about the design!
Glad you like the crate. The misunderstanding is subtle: When you index, the type system upholds the crate's invariant that an index can only ever be used with the arena it was derived from (via the invariant lifetime tag). The mk_arena macro expressly doesn't leak the tag, so it is safe to use. You can of course use unsafe Rust to construct a SmallArena, and your code will be correct as long as you never again touch the tag used to construct the arena.
If the new function was safe, it would also be unsound, because safe Rust code could misuse the function to construct two arenas with the same tag, insert one element into one and use the resulting index on the other arena, resulting in an out-of-bounds read from safe Rust.
I see, that's a good point in favor of it being correct to mark SmallArena::new as unsafe.
What do you think about my point that the call sites need auditing? For example, if I call SmallArena::new twice with the same type tag, I can now give the wrong index to the wrong arena, causing unsafety. Auditing SmallArena::new itself wouldn't help me avoid those problems; the thing I'd need to audit carefully is, at the call site, "am I giving the right index to the right arena?"
Actually, I think I just realized something about the "Safety" section here. When it says "You must never use this value in another arena" - did you mean "this tag type value?" (I read "value" and thought "as opposed to type," e.g. the index and not the type tag.) Was the intention there to say "Calling SmallArena::new twice with the same type tag is an error that violates the library's safety contract, and unsafe is telling you to audit that your code does not do that" or something like that?
If that's the intended design, I might suggest a wording tweak to the "Safety" section there, but that design seems totally reasonable! Otherwise someone in my position (who's looking for the use case of "number of arenas varies at runtime and isn't knowable at compile time") might mistakenly think SmallArena::new was appropriate for that use case as long as you're careful about auditing call sites.
Yes, the value mentioned in the safety section is indeed the tag type value. And yes, I agree that the wording could use some improvement.
Hot loading is standard behavior for interpreted languages like Python
I thought Python programs had to be restarted to pick up new code, and hot reloading requires nonstandard addons and careful coding. It’s more common to see hot reloading in image-based development platforms such as Lisp or Smalltalk. Or in Erlang.
Could not using cargo build to build decrease build times? Not sure if setting up bazel is more or less work than porting to zig. And not that build times were the only consideration.