Inside Zig's New Writer
41 points by vpol
41 points by vpol
it’s very interesting to me how often zig is willing to make big breaking changes despite it’s user base. bun, ghostty, tiger beetle, just to name the popular ones. I don’t know how they keep up with the churn. we still haven’t managed to upgrade to react 19 at work and that was a relatively minor upgrade (from 18).
Speaking for the Zig project only, we do try to sweeten the deal by providing real, measurable improvements alongside breaking changes, not only with machine code performance but with developer velocity (i.e. compilation speed). For example in this 0.15 release last week, usingnamespace was deleted, but in exchange, incremental compilation provides type checking feedback in a matter of milliseconds. Streaming I/O API changed, but the x86 backend is now the default, offering 5x faster compilation. Build system API changed, but threaded codegen reduces development feedback cycle time. Etc.
Ultimately, these projects that you mentioned took a gamble on Zig - trusting in its technical direction even though it was at an early stage. Whether that paid off for them is not for me to say, but the end result is that it breathed the life into the project to get us where we are today, and thanks to them, the next cohort of projects that build upon a Zig foundation will reap the lessons learned over the last 10(!) years; no legacy baggage in sight.
Some more stats that the casual observer might find interesting:
Project | Lines of .zig
----------: | ---------------:
Zig | 1,300,584 (100%)
Bun | 586,176 ( 45%)
Ghostty | 196,662 ( 15%)
TigerBeetle | 143,741 ( 11%)
Nobody felt the pain of those breaking changes more than me…
Hey, I’d like to thank you for these breaking changes!
I had no opinions about writers and buffers, other than a mild annoyance at generic writer type parameters, but after working through my confusion while upgrading my TTY browser runtime, and watching your Don’t Forget To Flush talk I found myself actually fascinated by the topic in a way that feels competence-expanding in the same way as when I first saw your talk about data-oriented design in the Zig compiler.
Growing as a programmer aside, the performance of the Zig compiler, especially now with --watch on x86_64, is absolutely delightful. In so many ways Zig is what keeps me enthusiastic and happy about programming as I descend into old age…
Can’t speak for bun or ghostty, but it’s definitely not “despite TigerBeetle” (I worked on TigerBeetle since Zig 0.9). Zig breakage is a non-issue for us. We don’t have dependencies, so we don’t have to coordinate upgrades of code we do not own. We also generally re-implement parts of standard library we rely on heavily. The residual upgrade work is tiny, I probably lost more time to wrangling windows API this week than it would take me to upgrade TigerBeetle to writergate.
Interestingly, the three scandals this release (writergate, removal of bounded array, deprecation of managed ArrayList) is something I’ve personally advocated for (e.g https://ziggit.dev/t/sketch-of-type-safer-buffered-io/4810).
Zig is definitely the right choice for TigerBeetle, it would have been a worse database in any other language.
Can you expand on why you like the removal of std.BoundedArray? I thought it was quite convenient for small lists with a known upper bound. For example I used it in graph algorithms that needed to store 0, 1, or 2 parent indexes (and various other structs, making it annoying to reimplement for each one). Whereas now with (unmanaged) ArrayList.initBuffer:
For that use case specifically (graph algorithm; 0, 1, or 2 parent indexes), don’t you benefit from keeping each node small? In this case, already you were wasting 62 bits on the len field of BoundedArray.
See https://ziggit.dev/t/boundedarray-vs-open-coding/9047/4
I find that open-coding is usually less annoying than using BoundedArray, and often it’s better to operate on slices anyway. Zig doesn’t have Deref & operator overloading, so the cross-over point where a generic data structure starts to make sense is usually further along. But it’s hard to to speak in generalities, without looking at the specific examples. Specific examples in TigerBeetle, where we use BoundedArrays, are usually better without, e.g. https://github.com/tigerbeetle/tigerbeetle/pull/3181
I am curious why they didn’t go the way Rust does with trait bounds? With trait bounds, each concrete type is specialized so dynamic dispatch is fully eliminated. The downside is increase in the compile time and in the executable size.
The trick is that you want both fast runtime and fast compilation/small binaries. You can easy get the former by making everything monomorphized, you can easily get the latter by monomorphizing the world. Zig is trying to have the cake and eat it to, that’s the whole idea behind writergrate.