Mitigating the Billion Dollar Mistake
12 points by rodef
12 points by rodef
This article is continuation to: Was it really a Billion Dollar Mistake?.
Unfortunately existing languages like Java cannot have these problems solved, but newer languages that want to stylize themselves similar to that could solve them.
Typescript and Kotlin do all three of the mitigation options he lists, and IMO it works very well.
As I said, I think this works very well, and it’s rare that I have to use an escape hatch like “!!”.
C# does all of these, too, and is also a counterpoint to the "existing languages ... cannot have these problems solved". The change did get made along with a bunch of other incompatible changes so there was one big flag day, but I haven't heard a terrible amount of complaining about it.
If I understand correctly, the underlying thesis may be re-stated as:
In languages which distinguish values and pointers:
There is a performance cost to mandatory explicit initialization which makes it unsuitable for certain kinds of situations
The additional code required to initialize everything explicitly obscures the underlying logic. On the other hand, one can get used to default zero initialization (with null pointers being a special case).
The bugs introduced by virtue of null pointer dereferenced are not sufficiently numerous or serious to warrant paying the costs in 1 and 2.
If my understanding is correct, then (1) should have examples of programs in a language like Rust (which require mandatory initialization) and those should be observably slower than the equivalent Odin programs. (1) probably needs some more evidence that such programs are representative of the ~typical program in the language of interest. Understandably, this is hard to supply/prove.
If you look at sources, zero initialization itself has ~zero overhead for stack variables (see: https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2022/p2723r0.html). Zero initialization overall has overhead of maybe 1-2% at most (https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf), if you implement some optimizations (e.g. batching). This is relative to no initialization. IIRC, JF Bastien measured the overhead of pattern-initialization and that was <5% for most cases but I don't have a source for it.
Re (2): This is a bit hard to refute modulo doing concrete measurements on the speed of code understanding. IMO, it seems plausible that this doesn't matter, or it's heavily dependent on the coding patterns in use.
Re (3): I think this is where the readers of the precious article disagreed with the author. However, the author seems to dismiss them:
you don’t actually understand the costs fully if you are answering the way that you do
For this, the author cites personal experience:
I’ve been in projects where a lot of the time in a program in spent in the destructors/Drop traits of individual elements, when all they are doing is trivial things which could have been trivially done in bulk.
Sure, citing personal experience is valid, but then if you're dismissing other people's personal experience as invalid, surely that's not reasonable, is it?
Then there's also the more obvious counter that Zig is a modern performance-oriented memory unsafe language (similar to Odin), with native support for SoA and multiple talks by Andy Kelley on data-oriented design. But even Zig doesn't allow pointers to be null by default, you have to opt-in to that. IIUC Zig also requires explicit initialization but you can opt-out of it via undefined. This calls into question both claims (1) and (3).
Thank you for reading the article and for the comment! However your summary is missing quite a bit.
I didn't want to repeat myself in this article but a huge aspect of the previous article this was about the individual-element mindset vs the grouped-element mindset. Your point (1) is partially correct but it's not about the cost of explicit initialization, but the cost individual-element based initialization rather than grouped-element initialization, as well as destruction of those elements. My personal anecdote regarding destructors is part of this argument too. The point here is that all of that destruction could have been and should have been done in a bulk operation, which would have improved performance of that a lot. Doing things in bulk will be faster than doing things individually, especially even for the trivial case of allocation.
To take malloc as a basic example, calling it N times vs calling it 1 time has a completely different performance overhead to it, even if the total requested memory is the same. And calling free N times in different places vs calling it 1 time again has a completely different performance overhead. Then compound that across the program for everything. The individual-element mindset I am arguing will result in slower code and harder to manage code because you are now managing individual unique lifetimes of individual elements, rather than thinking of the grouped lifetimes of the grouped elements as part of a system. When initializing things across a code at unpredictable times, the performance of the thing is harder to keep low too because the thing is scattered.
When designing a programming language, I am considering about how do I design a language to aid people in programming. And part of that is about nudging people in a way I seem "better", even if it is a subtle nudge that may not do much for some. You can argue that the nudge could be done as part of the general community ethos, but that's a much weaker "nudge/enforcement" than doing it at the language/compiler/core-library level directly. And this aspect of initialization of variables either being implicitly zeroed or explicitly initialized with an individual element is one of the minor design nudges which can have non-obvious architectural design decisions as a project scales.
So I am not dismissing anyone else's personal experiences, rather saying that if you look at other experiences, they do correlate with the view that grouped is better for performance and easier code management because lifetimes and storage are grouped, rather than the individual-element approach with individual unique lifetimes and storage leading to harder code management.
...examples of programs in a language like Rust (which require mandatory initialization) and those should be observably slower than the equivalent Odin programs.
In your example, if they are equivalent, then they would both be doing the same mindset. What you want to compare is the different mindsets within the same language, not across languages. And we can already see when projects need performance, they resort to grouped-element based architectures. This is how virtually anything works when you need to scale in fact. And the language in use is no different.
n.b. I personally really hate language benchmarks because they rarely measure anything useful, and this is no different.
Regarding the sources you link, there are stating that zeroing memory has a very very minor cost compared to uninitialized memory, and the second one even suggests doing bulk zeroing over individual element zeroing to improve things. I am not sure why you brought this up, especially when these articles are using zeroed memory for security purposes and not necessarily architectural performance benefits, because it's mostly irreverent to the discussion of the article: individual-element vs grouped-element based thinking. At least with Odin's zeroing (of everything too, including struct padding), that does give it some security (and determinism) benefits. Other languages do not necessarily zero the padding and leave that "undefined". And as I say in the previous article, in many cases getting zeroed memory is of no extra cost with things like mmap/VirtualAlloc/etc.
P.S. The second paper that you gave is also from 2011 and running on IA32 hardware, and even some of the very first quad core hardware. And the first paper cites the second for the performance numbers. So it's not necessarily a useful paper to look against.
As for how other languages do things, that doesn't bring any claim into question, it just means I disagree with the design philosophy of those languages. They do things different to what I am suggesting, and how they want to nudge their users into programming in a certain way is just a different philosophy to language design, and of course I made different decisions with that.
I know a lot of people view the explicit individual initialization of every element everywhere approach as the “obvious solution”, as it seems like low-hanging fruit. As a kid, I was told to not pick low-hanging fruit, especially anything below my waist. Just because it looks easy to pick, a lot of it might not be unpicked for a reason. It does not mean that you should or should not pick that fruit, but rather you need to consider the trade-offs.
As an adult, I know that I can shape a fruit tree such that it primarily gives low-hanging fruit. I also know that I can let the tree grow tall, so that its branches are leafy and hold many bird nests and offer high-hanging fruit for the tree-dwellers, while still offering plenty of low-hanging fruit.
Ah but if you wish to attract native birds, then certain kinds of trees should not be grown at all! Tall trees can also interfere with power lines.
This is still a programming metaphor, I promise.
Indeed. First, what kinds of trees ought we choose? If we choose only one species then we risk blight. If we choose only one type of fruit then we risk pests. We do need a diverse choice, and further, we want to form a rich canopy whose inhabitants pollinate our crops and also who form their own food chain so that the leaf litter is enriched enough with their droppings to sustain the trees without fertilizer. And of course we want to not plant certain selfish species which generate a monoculture or which destroy nearby competitors.
Second, how ought trees interact with infrastructure? As trees grow, they generally choose one of two strategies. Either they put up twigs which are immediately woody but thin and brittle, or leafy blades (also called shoots) which are wide and yielding but soft and pliable. Twigs tend to get snapped off by overhead lines, causing the tree to grow around the line and giving it a wide berth; this is safe for infrastructure but limits which shapes the tree can take. We see in forests that many twiggy trees are naturally crown-shy; the abrasion between twigs and their environment, or perhaps the occlusion of light caused by nearby things, causes them to stop growing. Twiggy trees restrict their shapes from expanding into spaces already occupied by trees or other tall things. A tree made entirely out of grassy blades, like a palm tree, always grows vertically and will nudge overhead lines out of the way, potentially stressing them until they snap. Interestingly, it's known that palm trees are always unsafe to plant underneath high-voltage lines because they are guaranteed to grow towards the lines and eventually spark a fire.
On that second topic, there are more interactions between plants and infrastructure than we can easily imagine. A neighbor's agave has had its flowering stalk up for about half a year, starting in summer and somehow persisting through winter. The stalk happens to be less than a foot from a low-voltage line (possibly cable Internet) and bumps against it occasionally, appearing to lean on it sometimes, but always supporting its own weight.