Compiler Engineering in Practice - Part 1: What is a Compiler?
10 points by dhruvp
10 points by dhruvp
The focus on “miscompiles” leads me to reflect that compilers have gotten so much better in my lifetime.
I used to find and report compiler bugs pretty often, back in the 80s/90s, in Lightspeed Pascal, THINK C, Symantec C++, Metrowerks C++. Fortunately most of those caused bogus errors or crashed the compiler, but there were a fair number of miscompiles. Those are damn hard to discover!
I can’t remember the last time I found a miscompile. I’ve found bugs recently in e.g. Clang's implementation of C++ coroutines, and I’ve crashed MSVC a number of times, but I don’t remember any where the binary code did the wrong thing.
I wonder how much of this is improved coding & testing styles, improved tooling like sanitizers, and how much is due to the many-eyeballs benefit of open source? (Those old compilers I mentioned were all proprietary. I don’t think I’ve used a closed-source compiler since Y2K!)
I can only speak for LLVM as its the only large compiler I worked on. Regressions actually happen all the time, but most are quickly caught and reverted due to intensive and continuous testing by various parties of latest main (both public and internal CI testing stage2 builds and llvm-test-suite (random collection of some applications and benchmarks), continuous downstream testing on important workloads, etc.). Subtle regressions can linger for months/years until found. Due to complex interactions, code review is only of limited use and sanitizers are not helpful w.r.t. miscompiles (actually, I only ever built LLVM once with ASan). In-tree tests only catch some errors, but inherently cannot test all interactions/variants/etc.
As example, just last week I landed a LLVM commit that caused miscompiles — stage2 builds of LLVM would generate "slightly" wrong code for MIPS in some cases. This was quickly caught by some post-commit CI runners doing stage2 builds and reverted quickly. Tracking down the root cause fortunately just took half a day (root-cause analysis in general can easily take several days in more complex cases).
PS: Regarding miscompiles and buggy code in general, I often tell my students: code that is almost correct is like a plane that almost flies. :-)
Interesting! Does the CI testing run tests on the programs it builds, or does it compare the generated code with a snapshot and warn if it changed?
Nice article! This one looks like an intro. And I am looking forward to more detailed follow ups.
That said, the heavy mentioning of AI, in particular "a miscompile of an AI program can cause bad medical advice," irks me. But I get that the author probably work on that area, so it is understandable.