Quadrupling code performance with a "useless" if
120 points by purplesyringa
120 points by purplesyringa
It looks like annotating the if-statement with the C++20 [[unlikely]] attribute is sufficient to prevent clang from eliminating it: https://clang.godbolt.org/z/r4xYWfPfe
I haven't checked other compilers though.
I thought I tried it and it didn't work, but apparently it does, thanks! Updated the post. Maybe I accidentally checked under GCC instead of LLVM (it doesn't help under GCC).
Do note, though, that [[unlikely]] has the effect of moving code out of the fast path, which is why LLVM omits suboptimal code like
cmp r8b, dl
jne not_equal
continue:
...
not_equal:
mov ecx, edx
jmp continue
instead of the more reasonable
cmp r8b, dl
je skip
mov ecx, edx
skip:
...
And, of course, it tries to be smart about not loading a value from memory multiple times, even though that adds one instruction to the hot loop. So I think my trick with volatile is still useful, but it's good to know that it might just work as-is!
FYI this is called "value speculation" elsewhere: https://mazzo.li/posts/value-speculation.html
Cool stuff, thanks! My first thought after I reinvented this method is that it might apply to linked lists in arenas, and it seems like the linked post covers exactly that. Added a link to mine.
This article made me wonder if people will even care about this kind of blog post in 2 or 3 years, when fewer and fewer coders will actually care about their LLMs' output.
How many people care today? Very few.
It's niche knowledge on the broad scale of programming.
That still makes it a very useful and good post for people that do care.
Godbolt is very popular, despite LLM-pilled developers thinking "do you look at the assembly?" is a an oh-so-awesome retort when anybody questions the habit of not reading the code. Yes, I look at the assembly and I will continue to do so.
People who think that the details don't matter are crap developers and will always be crap developers. A good civil engineer cares about the nuts and bolts in the large projects that they build, they don't feel too fucking above it all to get down in the weeds about the small stuff. Likewise a good developer will always have to care about the details. Because the totality is made out of details and there is no escaping them if you care about what you build.
In defence of not looking at the assembly: very often it is the case that enough of the information you need for making performance improvements, about which optimisations did or didn't fire, is present in a higher level IR too, and those are usually easier to read. ;)
I don't think it is any different from today. Most people don't care about the compiler output for handwritten code. Will less people care about the compiler output of their AI-written code? Probably, but I don't think it fundamentally changes the equation.
I think it is just a reshuffling of hierarchy of care.
Ten years ago, writing a Bash script and noticing it is somewhat slow, I would definitely already not be thinking about instruction-level parallelism and manipulating CPU predictions!
There might be a performance core worth optimising in a compiled and optimisable language, within that core there might be hot paths worth looking at in minor details and maybe thinking how this vectorises, within the hot paths maybe there is one or two hot loops where checking what the current-decade CPUs do differently is worth the effort.
Even with an LLM, the hottest point worth checking for performance will probably be exceptional but worth checking LLM's work. But also maybe the advice about proper use will age well. I mean either consciously deciding that garbage quality is fine, or making LLM get 70% there and hand-fixing the remaining 30% — well maybe it gets to 85:15, but the article is still a very 1% thing.
Mind boggling to me that I can make a thing faster and there's always people that ask "but why?" What kind of mentality is that? The pursuit of excellence does not need justification. Also, I find in so many cases, we can't know the impact of an improvement until we do it.
- @mitchellh (Source)
You will care if you need to make a product work and you need 4x more performance for it to run on the provided hardware.
I don't write C++, so in that sense I already don't "care" today; but I still found it interesting, and I assume I still will find these kinds of things interesting in 2 or 3 years.