Everyone Should Know SIMD

56 points by seg6

payne

I guess everyone should know SIMD but I would argue that not everyone needs to know SIMD. Most programmers work on much higher levels of abstractions where this is not only not needed but would also constitute as premature optimization.

purplesyringa

I think it's still valuable to know what SIMD can and can't do, at least. Even if you're programming in a high-level language, the relative performance of the primitives you're using still depends on how they're implemented. Simply knowing that something is optimized with good SIMD tells you that it likely has the same performance on 1-16 bytes, that you might get a small benefit from data alignment, that the length of the input in bytes determines performance more than the raw count (e.g. if you can choose between 8-bit and 16-bit values), and that compiling for AVX instead of using the default settings will likely double your performance. That's already pretty useful!
- jlarocco
  
  What I'd like to see is better compiler warnings, and optional warning levels that note optimization misses. Specifically in this case, when the compiler fails to auto-vectorize a function or loop, print a note explaining why, and suggest a code change that would make auto-vectorization succeed.
  
  SBCL Common Lisp compiler will print notes when it can't apply an optimization, and it's really handy. Although, TBH, a few of the notes can be cryptic.
  - purplesyringa
    
    clang provides something like this, actually: https://llvm.org/docs/Vectorizers.html#diagnostics
- Sarcasm
  Feel like Zig make this more pleasant to work with than some other languages.
  
  Sometimes I contemplate using SIMD but then I wonder:
  
  Is there some restriction on the buffer alignment?
  
  How to build my software once and run it on heterogenous hardware? CPUID dispatch does not seem straightforward.
  
  I'd often want to bench/test the SIMD implementation against the baseline implementation but it means they need to coexist at runtime.
  
  Well, picking the baseline target at compile time may not be such a bad idea after all, it makes writing SIMD straightforward.
- 0x2ba22e11
  
  One thing I really don't understand in the example is why it's using u32 for each lane when u8 is enough precision and range for all the steps? @mitchellh?
  - purplesyringa
    
    The post says the values are codepoints, maybe this is after UTF-8 decoding?
    
    0x2ba22e11
    
    Aha, I missed that. Thank you!
    
    (Obvious question then is whether maybe this loop could be made to use less memory bandwidth by doing it on utf8 bytes before decoding, but that depends on the program's structure.)
  - abbe
    
    Title reminds me of What every computer scientist should know about floating point arithemetic, and What every programmer should know about memory.
  - erock
    
    Great article and something I was curious about when exploring the ghostty codebase. Also while exploring the foot codebase I noticed they use PGOs for compiler-optimizations. It looks like zig doesn’t support PGOs at this point but I’d be curious if ghostty could get some gains out of employing a similar optimization strategy on top of simd