schrodingers-toctou: The binary you run is not the program you wrote

7 points by juri

fanf

tl;dr: someone got an AI to complain at length about data race vulnerabilities. They are shocked that what looks like one read in the source code might not be one read in the machine code. They don’t like the standard fixes: marking every load from shared memory with something like READ_ONCE() requires too much discipline, and they ignore the possibility of using atomic types to get the ergonomics they want. They have invented a huge number of bogus issues that are only problems if the attacker is already on the other side of the airtight hatchway.

JoachimSchipper

The analysis itself is indeed not new, but the use of LLMs - though it makes for annoying text - does give the author the ability to scan "100+ security-critical projects" and find lots of issues there. I thought the presence of so many issues was quite interesting!

I'd have hoped that people were aware enough of the issue to know that reading across security domains is quite scary, but apparently there's still a lot of issues to be found.

This is (also) a BlackHat talk, and not the author's first BlackHat talk; there is real skill involved, not just LLM slop. But I do wish the author had written his own text.
- fanf
  
  It would be better if they only listed “issues” that actually cross security domains. I got the impression from scanning their list that it’s mostly bullshit slop. They might have a few worth paying attention to, but it’s buried in enormous quantities of irrelevant padding.
  - JoachimSchipper
    
    I also did a quick look. I'll admit that it's hard reading, and that the research would be more useful if it additionally analyzed an actual binary, demonstrated some effect, and generally did a better job of sorting the most useful findings; but I also don't think it really is "bullshit" - although I won't argue too much with "slop".
    
    Specifically:
    
    Xen claims cross-domain (guest), but only on a build configuration that disables HVM. I'm not an expert but I think every vaguely modern CPU will run HVM - but Xen is sufficiently widely-used that someone will use a non-HVM configuration? If so, arguably a real finding in a rare configuration.
    
    OpenEnclave does look to be host-to-enclave, and the audit is at least right that the code incorrectly claims that the target variables are volatile; I'd be digging into the claim urgently if I had reason to care about OpenEnclave. (I've never heard of OpenEnclave, but my impression is that other TEEs, especially open-source, are fairly soft targets.)
- lonjil
  
  But he's not just pointing out that some software has data races (it's well known fact that this is the norm in complex C code bases), he's pointing the finger at compilers as evil-doers.
  
  This is either an elaborate shitpost meant to somehow highlight how hard C is to write correctly, or he's someone I would say you shouldn't listen to anymore.
oliverpool

Check your own code with observer-effect/AUDIT-PROMPT.md

This feels quite wrong. Is the best tool to test this an LLM ?

I would have expected a deterministic script to check the source against the generated code (eventually written by some LLM...)
refi64

Note that two days before this was released, the author wrote on formerly-Twitter:

Been too long without releasing something that made me laugh… time to fix that.

So I'm not sure how this is supposed to be interpreted.
olegkovalov

Love another project from the same author https://github.com/xoreaxeaxeax/movfuscator