A Final Return for OpenBSD Anti-Return-Oriented Programming Mitigations
18 points by fro
18 points by fro
The short summary: OpenBSD mitigations are pretty effective at removing hidden sequences of instructions ending with a ret. Unfortunately, there are other sources of instruction sequences ending with a ret, not just misaligned instructions. For example, functions tend to end with ret, and those returns can also be used as gadgets.
You don't need many gadgets to create a Turing machine, so the mitigation doesn't help so much.
We compare our GCC-ported standalone utility to the original OpenBSD LLVM mitigation and discovered that our standalone utility is weaker compared to the original LLVM-based mitigation. However, due to the overall weak reduction in gadgets for both the LLVM-based and GCC-based implementations, we conclude that seemingly obvious mitigations may prove to be anything but, and caution providing security improvements without significant testing and evaluation.
I'm looking at the paper, and I might be misunderstanding something, but it looks like these ROP mitigations barely remove any gadgets (places that attacker can use in their ROP attack), and with noticeable binary size and performance overhead? So basically ... it's all rather meh?
Correct; you are trading measurable binary size and performance impacts for what amounts really to nothing in terms of security.
I spoke about this paper and the previous paper at NYC*BUG not that long ago. The slides are here: https://www.nycbug.org/media/2026-06-03_Lets_Review_Some_OpenBSD_Mitigations.pdf and there is a video on YouTube: https://youtube.com/live/hTI2KstsAfY
We have a third paper in this sequence out for review right now, so I can't say too much until we get reviews back, probably towards the end of this month. But for now I'll say we found another OpenBSD anti-ROP mitigation that hurts performance and gains nothing in terms of security. There will be a post on the blog once that paper has been accepted for publication.
This shows the findings described in https://lobste.rs/s/tuyk3l/semi_retirement_really_changing_my
I'm wondering if this could be helpful as a high-level signal of active exploitation. Meaning, if instead of crashes, I get a SIGTRAP, instead of a more generic SIGSEGV or something. Then, this could be interesting to monitor for security purposes ? As the question is usually not if people are gonna get inside but how/when/where. I'm curious to hear what other people would think of this :)
That would be a different kind of mitigation: things like pledge, unveil, and OpenBSD's library and kernel relinking are more along these lines. Relinking in particular might be something interesting to study next. I am sold on pledge and unveil being absolutely great and I wish more operating systems would adopt them.
I guess both are useful to get this signal ? Meaning, if someone wants to try ROP-ing in a publicly exposed nginx, that would be interesting ?
On the pledge's side, totally agree it's quite good UX I think.
I wish systemd's version would be available as a lib consumable outside of systemd. That'd be nice for the overall community I think :)
(See https://www.man7.org/linux/man-pages/man5/systemd.exec.5.html, SYSTEM CALL FILTERING has things similar like @basic-io @network-io, etc)
Except that's not what alternative register selection or compile-time instruction rewriting buys you. They don't give you a signal for active exploitation, or any exploitation (or any signal) for that matter. If you tried to trap in those locations, you would never get any work done because you would be constantly stopping at legitimate instructions. And I do mean constantly, because a huge percentage of subroutines end up getting at least one instruction rewritten from these changes.
By way of imperfect analogy: imagine that from now on every time you wanted to say the word "have" you instead said "possess" because you heard attackers can ruin your day if you say "have" and the words immediately preceding "have" might in some rare instances be reinterpreted to make you say something bad you never intended to say. You saying the word "possess" doesn't suddenly become a signal that your day is ruined; attackers would just ignore "possess" because it isn't "have." So you get no signal. That actually sounds good, until...
You realize you are slowing down your speech (two syllabus for "possess" versus one syllable for "have") and making your writing a little longer for nothing, because indeed nothing bad will happen to you if you say the word "have." And if you stopped every time you said the word "possess" all you would do is prevent yourself from saying sentences that you need to say to achieve your goals.
Even if on very rare occasion an attacker could reinterpret your words (polymorphic gadgets, uncommon and rarely useful) to ruin your day, they don't bother because they don't need to because all the other words you already said elsewhere is enough to ruin your day, no reinterpretation needed (intended gadgets, which are far and away both more common and more useful).
Ironically, the removal of these "mitigations" is what would get you that signal. Because then you might see attackers trying to use those polymorphic gadgets. And then if you saw that, you could write a much more targeted defense for those gadgets. But in reality, attackers will just use the intended gadgets. That would bring us to a conversation about RETGUARD, pledge, IBT/BTI, and kernel and library relinking. And that's where the interesting conversation is.
Alternative register selection and compile-time instruction rewriting are really just failed experiments that have outlived their time and probably should just be removed. The paper we have out for review right now hammers this point home much more forcefully. Hope to be able to talk about that one soon.
Thanks for answer, it's much more clear! I went to it too quickly :'). I think my pov makes more sense for things like 0xCC padding (instead of nop padding) that openbsd implemented. I understand it's not really a protection though, more of a signal.