Attention people who think UB is fine
31 points by abareplace
31 points by abareplace
Seriously. People need to stop thinking UB is about allowing different behavior.
It is absolutely not. No amount of arguing changes this.
UB is not implementation defined behavior. That term already exists.
UB places no constraints on the compiler. If the standard does not define the result of doing X in condition Y is undefined, or it just does not define the result (either explicitly, ID, unspecified, or erroneous), then there are no constraints on the compiler.
It does not matter what your implementation, platform, or hardware do. If the behavior is UB according to the language, the compiler can do anything, or nothing, or different things in different places. If the language says it is UB, it is UB.
If you want different implementations to be able to provide different behavior, you want implementation defined behavior. That is literally why ID exists separately from UB.
some people insisted that it’s demonstrably fine, we know what the code will do, and at worst we’ll get a segfault. This is false.
UB places no constraints on the compiler.
I think it comments like yours and posts like this contribute to an extremely negative sentiment around the state of undefined behaviour in languages like C and C++. I don't mean this to imply skepticism or suggest blame, but to state, as neutrally as I can, that examples like those shared in the article are genuinely alarming, and the consequence is negativity.
(The “falling through a function” example comes up a lot, probably because it is very simply yet very provocative, though I think -fsanitize=unreachable may handle this problem, judging from its effect on the Godbolt example the author provides. There appear to be other flags I can use to catch other, similar examples, and while these may be worth mentioning, I don't know that they materially change the conclusions of the article.)
I think this negativity results in some wondering whether the compiler folks have gone mad, but in previous discussions of undefined behaviour, comments from folks like ~david_chisnall or ~lonjil suggest that manifestations of undefined behaviour are not a result of some negligence or some malicious intent.
e.g.,
it is impossible to write an optimizing compiler that behaves sensibly in the presence of UB. The only real solution, which pretty much every programming language in the world other than C and C++ do, is to make it harder or even impossible to have UB in your code.
In fact, authors like Chris Lattner have even been quite explicit in pointing out “advantages of undefined behaviour in C”.
As an aside, when authors like ~matklad discuss memory safety, they consider undefined behaviour in their definitions, though this leads to conclusions that I'm not sure I agree with, such as:
This is useless! Here, I’ve just made a language called Lil-C, which exactly like C, except that every UB is formally defined to trap. It is safe! And it can run any C program! Have I just done anything useful? No!
This actually seems like it would be quite useful…?
Anyway, by this post and the above comment's reasoning, it is not possible to constrain the consequence of undefined behaviour; as a result, it is not possible to retroäctively detect that it has performed an undesirable transformation. However, since undefined behaviour arises as an emergent property across multiple (desirable, reasonably optimisations,) I also cannot simply read through a lists of undefined behaviours in C and expect to be able to avoid triggered it intentionally. Understandably frustrating!
But, as a human being, I am allowed (if not expected) to have contradictory desires, and to want all benefit with no sacrifice. I want my programming language to be extremely simple, but also extremely fast, but also very safe, but also without undefined behaviour, but also without giving up optimisations that other languages (less safe or more rife with undefined behaviour) are allowed, and I might be convinced not to write any new codes in C or C++ (in fact, I have found myself running “toward” Zig more than “away” from C,) but I definitely want to make substantive changes to existing codes in C and C++ and refuse to replace or rewrite them. (Regarding this last desire, you know, I figure I already dug the hole… and I'm not going to lose sleep over digging it just a little bit deeper…)
So, rather than simply being negative about undefined behaviour or being negative about C and C++ code (that I refuse to do much of anything about…,) perhaps what would be useful to know would be how I can actually go about triangulating myself. I know I'm in a hole, I know I am digging it deeper, I know I won't climb out, and I know I might hit a pipe… but before the water rises up to my neck, is there no agency I can have in my last moments?
In other words, even if I won't rewrite my C codes, and even if I do want to make substantive changes to them, and even if I know I will suffer from the consequences of undefined behaviour, and even if these will be difficult to avoid through additional mechanism or additional diligence, I still have some obligation-to and corresponding freedom-of action. What do I do in the next (… maybe the last? … hopefully the last?) hundred thousand lines of C that I write? Just flail and drown?
I think it comments like yours and posts like this contribute to an extremely negative sentiment around the state of undefined behaviour in languages like C and C++.
And I think that’s a reasonable reaction. Even the C committee is at last working to eliminate as much undefined behaviour from the standard as possible, because everyone agrees the current situation is intolerable.
undefined behaviour arises as an emergent property across multiple (desirable, reasonably optimisations,)
No, that’s backwards. UB was introduced to cope with corners of the spec which were too difficult to pin down, because of divergent hardware behaviour, or because someone might want to enter a program on punched cards, or because it’s just too hard to determine what havoc a wild pointer might cause, etc. It wasn’t until years later that compilers started exploiting UB to make optimizations that would be unsound according to the original intended purpose of UB. UB is part of the language specification, not a result of compiler optimizations.
What do I do in the next (… maybe the last? … hopefully the last?) hundred thousand lines of C that I write?
Study Annex J, crank up the warnings, write thorough tests, use sanitizers, use static analysis.
extremely negative sentiment
And I think that’s a reasonable reaction.
I know I sure don't feel great about it…
the compiler folks have gone mad
It wasn’t until years later that compilers started exploiting UB to make optimizations that would be unsound according to the original intended purpose of UB.
If it's genuinely the case that the compiler folks have just gone mad, it makes some of this easier to stomach, but I'm not sure that it really changes much. Presumably, the compiler authors were under pressure to deliver ever-increasing units of performance, and this overrode any reluctance they may have felt toward exploiting undefined behaviour, but that suggests this could happen to anyone…
UB is part of the language specification, not a result of compiler optimizations.
What is considered undefined behaviour is part of the language specification, sure. There is, after all, the old joke about the SIMPLE programming language.
However, it is not the presence of undefined behaviour that is of concern but the byzantine behaviour that arises as its consequence.
I believe the argument made in the comments I cited is that the undesirable behaviour is an unavoidable result of desirable optimisations. Somewhat similarly, we might argue that “undefined behavior in programmer-visible abstractions represents an aggressive and dangerous tradeoff: it sacrifices program correctness in favor of performance and compiler simplicity. On the other hand, UB at lower levels of the system, such as machine code or a compiler IR, is an internal design choice that needn’t have any effect on the programmer-facing parts of the system”.
Even the C committee is at last working to eliminate as much undefined behaviour from the standard as possible, because everyone agrees the current situation is intolerable.
Study Annex J, crank up the warnings, write thorough tests, use sanitizers, use static analysis.
I think this is why, as an alternative to just crying one's self to sleep every night, naïvely trapping on any undefined behaviour (e.g., via mechanisms like Fil-C) seem to be attracting attention, even in the absence of the C committee taking any action.
If it's genuinely the case that the compiler folks have just gone mad
They haven't gone mad, it goes something like
Now optimizations combine in spooky and unpredictable ways sometimes. Like a particular rewrite of an IR may open up new opportunities for inlining or constant folding, and now a handful of very sane optimizations look like the compiler devs went crazy "exploiting" UB. Making decisions assuming the absence of behavior is something we do all the time.
naïvely trapping on any undefined behaviour (e.g., via mechanisms like Fil-C) seem to be attracting attention
The problem is that this is equivalent to removing optimizations and increasing complexity in the design. You need to detect the presence of something that by definition has no presence in correct programs then emit instructions for it that are carried through optimization passes, linking, and loading.
Where I think the gap lies is that UB is supposed to be things that are so unreasonable in correct programs that no compiler could reasonably define a check for it. But C is filled with UB that's easy to detect and reject, which is why the language is moving to define previously undefined behavior (usually as compile errors).
Study Annex J, crank up the warnings, write thorough tests, use sanitizers, use static analysis.
I read and write C for my day job, literally hours every day, have for years. This is how you do it. I'm still a little baffled by how much, uhh, discourse there is about C and it's perceived shortcomings. Not that I think UB is fine, or C at the edges is easy, or anything else. Rather, it's like, if that's the tool you're using, by choice or otherwise, why wouldn't you learn and do everything you can to maximise its strengths and minimise its weaknesses?
I don't get this response?
My entire point is that UB is not about supporting different platforms, that "implementation defined behavior" is a term that exists in both the C and C++ standards specifically to say "this operation has defined behavior, and the implementation (compiler, platform, etc) defines that behavior".
The big problem is when people apply "UB is for cross platform support", and then treat UB operations as being ok because they know the platform that they are targeting. Which it isn't.
If you want to make a difference, write proposals to WG16(C) and/or WG21(C++), to make platform defined operations be specified as having defined behavior (defined, implementation defined, unspecified, erroneous) rather than being UB.
An implementation is allowed to provide a definition for what the C standard leaves implicitly or explicitly undefined. There are also standards, such as POSIX, that themselves define some things which standard C leaves undefined.
This is not to say that all UB is like this, or that all people who write C and introduce UB into their code know what they're doing, but some people do know what they're doing.
If you are writing C and you are writing C against one of these other standards, or specifically against a specific implementation, there is nothing wrong with relying on the specific explicit definitions of behaviour which may otherwise be undefined, as long as you're doing it consciously.
(I maintain, however, that nobody that is looking for advice on whether they should write C should not be writing C. The people who are writing C and are not looking or advice on whether they should continue to write it should also seriously consider moving to something with fewer footguns if they can.)
An implementation is allowed to provide a definition for what the C standard leaves implicitly or explicitly undefined. There are also standards, such as POSIX, that themselves define some things which standard C leaves undefined.
"leaves undefined" is not what UB means at all is sometimes what UB means
"leaves undefined" is not what UB means at all
So, can anyone articulate precisely what is the difference between the two? Is it really the fact that there's two flavors of "undefined", as in "just" behavior that the C standard had left undefined and UB?
I really cannot think about how, precisely, is the ontological difference between the two supposed to look.
“leaves undefined” is a subset of undefined behaviour. The C standards say:
If a “shall” or “shall not” requirement that appears outside of a constraint or runtime-constraint is violated, the behavior is undefined. Undefined behavior is otherwise indicated in this document by the words “undefined behavior” or by the omission of any explicit definition of behavior. There is no difference in emphasis among these three; they all describe “behavior that is undefined”.
This is an example of UB existing to cope with corners of the spec which are too difficult to pin down, as I mentioned in an earlier comment.
You cannot write code that depends on UB.
There are also standards, such as POSIX, that themselves define some things which standard C leaves undefined.
And that does not matter.
I C says an API is UB if used in a specific way, it does not matter what the API implementation does or says. If the C standard says that is UB, it puts no requirements on the compiler.
There is no difference between using an API that posix says is defined but C says is not vs dividing by zero, or overflowing an integer.
If you write C code that depends on implementation defined behavior (POSIX), but C says is UB, your C code is wrong.
The fact that you can get away with it today, doesn't magically make your code correct. It does not mean that it will work if you change header orders, or inlining changes, or anything else.
If your code invokes UB according to the language, you are not writing conforming C. Maybe your compiler allows it (for example -fwrapv, -ftrapv, -fno-delete-null-pointer-checks, etc) but that is no longer conforming C, and your program is wrong vs the standard: any other C compiler is free to assume your UB code does not occur, or is unreachable, etc.
So nobody should program for POSIX in C then? I can't use dlopen() because it mandates that a void * can store a function pointer? I can't use memset() on a structure with pointer fields and assume they're now NULL?
To a point, yes, those violate the C standard, but if I'm targeting POSIX, I have less UB do worry about than pure C (not much, but some).
So nobody should program for POSIX in C then?
What?
I can't use dlopen() because it mandates that a void * can store a function pointer?
You're right, much like signed integer overflow, it often works. It's also not correct C.
I can't use memset() on a structure with pointer fields and assume they're now NULL?
You can, that's what Implementation Defined behavior is for.
To a point, yes, those violate the C standard, but if I'm targeting POSIX, I have less UB do worry about than pure C (not much, but some).
This is literally my point. You can't say "I am targeting POSIX and therefore what the standard says is not relevant". If C says X is UB, it is UB. Being able to get away with it does not make it non-UB, and it does not make your code correct.
For many many years people relied on null dereferences as having safe termination semantics, and then compilers said "well that's UB" and suddenly all that incorrect code became insecure incorrect code.
If your code relies on UB, your code is wrong according to the standard. If a compiler chooses to, that incorrect code can become a symptomatic bug in future.
Getting away with UB today does not change anything.
Let's look at your first point: void* cannot hold a function pointer, that's UB. Code that stores a function pointer in a void* is wrong. What it should be is implementation defined behavior, but like so many examples of implementation defined behavior in C and C++, it is labeled as UB.
If you want to store function pointers in a void*, and have your code be correct, you need the C committee to make it implementation defined behavior. Or you can join the linux kernel and acknowledge that standard C cannot be used for systems programming due to it incorrectly labeling implementation defined behavior as UB.
Anyway, my point is simple: UB does not enable different platform behaviors, it mean any code relying on that implementation defined behavior is wrong, even when a platform does define the behavior.
The first example reminds me of https://kristerw.blogspot.com/2017/09/why-undefined-behavior-may-call-never.html (2017).
I give up. I do not understand why people responded to my comment justifying why they want to use UB as if my complaint were them doing that. My complaint is very simple: something being UB means an implementation can do whatever it wants. It does not mean that you, the developer, can rely on that behavior.
That's literally what UB means.
I did not, at any point say "you should not be allowed to rely on this".
I said that by labeling such operations as UB, C and C++ said that you are not allowed to.
In every one of these cases, all you need to do is convince the C and C++ committees to just define the behavior. Doing that simply requires replacing "is undefined" with (or actually saying at ll" "is {implementation defined|unspecified|erroneous} behavior." Those changes instantly gives you what you want, rather than what you are actually being given.
Anyway, I'm done talking into a wall of people rejecting the plain language of the spec, and the demonstrable behavior of C and C++ compilers when reasoning about UB operations. Instead I'll focus on actually improving the standards instead.
People respond to your comment in that way because they do exactly what you yourself suggested they do:
Or you can join the linux kernel and acknowledge that standard C cannot be used for systems programming due to it incorrectly labeling implementation defined behavior as UB.
That's what most C programmers, in fact, are doing. They aren't writing standard C, they are writing "POSIX C" or "GNU C" or the intersection of "GNU C" and "MSVC C" or whatever.
I give up. I do not understand why people responded to my comment justifying why they want to use UB as if my complaint were them doing that.
I don't get this response?
&c.
People are responding in that way because it is not clear what, exactly, you are complaining about, except that you are complaining about something and this something involves what everyone else is doing.
It is not helpful to get up on a tribune and shout "you, all, listen here, you all absolutely cannot fly for any reason whatsoever" if people around you are, in fact, routinely flying. Maybe with varying success, maybe some of them are crashing and burning randomly, maybe it all works due to sheer chance (that sheer chance that occurs 19 times out of 20), but they are indeed flying.
Just getting up there and shouting "no, you all cannot do that" is not really helpful simply because it is not clear what, exactly, are you saying and how exactly does it relate to the reality around you.
Sure, if you write an actual std::unreachable() at a point in code that is quite clearly actually reachable, or take a literal null pointer and dereference it, the compiler may generate code that has what could be seen as pretty whack behaviour (and even that is happening less as there's been more focus from the compiler engineers on avoiding it - witness the output of the GCC compiler that is referenced in the actual post). But coders generally don't do these things, because they don't make sense in the first place. I'm not saying these sort of thing absolutely never happen, but using them as examples of how UB can "always" have unpredictable results seems off.
(As an aside, the existence of std::unreachable() is designed to make it easier for the compiler to optimise safely, i.e. it can emit trapping instructions such as ud2 in cases where it sees definite UB on some path, and omit those traps when the code has been marked as expectedly unreachable via std::unreachable(). I.e. there's a reasonable alternative for the compiler to optimise other than by relying on the mere presence of UB to identify paths that won't actually execute. For this reason, programmers need to be especially careful when using std::unreachable(); get it wrong and there really are no compiler-provided safety nets, unlike some other forms of UB where there may be. And already it should be clear that the linked post is vastly oversimplifying things).
Anything involving pointers is pretty prone to horrible consequences if there's UB - while a null pointer dereference typically does manifest in an immediate crash, things like buffer overrun and use-after-free can be much more subtle and much more insidious. This isn't news. It also applies mostly to a very limited subset of causes of UB. Throwing all UB in the same bucket is really misleading, at best. You'll occasionally see people (in reference to just how bad UB is) remark that even just having a missing newline at the end of a source file is UB. They always neglect to mention that it never, ever, actually causes the compiler to do anything significantly different than if the newline was present (beyond perhaps emitting a warning).
I also strongly suspect that the vast majority (I'm saying "vast majority", not all*) of actual runtime UB occurrences that don't involve dereference of invalid pointers are either harmless in effect or at least do exactly what you would expect (eg integer overflow typically wraps if that's what happens on the underlying architecture). The post is needlessly sensationalist, as was the preceding one.
*: notable exceptions include two classics, "the compiler ate my overflow check" and "I logged the value of an uninitialised variable but nothing happened" - even the latter one should cease to be an issue going forward with language changes that have been coming down the pipeline