Does using Rust really make your software safer?
51 points by folkertdev
51 points by folkertdev
Using almost anything other than K&R C makes your program safer.
FTA:
The tools used to write software don’t help prevent mistakes, and in fact make it hard to detect them if they have been made.
External input is implicitly trusted instead of being explicitly validated.
I’ll address the second item first.
We have ~350K LoC, all C. Everything builds with no warnings. We use multiple static analysis tools. We use fuzzers to test the portions of the code which face the network. We use assertions everywhere. We’ve written internal functions to sanity check data structures.
It might crash due to some internal oversight or state machine illogic. But we’re pretty darn sure it’s safe. We never trust input from anywhere. This is a design decision, and a habit.
For the second item, Rust is “better than C” because of its tooling. Rust prevents bad patterns which C allows. This makes the code safer. The Rust compilers also do more / better checks for safety than are done by the C compilers.
The result is a safer program. But nothing prevent a Rust program from reading the network (safely), connecting to an SQL server (safely), and then using the network data unchecked in SQL queries. We then have the “little Bobby Tables” issue.
In order for Rust to be safe_r_, it should also allow constructs which catch and complain about those kinds of errors, too.
Safety is not security.
Rust guarantees safety because that is objective and measurable, thus achievable.
No general purpose language can guarantee security because it depends on each user’s context/threat model. To use your network accessible DB example, it’s something that can be secure (say on localhost, or some GraphQL deployments).
It doesn’t help we all conflate safety and security a lot, I even caught myself writing “safe” instead of “secure” in the previous sentence.
That being said, Rust is also more secure than C because it eliminates most of the security issues due to unsafety, and it gives you better tools to model safety and security.
To illustrate that concretely, there are Rust libraries that statically prevent deadlocks for instance (ex: happylock), and some that allow safe transmuting/casting of compatible types (ex: internally in zerocopy, see also Safety Goggles for Alchemists, presentation recording, blog).
These are doable with current Rust, though would benefit from deeper integration. C does not have the building blocks required to achieve these kinds of guarantees.
But we’re pretty darn sure it’s safe.
“pretty darn sure” isn’t certainty, however, and these assumptions of safety are how so many security vulnerabilities leak through C codebases.
In order for Rust to be safe_r_, it should also allow constructs which catch and complain about those kinds of errors, too.
It does, though? You can encode far more information in the types than C and ensure correct construction of, for example, untrusted SQL queries from the network.
“pretty darn sure” isn’t certainty, however
Sure. I’m not going to claim it’s proven safe. I’m going to claim that the static analyzers / fuzzers checked the code paths, and there are no exploitable code paths found. I’ll also say that our APIs are structured so that the portions which deal with unsafe data are hidden behind an API which is simple and trivial to test. The rest of the code which needs to parse network data calls that API: “get me N bytes”. And that API either returns the data, or an error indicating that the data doesn’t exist in the packet.
As for the SQL bit, I’ll have to check that out (I don’t do Rust). In order for it to be truly safe, those checks would have to be done at run time. Because the set of untrusted characters varies by SQL server product, and by versions of SQL servers.
As for the SQL bit, I’ll have to check that out (I don’t do Rust). In order for it to be truly safe, those checks would have to be done at run time. Because the set of untrusted characters varies by SQL server product, and by versions of SQL servers.
The gist of the argument being made is that Rust’s type system gives you strong tools to teach the compiler to enforce correctness invariants. A very simple, very basic example from the standard library would be that, without using unsafe (which can be #![forbid(unsafe_code)] disallowed on a per-module or per-crate basis), you cannot bypass the UTF-8 validity check in String‘s constructor or the enforcement that any transformation of a String will uphold that validity invariant. Thus, in practical terms, it’s impossible in Safe Rust to have a String that isn’t guaranteed to be valid UTF-8.
(Yes, if you abuse unsafe, all bets are off… but all bets are also off if you start poking data into /proc/<PID>/mem or running your server with defective RAM. If you don’t use unsafe as directed, in the “as little as possible, only with proven cause, and with strong static and dynamic analysis” manner it’s intended for, then you’re only a step above pretending that you can use code that invokes Undefined Behaviour in C because you test what the compiler produces from it.)
In Rust, the type system has the power to do things like the typestate pattern (validation of correct traversal of a finite state machine, such as HTTP, at compile time). C++ can get close… but it’s more awkward and because everything is unsafe by default and C++ lacks a borrow checker and compile-time enforcement of destructive move semantics, it can’t prevent you from holding onto a moved-from state.
For SQL, your database abstraction would be using it to build an API that prevents little bobby tables attacks by only allowing SQL queries to be constructed in ways where any variable parts either use parameters (if the DB supports them in the position in question) or simulate using parameters by using contextually-appropriate, DB-engine-specific escaping under the hood. As with String, that would include any runtime server version checks necessary to ensure the invariants are upheld.
Just to be clear, since this is both a nitpicky and valid point, when people say things like “if you don’t use unsafe your code is guaranteed to do XYZ” this includes the implicit assumption that whatever safe API you access is free of error.
Just like how you don’t get guaranteed working code because it compiles, you don’t get guaranteed SQL-injection free code because it type checks. There’s always the possibility that your DB-engine escaping code has a bug where it fails to escape a certain sequence and someone can exploit that.
However, the advantage is that:
If you were using a language with a weaker type system, you make it possible to write db.run(get_string_from_user()). Rust’s type system (wielded properly) can make this a compiler error and require you to write db.run(sanitize(get_string_from_user()). You still have to trust sanitize and db.run, but note that in the language with a weaker type system, you additionally have to trust any of your own code (so all you have is the hopeful guarantee that the sanitizing code is correct).
So to reply to GP’s comment
As for the SQL bit, I’ll have to check that out (I don’t do Rust). In order for it to be truly safe, those checks would have to be done at run time.
The point is that the runtime checks are indeed enforced by the type system, but probably not in the way you’re thinking. The enforcement is by library convention (i.e. the library maintainers ensure that every possible way to create a sanitized String does the right checks and escaping). (to my knowledge) There isn’t anything in Rust yet that lets you write the type of Strings that don’t contain a backslash.
Just to be clear, since this is both a nitpicky and valid point, when people say things like “if you don’t use unsafe your code is guaranteed to do XYZ” this includes the implicit assumption that whatever safe API you access is free of error.
Perfectly fair… I just generally leave that as self-evident and not needing mention because, in my experience, it has tended to get brought up most by C or C++ programmers who feel threatened by Rust’s popularity and are trying to make a bad-faith argument along the lines of “Rust can be broken by the C code that it depends on and there’s no mature all-Rust platform all the way down to the motherboard firmware. Therefore, Rust is worthless. Therefore, I can ignore it and you should go away.”
Fundamentally, the relationship between Safe Rust and unsafe Rust is the same relationship as between a Rust project and its non-Rust dependencies. There will always be bits that need more or less human auditing but the more code you can trust the compiler to check the invariants on, the less space there is for mistakes to hide in.
The point is that the runtime checks are indeed enforced by the type system, but probably not in the way you’re thinking. The enforcement is by library convention
So… like C? :)
Just kidding.
To get the general gist, here is a non-rust example: JOOQ.
This let’s you write grammatically correct SQL’s via the type system alone, with even the parameters having the correct types.
In order for Rust to be safe_r_, it should also allow constructs which catch and complain about those kinds of errors, too.
As @halkcyon wrote, it already does. Much of Rust’s safety is implemented in libraries that use features of the language to ensure the library cannot be used unsafely, except with explicit annotations.
It’s easy to imagine a simple and safe API for SQL queries: ad-hoc queries or queries with ? placeholders are &'static str to guarantee they don’t contain runtime SQL, and more complicated queries are constructed with a query builder API that ensures the SQL is valid. Anything else is unsafe.
A real-world SQL API would need to be less of a pain in the arse to use, but a simple sketch is enough to show that a very basic Rust language feature (static lifetimes) can make a very basic SQL API safer than in other languages.
ad-hoc queries or queries with ? placeholders are &’static str to guarantee they don’t contain runtime SQL
Box::leak can return &’static str.
There are ways around that. For example, using macros that only accept literals, not identifiers.
Box::leak is also not currently const, so designing to require all queries to be compile-time constants would work.
Box::leak is also the kind of thing that jumps off the screen to a code reviewer.
This moves the claim from “statically detectable” to “reviewer detectable”, which could also be applied to memory unsafe languages.
The point is that the scope of “has to be detected by a reviewer” in a memory unsafe versus memory safe language is much larger and harder to recognize.
That function’s name “leak” screams out that you are doing something dangerous. C doesn’t have a “leak” function to warn you It will just leak the memory if you forget to free it.. The API described above makes it clear what type of string is acceptable and what type of invariants are expected of that string. Normal usage will just work. If someone uses “leak” to break the api invariant that is a deliberate choice not accidental and deliberate breakages are always going to be possible by
No language can fully fix those but it can give you better and safer tools to use by default and Rust does that.
Safe rust indeed provides better defaults to avoid UB, but that’s not what’s being pointed out:
Box::leak is still safe Rust. This is just highlighting that the misuse/correctness boundary here isnt just safe vs unsafe (which is the default worldview of memory unsafe languages). That observation is separate from whether having the safe/unsafe distinction is helpful or not.
Now now, you can’t use a mistake in my halfarsed sketch of an API to criticise Rust. You can use my mistake to criticise me: I have not written enough code in Rust to be an expert, but I bloviate nevertheless.
The general view of this issue (not specific to Rust) is how we as programmers can make safe interfaces out of unsafe parts. It applies to every layer of the stack, right down to digital logic as an abstraction over analog electronics. In order to make a safe abstraction you need to have a thorough understanding of the parts you are working with.
Obviously I didn’t understand 'static properly and left a hole in my abstraction. Whoops! Good thing it was caught by @bonzini in a design review before I perpetrated terrible code :-)
And you can’t use my mistake to excuse unsafe languages. It’s possible to use the same strategy (ensure SQL code is not assembled at runtime) in C and C++. It’s much harder to use that strategy in memory-safe languages such as Python or Java.
Going back to the general view, once you have designed a safe interface, how can you guarantee that the assumptions that you relied on remain true? Rust allows a programmer to put clearly delimited boundaries between safe and unsafe APIs. In C and C++ it’s ad-hoc: each library has its own conventions (if any, if you are lucky) of safe and unsafe usage, and libraries can’t easily get tools to enforce those conventions except for a few common limited-scope mistakes that have been enshrined as compiler attributes.
Rust’s “safety” is only about what it defines as memory safety: the ability to cause UB from memory operations. It doesnt protect you against leaks, panics, lack of sandboxing, deadlocks, race conditions, and other logic bugs. The protection also comes at a cost of prohibiting certain valid programs — Anything with non-linear lifetimes or access patterns (escaping callbacks, self-referential or intrusive data structures, interior mutability) requires either unsafe or runtime overhead to make it safe. So it’s not necessarily possible to make a desired API safe all the time.
Tangent aside, the comment on Box::leak is just highlighting that “it’s memory safe” isnt always a silver bullet (having to move back correctness checking from compiler to reviewer). It’s also meant to be interpreted in insolation of this specific instance, rather than “excusing” the benefit safe Rust provides; Not having to look for certain bug classes like data races and UAFs (if they can be encoded in the type system).
It doesnt protect you against leaks, panics, lack of sandboxing, deadlocks, race conditions, and other logic bugs
While rust doesn’t prevent 100% of such bugs, it certainly makes them less likely in many cases.
Take race conditions. In terms of memory safety, Rust and Java are equivalent here: Rust by virtue of statically preventing data races, Java by virtue of not having wide pointers and guaranteeing that accesses are atomic where it matters for safety.
Still, it is way harder to avoid race condition in Java than it is in Rust, because you also can use Rust type-system to clearly mark semantically non-thread-safe parts.
I think it’s fair to say that, while Rust doesn’t prevent race conditions, it protects against them better relative to Java.
leaks, panics, lack of sandboxing, deadlocks, race conditions, and other logic bugs
Ah, I must add an important clarification here. In this list, one item is not like the other: sandboxing needs to assume that the code itself is Byzantine! For sandboxing, Rust indeed provides zero guarantees, and it is a common misunderstanding that you can use Rust abstractions to safely execute untrusted Rust code. You can’t.
What forms of race conditions does Rust help best with?
For critical sections, Rust benefits from RAII which Java lacks. But that can be achieved without ownership/borrowing like with C#‘s using (var guard = ) scopes, or with a weaker version with ZIg’s defer.
For general TOCTOU bugs, idk if Rust helps much. Maybe with a transactional-memory system enforced by the type system? Although that’s rare to see provided.
One specific example I remember is storing a non-thread-safe data structure in thread-shared cache.
I made this exact bug twice, once in IntelliJ Rust and once in rust-analyzer. In IntelliJ, it took a week for vlad20012 to debug it, in rust-analyzer I got “Thing is not Sync” immediately.
Although JVM made this bug safe, rather than outright UB, Rust prevented it altogether.
leaks, panics, lack of sandboxing, deadlocks, race conditions, and other logic bugs.
Not by default (except race conditions) but Rust’s linear typing allow you to define such APIs that, once verified, can prevent logic bugs. The difference with other mainstream languages is that you can enforce a behaviour to the users of your API so that they have to use it in the correct way which is enforced by the compiler. So you just need to carefully provide the API then rest is handled by the type checker. The ArrayList example in https://blog.polybdenum.com/2023/03/05/fixing-the-next-10-000-aliasing-bugs.html is a good example of that.
By default it protects against data races, but not race conditions e.g. atomic.store(atomic.load() + 1) and channel.len() > 0 && channel.try_recv().is_some(). But “newtypes” (the affine version Rust provides) are indeed great for state validity and capabilities.
An imagined safe SQL API to me would imply that you parse and validate the incoming SQL text, to ensure it isn’t writing to arbitrary tables or dropping data, and if it’s doing destructive actions, to ensure parameterized queries and painting within the lines. I remember reading about something similar recently in the Rust project where they introduced a sql parser to ensure migrations were correct in their test suite.
I’m very far from the topic but this is what I understand:
I find the article very interesting because it actually tries to show in action what people are skeptical about.
Using almost anything other than K&R C makes your program safer.
That’s not saying much. C++ isn’t much safer, as it relies on conventions and a lot of knowledge for best practices and to avoid landmines, and you can still get burned when accessing memory.
And if we exclude C/C++ we’re in the land of garbage collected languages. Which are safe, since they have a runtime with much stronger safety guarantees, but then, they can’t be used in all the places dominated by C/C++, otherwise C/C++ would have seized to be an option a long time ago (at least for new projects).
Also, Rust is safer for SQL. For example:
Rust isn’t the only language that can do this. I have plenty of experience with Scala, for example, and Scala can manage to do most of that just fine (actually it feels a little more productive than Rust, TBH), but then Rust isn’t a GC-managed language, and that’s where its value lies.
Does the code use any libraries not written by you? If so have you vetted the code in those libraries as well? These are the first questions I think of when I hear people talk about their safe C codebases. Which is not a knock on your team. You are clearly doing your best to be as competent as you can with C. And it’s quite possible that a host of factors restrict your choices here and C is the best choice.
But in the general case Rust prevents the kinds of error the article talks about and I’m fairly confident that the people who wrote the code for that OS are also trying their best to be as competent as they can with C. The point in the article is a good one. That specific category of error is prevented by rust. And not just that error but also additional categories of error are also rarer in practice for this experiment.
Does the code use any libraries not written by you? If so have you vetted the code in those libraries as well?
That’s not relevant. Either the code is fine, in which case there’s no issue. Or the code isn’t fine, in which case it’s generally someone else’s problem. While we might be affected, we are not responsible. Plus, no one is reasonably expect us to vet everything. “OMG, your application runs on Linux, did you vet the whole kernel?”
No.
You are clearly doing your best to be as competent as you can with C.
Well, that’s refreshingly condescending.
My point wasn’t that the C code was perfect. My point was that it was as safe as we know how to make it. It’s safe enough that we are very confident there are no externally exploitable issues with it.
And it’s quite possible that a host of factors restrict your choices here and C is the best choice.
25+ years of legacy code, and no one is paying us to rewrite it in Rust. So for our purposes, Rust might as well not exist.
While we might be affected, we are not responsible.
This might just be that I’m misunderstanding your phrasing, but I don’t believe that’s typically true, either morally or practically. If I ship someone a buggy application or device or whatever, and the root cause lies in a library that I imported, that’s not really relevant to the person I shipped my code to. They got buggy code that didn’t do what they wanted. I am responsible for ensuring that doesn’t happen. Therefore I am responsible for fixing the problem.
The fix might well be that I wait for the library to be updated and then release again with the new version. But if the issue is serious and the library won’t be updated quickly enough, I might need to either patch the library myself or replace the dependency altogether. Either way, whether I’m the one fixing the problem or not, it’s my responsibility to ensure that it does get fixed, because I’m the one who shipped code with this dependency.
If I ship someone a buggy application or device or whatever, and the root cause lies in a library that I imported, that’s not really relevant to the person I shipped my code to.
“Open Source”.
I didn’t ship you code. Debian / Ubuntu / RedHat shipped you code. If you downloaded source from github, or even a release “tar” file, I didn’t ship you any external libraries. They were already on your system.
We have no moral obligation to fix code that we didn’t ship.
I agree that we can sometimes work around issues in other libraries. In FreeRADIUS, we check for versions of OpenSSL with known issues, and complain about them. But we can’t do much else. Ultimately, a vulnerable OpenSSL was already on your system when you installed our code. And there is really very, very little that we can do about it.
The situation would be different if we were shipping a “closed box” product, of course.
Well, that’s refreshingly condescending.
For what it’s worth I didn’t intend that comment to be condescending. It was supposed to be a compliment. Many people writing C don’t go to the lengths that your team is going so it’s a good thing.
Let’s not forget that Rust adds safety guarantees on top of everything else you can do.
So saying that Rust could be unsafe if written badly, but C can be safe if you “just” have expert-level programmers diligently testing and fuzzing everything isn’t saying anything. You can use your expertise and diligence to write even better Rust, and you can use the safety net to tackle even more ambitious problems. The safe subset of Rust and its more expressive type system can give guarantees out of the box beyond what C static analyzers can do. Rust supports fuzzers and sanitizers too.
I think this is interesting, but to be more rigorous, you would need to ask some colleagues to write the code in C.
Developers today are somewhat more conditioned to think about network attached software as subject to exploits, so you have to ask “would a modern developer write the same vulnerabilities in C?”
I hold the boring opinion that Rust has real safety benefits over C, even with modern development practices, even with sanitizers and everything else that a modern C developer might use (and the same for C++). However, if you’re going to produce a useful test, you have to actually test that claim.
From the article: “We’ve also taken a stab at a proper C implementation (by an experienced C programmer), and even armed with the knowledge we gained through this experiment, it still took at least three times the amount of time to get a secure version.”
Thanks for that, I actually missed it, as it’s below the table of results–guess I read too quickly.
But also, …what does that mean? Are they saying the C programmer got good results but it took longer? Interesting, but not the claim at hand. Or are they saying the C programmer was briefed on the issue? I don’t think that makes sense as a test.
Well, lol, DNS name decompression is a perpetual source of slapstick bugs. Comparable to ASN.1 :-) I agree with the point of the blog post, and it’s a great example. Since I know a bit about DNS names, I have some pedantic notes:
A real-world DNS name decompressor needs to return the offset of the next byte in the message, because the next offset is not necessarily related to the length of the decompressed name. Without it the caller can’t parse the rest of the message.
The Rust function does two things, uncompressing the name and replacing label lengths with dots. You can’t convert a DNS name to legible text this way, because names can contain arbitrary bytes: for practical safety it’s necessary to escape weird bytes. This conversion should be a separate function from name decompression.
DNS name decompression functions often build a vector of offsets to the start of each label in the decompressed name (when they return the decompressed name in wire format). This is useful when working with suffixes of the name for things like subdomain checks.
Checking for “prior occurance” [sic] is a good way to avoid compression pointer loops, but many prominent DNS implementations put a limit on a loop counter. The two implementations with one amber square are fine.
This Rust implementation does not allow names to overlap, which is probably a good thing, but it’s stricter than most.
Having written code to decode DNS packets it would have been nice to see a link to the stress test data. I do a loop check for decoding DNS packets, but missed that a pointer shouldn’t point to another pointer, and that pointers should always point backwards. I do check that I don’t read outside the original packet data, nor does my code allocate any memory (it requires the user to pass in a sufficiently sized block and if not, returns an error). I do check the compression bits, and return an error if they’re either 10 or 01.
I should have included a link to https://dotat.at/@/2022-11-17-dns-decompress.html which has several more points of more or less obscure and/or painfully acquired knowledge.
Pointers-to-pointers are not worth checking as a special case, in fact I don’t recall seeing any code that rejects that specific anomaly. Requiring a “prior occurance” or limiting the number of pointers works well enough to eliminate the bad effects of pointers-to-pointers.
Top bits 01 and 10 were proposed as EDNS extended label types in the late 1990s, related to A6 records, bitstring labels in the reverse DNS, and the idea that IPv6 could have easy network renumbering with help from the DNS. One of those second system syndrome ideas that turned out to require a synchronized forklift upgrade of the whole Internet, which died when the IETF worked out that the upgrade strategies of the 1980s had been broken by several orders of magnitude of growth.
I agree with the philosophy here, and I agree that Rust (and most modern languages) handles this in a way that is much less error prone than C. But the methodology is just totally wrong.
Asking 4 developers to build something in Rust and noting that those implementations have very few errors does NOT prove that writing in Rust is safer than writing in C. Randomly assigning those developers to write in either C or Rust (assuming all participants knew both languages reasonably well) would be the way to test that.
It’s a cool experiment. If someone repeats this kind of thing then it would be neat if they asked some similarly experienced colleagues to write solutions in C.
Cos otherwise it could just be that your engineers tend to do a good job on this kind of problem for whatever reason.
then it would be neat if they asked some similarly experienced colleagues to write solutions in C.
https://lobste.rs/s/zjnc6g/does_using_rust_really_make_your_software#c_o8wgvw
Is it common to solve such tasks in plain C? I would rather expect that any larger C project would develop its own (or adopt some existing library) set of functions for repeated tasks like parsing binary data, working with text strings, binary streams and buffers etc.
I stumbled upon similar bugs in C codebases too, but it was rather smaller projects.
This particular C is not an example of good C, obviously :-) I’ve written before that the first thing I look at in DNS code is its name decompression, because that’s a very quick way to find out if the author has been taking care and paying attention to past mistakes. This example is far worse than most.
In my experience, DNS code in C often has safe wrappers for accessing buffers and DNS names. However, name decompression is very weird as binary formats go, and it tends to be performance-sensitive, so it often bypasses the safe APIs. Fortunately it’s relatively easy to thoroughly fuzz compared to other binary formats.