Be alert: targeted attacks on prominent Rustaceans
73 points by itamarst
73 points by itamarst
recently, systemd-the-organisation looked at introducing rust as a part of systemd-the-init. luca boccassi, one of the senior maintainers[1] of systemd-the-both, raised some good points about it. i've stripped what i'll describe as somewhat extreme exaggerations around cargo and crates.io:
[...] you have thousands upon thousands of recursive dependencies, and one getting typo-squatted or taken over is all that is needed for massive supply chain attacks [...] and all of those thousands of recursive dependencies not only brings in runtime code, but even arbitrary build time code (build.rs) that stealthly runs as your user on your machine during compilation, and thus can access your browser cookies and your ssh keys and whatever else. This not only puts final users at risk, but all developers too.
i'd recommend largely ignoring the rest of the thread because it devolves into the usual byte-wasting seething from both sides ("it's a toy language for toy people", "how very dare you", "your mother was a hamster", and so forth).
i've been thinking a bit lately about (further) securing one's development environment. every now and then i'm reminded of the build.rs file[2] and how normalised it is, even to me, and it makes me shiver a bit. all the way down there, some sub-sub-sub dependency can be compromised and the knock-on effect can be utterly enormous. this process can be (and is by default in every case i'm aware of) launched by simply opening your text editor with a lsp server attached, because the second thing it does is run cargo check.
the library situation in rust is both an incredible strength (you can easily lean on the shoulders of a billion giants) and an incredible vulnerability (any one of those giants can, through intention, negligence, or simply unfortunate circumstances, cause ridiculous damage). likewise, "build scripts as a function built into the language implementation" is extremely powerful both for good and for bad.
docs.rs, which provides a hosting platform for rust's //! doc comments, approaches this by running the build inside a heavily restricted container. rubydoc.info recently suffered a mass attack from openai bots exploiting this same sort of thing[3] (post courtesy of our own ~simonw).
yes, makefiles and such can execute arbitrary code too, but whataboutism pointing to things invented before most people on this website were born isn't a shining argument that the current way is "just fine". i think the problem has been exacerbated by the general proliferation of the idea that you just $toolchain add $dependency, occasionally run $toolchain update $dependency if your github bot ci isn't doing that automatically for you (it really shouldn't, but sadly usually does), and otherwise never again think about it in any capacity until oops, another one.
let's do some quick maths on one of my own "sandbox projects", a very small web application that backs onto a postgres db:
# first five lines in this particular file are
# package metadata.
$ tail -n +6 Cargo.toml | wc -l
25
$ cargo tree --prefix none | cut -d ' ' -f1 | sort -u | wc -l
239
yum. 239 unique transient dependencies, excluding any that are the same dependency but are semver-incompatible, as cargo collapses semver-compatible dependencies into one if their version ranges overlap.
i'm not good enough at programming programming languages to properly identify a better path, but this isn't really ideal. some people might point to go generate as an example of something with a smaller blast radius[4], but build.rs files are largely akin to makefiles vs just "generating code", which is predominantly done in rust via actual macros. i'm unsure there is a way "out" at all, really, especially as both programs and programming have become more and more complex. i'm not entirely convinced the whole "dependency cooldowns" movement is the right direction, because that introduces another set of problems that are then insurmountable with that model. it would be, at best, a sidegrade.
in the meantime, i make use of a combination of "i use nix btw" and bubblewrap. i might look into systemd-nspawn when i get time, as contrary to the bubblewrap readme it does support rootless mode.
go generate as something of a(n intentionally) primitive macro system. even if it were more, go generate doesn't traverse dependencies, so you won't be running that command and immediately calling the //go:generate blocks in every sub-dependency on the planet.I’d recommend posting the output of the dependency analysis somewhere—they can be pretty subtle and I don’t think the count of dependencies is always that helpful.
For instance, sometimes a single project will distribute 10 or 20 crates because they make the crates modular.
Beyond modularity, they also speed up the build process, sometimes significantly. Naively counting dependencies is a terrible metric to base opinions on; the alternative is: do you want to write the hundreds of thousands of lines of code you're depending on instead?
Outside of your hyperbole of "hundreds of thousands of lines" I think the answer should be mostly yes. Systemd is a low level enough project where the same rules as having rust in the kernel should apply.
Do you think that's hyperbole when we're discussing dependencies in the hundreds? I just cloned the systemd repo, it contains over 1M lines of just C code.
The systemd repository contains quite a large number of things, a lot of them implemented from first principles in order to avoid failure cases in existing solutions or libraries (and probably ending up discovering others :D). So that's the main reason it's that large. And I think it also clarifies that for low level code it's wiser for a project to do away with depending on hundreds of thousands of lines of someone else's code. (This is just an armchair observation, and, to the best of my knowledge, is not endorsed by the systemd devs). So, speaking for myself, the answer would still be "yes".
So... This is a mess but mostly it is not a PL problem. It is a build system problem with some ramifications into PL/compilers.
But mostly it is a build system problem. And largely a build systems interactions with other build systems problems.
And the sad reality is that there is a lot of work to be done there. But there are no easy solutions.
Especially as nearly everything has to deal with the need for a C build system, which is a massively unsolved problem.
Without these needs, you probably would not need a build.rs. but nearly all solutions to replace it with something less free form breakdown the moment you need to handle C.
And you need to handle C.
Ironically, the C ecosystem fares really well here, since I can just use my distro's package manager to manage dependencies.
Instead of just blindly downloading the latest version the package author has uploaded, it's subject to my distro's policies - and I even get the choice of which distro's policies I prefer! Debian Stable keeps packages around for ages, only backporting security patches, and even e.g. Gentoo (AFAIK) usually waits for around a month before stabilizing a package.
I try to install libraries for other languages via my distro too, but the feasibility of that varies from language to language.
The C ecosystem only fares as well as it does because it did so badly upstream that the distros had to develop their own C package managers... and the assumptions they make about dynamic linking and build processes do very much make them C package managers. (See this for example.)
...plus, as https://wiki.alopex.li/LetsBeRealAboutDependencies lays out (especially the "Gotta go deeper" section), they only fare well in the "package maintainers generally aren't the same people as upstreams and that's an extra speed bump" sense.
For example, bespoke implementations and header-only libraries aside, there's no one auditing the package maintainers to evaluate how thoroughly they're auditing the updates to what they package.
every now and then i'm reminded of the build.rs file[2] and how normalised it is, even to me, and it makes me shiver a bit.
I don't think there's anything wrong with running arbitrary code during the build. If you're a developer, you're almost certainly building the package with the intention to run it right after. If you're a distro maintainer who's only building the binary to distribute it elsewhere, you should be doing sandboxed builds anyways, instead of just relying on the build system to keep you safe.
The real issue here is how the LSP runs arbitrary code. This is a horrifying choice. That being seemingly acceptable for an LSP server has honestly turned me off the entire LSP ecosystem.
I don't think there's anything wrong with running arbitrary code during the build. If you're a developer, you're almost certainly building the package with the intention to run it right after.
That's pre-WebAssembly thinking. I'm migrating as much code as possible to WASIp2 plugins and, for those crates, I really wish there was a way I could set a policy where Cargo would refuse to build if it would run a procedural macro or build.rs without doing it inside some kind of sandbox like a WebAssembly runtime.
It's sort of like how, for another project of mine, I have various previewers based on things like PDF.js and CHMate where they run purely client-side in an isolated origin with a strict Content Security Policy, and I'm careful to not use something like NPM to retrieve them... just downloading a zip or tar archive, unpacking it to a folder under static_preview using an up-to-date archive tool from my distro repos, and adjusting the definitions in the server code to hook the relevant mimetypes up to the relevant paths.
Wow.
I'm not in any way prominent, but I'm glad I'm already doing things which would make me think twice about something like a "missing codec" message, and treating cold outreach as inherently spammy/scammy.
Now to expedite retrofitting my development setup so my only Rust toolchains exist inside Docker containers so I can trust that I don't accidentally have rust-analyzer execute a compromised package unconstrained. (I'm already working to migrate as many transitive dependencies as possible into WASIp2-constrained plugins with APIs designed in pursuit of nanoprocess sandboxing principles.)
This is legitimately scary.
Any piece of software that depends on open source (which is almost every piece of software) has a network of human beings who are potential attack vectors - everyone with publishing rights to any of the packages in the dependency network for that software.
I guess our best defense right now is dependency cooldowns - giving new package releases a few days before upgrading to them, in the hope that supply chain risks will be spotted by someone else.
Dependency Cooldowns in Rust: https://cooldowns.dev/#cargo (And other programming languages)
We've had a few of the reverse - we're interviewing someone who is clearly using a live deep fake and LLM assisted coding during a technical challenge. The recruiter who sat in was also deep faked. This is also a known DPRK thing.