GitHub Actions leaking secrets when Miri output is cached
8 points by peter
8 points by peter
The Rust Security Response Team was notified that Miri stores all environment variables to
target/, allowing secrets to persist in caches.
Then proceed to blame Github Actions.
NB: All CI platform that provide caching and would naturally cache the build folder that is target/ would have this security hole. The culprit is cargo miri, not Github Actions.
Then proceed to blame Github Actions.
Yes, the root cause here is that it is so easy to write insecure GHA pipelines without noticing.
If you
then you're doing it wrong. To make this secure, everything running in the environment would have to be carefully audited to not preserve the environment, without even knowing which parts of the environment are sensitive. That's a completely unrealistic expectation. In taint tracking terms, when you ran the program with the secret in the environment, that program's execution is now tainted with the secret, and so is all of its output.
Lucky enough, this is rare -- our ecosystem scan found only a handful of cases where people make secrets globally available to all steps of a pipeline. But you get no indication from GHA that this is a terrible idea, and that is a problem.
This applies even for stdout/stderr: if you put a token into a URL, this can easily end up being printed somewhere. CI providers know this, that's why they filter configured secrets and remove them from the output. They are, basically, removing the taint. (In principle that could be used to actually leak the secret, but with the typical size of secret tokens that is not practical.) But for all other output, this is your responsibility, the CI cannot do it for you.
Now, GHA does provide the knobs to do this properly: you can control which parts of which jobs the secrets are actually exposed to. But it's easy to forget to use these knobs, and that's a fundamental problem of GHA.
(I am a Miri maintainer and author of the patch mentioned in the blog post. I view this patch as defense in depth, similar to stack canaries and ASLR: it's always a bug if you're relying on this 2nd layer of defense, but it's still useful to have.)
The point is that it's not specific to GHA. Jenkins, Gitlab CI, CircleCI, etc... All have the same "vulnerability" of "if you leak secrets, secrets are leaked".
I was taught in the 1990s by unix greybeards that environment variables must not be used for sensitive data because they are effectively public: many flavours of unix expose them to all processes via ps, and there are many ways for them to leak much further. Hence programs of that era use files or named pipes (which go through the kernel’s access controls) to provide access to secrets: X, ssh, gpg, etc. So it has been weird to see Heroku and so many subsequent designs depending on such an unsafe mechanism for security-critical information.
However, I think /proc data being world-readable is a Linux-y thing which I've often seen as somewhat undesirable (I don't like other users in a shared system seeing which files I'm opening).
IIRC, for a while I was working on OVH VPS that had some kernel patch applied that made a lot of /proc non-world readable. It was confusing, but perhaps it's the right idea?
(No matter what, the current situation is that /proc is world-readable and we should behave accordingly.)
I usually pass through environment variables the path to a file containing the secret, and that path is on a tmpfs (for Docker, it's usually in /secrets or /run/secrets) and injected via a Secret Manager such as Vault.
What I don't get is: as I understood from their description, the way to exfiltrate the secret from the cache is by force-pushing code that reads them while it runs in CI (and then prints to logs, sends over the network, or something).
But doesn't every CI run have those env variables set anyway, cached or not? (That's how miri picked them up to write them to the cache, after all?) So, can't the force-pushed code just read the env vars directly? As I understand, the only additional thing they get by reading the miri cache is env values from previous runs? Fair enough, this is an information leak (from an older to a newer CI run) that should be fixed; but usually the current env vars are most likely to be valuable. Thus almost every CI setup that cares about this leak (i.e. has sensitive variables) will continue to expose them?
I think the idea is that you, say, publish a release, and that workflow includes running miri, then someone can create a pr and get the same cache as your release workflow.
Aha, makes sense (I wasn't aware different workflows can share the same cache, interesting?).
PS. if that's true, then can't it also happen that the cache is filled in the PR workflow and executed in the release workflow (thus again giving the attacker access to the release secrets by way of positing hijacked binaries in the cache)?
From the post:
Typical setups allow CI runs on main (and other branches) to write to cache, and PRs can only read from cache (preventing cache poisoning).