Kees Cook Account Disabled for Suspected Malicous Activity
49 points by eyberg
49 points by eyberg
While right now it’s borderline offtopic, the post mortem of the incident will be absolute blast to digest.
They worked through it in the replies: TLDR is weird git rebasing in tandem with a script that inadvertently rewrote 39 commits instead of only 3. https://lore.kernel.org/all/20250601-pony-of-imaginary-chaos-eaa59e@lemur/
Look likes Linus is over-reacting again.
Does this look suspicious? Yes. Can this potentially be serious? Definitely. But this can all just be results from honest mistakes (e.g. rebases done wrong). It should be enough to just point this out and wait for Kees’ response. Going full “WTF Kees?!?!” is too much IMHO.
Is Linus’ response an over-reaction? Maybe. Can I blame him for reacting in that way, given how recent the xz debacle was? No, not really.
Immediately disabling access, and then figuring out what the hell has happened afterwards (so if it was malicious, the spread is contained) is completely reasonable IMO!
Those are separate actions. Immediately disabling access is good. The communication is terrible. Consider alternative:
Please disable Kees’ access until we understand where the change came from. Any recent submissions should be double-checked in case of compromise and keys need to be rolled. Kees, please send us details of your actions and any tools/environment details which may be relevant. Please preserve your current repository in case you can’t find the issue and others need to get involved in forensics.
Unless you really suspect someone of acting maliciously and have receipts, why would you ever want to compromise the shared trust, especially when you need their immediate help?
It reads to me like he suspects the account was compromised. In that case you never want to rely on previously established trust because there is a substantial risk that it is misplaced. Attackers often take advantage of our trust in others. It would be much better to have pre-arrange OOB ways for kernel.org folks to reestablish trust in the user’s control of the account without these public communications where display of mistrust is necessarily offending to at least one party.
There’s a difference between a “display of mistrust” and just yelling over email. You always have the opportunity to be respectful, even as you tell a maintainer you’re revoking their access until we all figure out what just happened.
Sure. Just to be super clear, I’m not saying that it is necessary to offend at least one party. I’m saying that in order to publicly display mistrust it will require offending at least one party. Avoiding that is one the benefits of doing it OOB (above and beyond the security benefits).
I’m surprised someone thinks so, but everyone’s different.
Why would mistrust require offensive language? I get expressing mistrust is going to offend someone. But I am but sure why does it require offensive language.
I think you’re misunderstanding my point. I’m not in any way saying that the language choice Linus made here is necessary. He displayed a lack of respect and civility. I have a hard time thinking of any situation where it is necessary. I was trying to point out that, before we even get to words choice, it is a problem that kernel.org apparently to handles account compromise indicators using the compromised communication channel. Take Linus out of the picture and imagine Greg-KH doing the same thing with more civil language. It would still be a poor security practice in general (for much the same reason we use MFA for everything). However since he would need to force behavior a malicious account holder would find difficult to do, it would require a display of mistrust, and that display would be offending (as in causing displeasure) to either the legitimate account holder, the attacker, or observers. That would be true even with more civil language.
Thanks for the clarification. I still don’t think displaying mistrust in what happened requires offense. This happens to me just about every day when I log into websites and they ask for additional proof because I’m using a proxy I haven’t used before. Yes, it is weird that I’m logging in from Malaysia when I just logged in from the Netherlands 15 minutes ago, I get why you want to ask.
In this case, I think the following could have happened:
If I commit code that appears malicious somewhere and I know I’m acting in good faith, I would not be offended to have someone flag these discrepancies. That gives me an opportunity to make sure I pushed what I thought I pushed and figure out how it got into a bad state. I’d be offended if something blew up in my commit, the maintainer LGTM’d it, and passed that garbage on to the end user, only to ask weeks or months alter about why the repo is not in a good state.
I completely understand that you’re saying this can all be done civally. The one place I disagree is that checking the validity of a push requires offense. It requires action (disabling the account, for example) and explanation, but none of that equals offense to me.
I think my language choices are just confusing the conversation so I’ll substitute. A public display of mistrust requires someone involved to directly or indirectly experience displeasure. It does not require disrespecting anyone involved. Here Linus did both. I think neither were required with better security practices. Given the observed security practices, displeasure was required while disrespect was not.
You seem to have actively maliciously modified your tree completely.
I will now refuse to pull anything from you until you explain what the f&*^ you have been up to, because this looks like you have been doing actively bad things.
You need to nuke that tree, and come up with a good explanation for this kind of shit.
Konstantin - please disable Kees’ account immediately until this is cleared up. Because this looks malicious.
I don’t see how any of that message can be read as “he suspects the account was compromised”, even after explanation, he constantly assumes the worst, it was only with a reproduction of the issue, (ironically unintentionally rooted in Konstantin’s code), did he relent.
Also just the fact that its “X is malicious, [Person who’s code indirectly/unintentionally caused the issue], disable X’s account” feels more a display of clueless rage than any well thought out action (especially as part of a gate keeped code base: All Linus had to do to avoid any issues is just not pull code, as far as I know, there was nothing that required immediate reaction time. If there was, you wouldn’t be telling the person who is malicious that you are onto them, would you?)
I don’t understand how this doesn’t lead to a public apology after publicly shaming and treating a person like shit, all for using tools that were expected to be used. Then again if Linus started doing the right thing now, he’d have to pause kernel development for the years of apologies he would have to write.
There’s trust as in social trust and trust as in certainty something’s not compromised. I meant the first kind - you want people to respond in their best capacity in this case. But that does not imply the second kind of trust as in their computers being clean.
Yeah… That email is not how you motivate people to calmly evaluate what happened in a potentially dangerous/time-sensitive scenario. If anyone was behaving like that in an incident with me, I’d ask them to step away and chill out.
So everyone had to work around a person known for rudeness again. Moving on…
There’s something to study here about the balance between security and community management. Assuming ill intent is a suitable basis for a security policy but not for personal tone in a public discussion.
I thought git-filter-repo is surgery for repo owners, not an everyday work branch tool. What’s it doing in such a script?
“Work around”? Linus was asked directly to pull the code and noticed something that to him suggested malicious intent.
Any normal git merge rebasing should re-write the committer. So to get the kinds of rewritten history that I saw, it almost has to be intentional. I don’t see how that has happened by mistake. (source)
So what are people “working around”, kernel safety?
Rudeness. The others on the thread were all putting in the emotional work to reestablish mutual trust so the project could keep an earnest contributor.
…noticed something that to him suggested malicious intent.
And yet by the end of the thread it was understood that there was no malicious intent.
https://git.kernel.org/pub/scm/linux/kernel/git/kees/linux.git/log/?h=hardening-v6.16-rc1-fix1
So, there’s something looking malicious somewhere here?
Yes the thing Linus points out is that this commit there is a commit that says to be made by Linus, but is not made by him: https://git.kernel.org/pub/scm/linux/kernel/git/kees/linux.git/commit/?h=hardening-v6.16-rc1-fix1&id=f8b59a0f90a2adfce5a9206ce5589ed0dc19543c
The real commit seem to exist, but has a different SHA hash: https://github.com/torvalds/linux/commit/9d230d500b0e5f7be863e2bf2386be5f80dd18aa
It might just be some git that went really wrong somehow, but I don’t think it is quite clear what happened yet.
edit: Kees Cook just posted the steps to reproduce the error and it seems to be caused by the b4 trailers tool
Steps: https://lore.kernel.org/all/202505312300.95D7D917@keescook/
b4 trailers: https://b4.docs.kernel.org/en/latest/contributor/trailers.html
This post violates the bright-line rule pushcx emphasized a few months ago against linking into the work and discussion spaces of projects, and leaving it up calls into question the existence and enforcement of the rule.
The rule is primarily concerned with people getting involved in the linked discussions, and that is much less likely for the kernel mailing list than a random open bugtracker. (hence why the mailing list archive links also weren’t banned from being submitted to the site)
(I still don’t think it was a good submission, and deleting it would have been appropriate IMHO)