This is not sustainable
89 points by binjip978
89 points by binjip978
Oh, you have a detailed package inventory or, dare I say it, SBOMs? That's really nice. Do show me where that lists the various vendored or hard-coded CA trust bundles. Yeah, I didn't think so.3
Interesting the Author linked to nixos issue here, if you follow that link you end up at nix-cacert-scan, a tool that scans your system nix closure for packages that contain TLS certs, and from there you can find out why any package includes it and add overlays to patch any such packages, even add automation to your pipeline to detect compromised certs throughout your whole system.
I think this is one of the strengths of nixos, it is a pity it isn't more focused on security, missing things like signed commits on nixpkgs, built-in kernel hardening, musl builds. nixpkgs is also so huge and interweaved, it would be nice if a secure core was separated out (before someone commercial does it.)
The vast majority of work on Nixpkgs is still done by volunteers, so any work orthogonal to the existing giant pile would have to be done by someone with the time and inclination. Unless companies with money and incentive do the obvious thing, and hire people to look into this full time, in which case they could probably have just about any security feature they want.
That said, I think NixOS is still not used very much in companies with lots of money. They tend to move really slowly, and probably won't listen to a handful of sysadmins screaming about how much of a paradigm shift Nix and NixOS are.
it would be nice if a secure core was separated out (before someone commercial does it.)
it is a pity it isn't more focused on security
Agreed. I'm running the latest revision of the currently supported stable version of NixOS (26.05), and there are still 5 packages in my closure that trust eMudhra certs, according to the tool you linked. Each of those packages has been updated upstream to fix this, but I guess the nixpkgs maintainers haven't backported the fixes (yet?). I have a lot of respect for the nixpkgs team, but I also wish my uv installation wasn't trivially MITMable.
For uv in particular: https://github.com/NixOS/nixpkgs/pull/579622 is currently making its way to 26.05 (and in fact, is already in 26.05-small).
The updates couldn't be backported because of backcompat concerns (and if the above PR weren't merged instead, I imagine/hope we would have just written our own patch to bump the roots).
Thank you! I really appreciate all the work you and the nixpkgs team do, I hope I didn't come off as ungrateful <3
ISTM that the right way to deal with automated vuln discovery in big complex stacks is to replace the big complex stacks with smaller simpler stacks, accepting that this will often mean significant loss of functionality.
And keep repeating that until some minimal threshold is reached, of tools so simple that they can just about do some essential minimal subset of the original function, while being so tiny and so simple that they are provably correct.
Which sounds good to me, honestly.
That's what I was thinking too. Compartmentalize things into smaller pieces that are easier to reason about. Always keep on latest release and update rigorously, prioritizing "up-to-date" over "known to work reliably". This will greatly sacrifice reliability for security, but given enough iterations over stability/fallbacks and graceful service degradation, this would eventually reduce the need for what the author describes as their main overhead: Contextualizing a security bug with your specific application stack. (See also: cattle vs pets)
How small does a program have to be before you guarantee there are no bugs in it? What if it's not a program but a bunch of interacting programs? Talking to a bunch of other programs on other systems? Operating on fallible hardware that has its own bugs in it?
Simplicity is good for making machines more resilient, but simplicity alone is not enough. The main thing I learned from brushing against safety-critical software is that you don't prove the program correct, you prove the system correct.
How small does a program have to be before you guarantee there are no bugs in it?
A very hard question to which I have no good answer. But far greater better minds than mine have made this point.
« There are two ways of constructing software: one way is to make it so simple that there are obviously no deficiencies, and the other way is to make it so complicated that there are no obvious deficiencies. The first method is far more difficult. »
— C. A. R. Hoare
How small does a program have to be before you guarantee there are no bugs in it?
Seemingly, it has to be absurdly small.
replace the big complex stacks with smaller simpler stacks, accepting that this will often mean significant loss of functionality.
I will believe in practical relevance of such strategy when EU (let's be honest, who else) actually outlaws either at least one use case of DOCX across governments, or at least one JavaScript feature (for either government or banking websites).
Which, to be clear, I will very much welcome.
Enjoyed this piece a lot. I find I'm less pessimistic than the author, even though I've been the one tasked with turning over basically every stone looking for where the eMudhra certificates were still hiding. Something about this reminds me a little of the afl-fuzz/heartbleed era. There's a survivor bias to a lot of the software we use and there's a huge surge of novel ways to test its fitness emerging right now, just like in 2014. A lot of it got sick back then and didn't die. A lot of it is getting sick again, and the stuff that survives will be tougher. I'm expecting this to quiet down after a while, though I suspect the noise floor has been raised permanently.
Your only option is to limit the blast radius. Assume compromise, detect exploitation, limit exposure, lateral movement, privilege escalation, etc. If that sounds like an old hat, well, yes, that's what sane infosec practitioners have been asking for funding for for decades. (We used to call it "Zero Trust", before that became an empty marketing term.) But that funding instead goes to the next singing and dancing
Isn’t this just a definition of a sane security process and “funding” is a salary of professional? To build project architecture in a finely layered fashion. Won’t save us from everything, but will limit the blast radius indeed. Or does “funding” refer to the development of the turnkey solutions for this?
IMO this situation has been decades in the making, AI and increased scrutiny towards software has just accelerated it.
I was about to add, but i think you stated it best!
Over many years now, not only related to cybersecurity, but at the orgs that i have worked at, there is increasing pressure to do more with less (as in, less people, less resources, less budgets, less time)...and that was BEFORE AI...but now leaders of all orgs think that simply adding AI into the mix will "fix everything"...but it merely adds gasoline to fires that either already exist or are about to ignite. Now, before someone replies "this is all because of capitalism"....i'm not so sure i agree. While, yes many orgs i have worked at think or better react this way, i have also worked at companies that (while still not perfect), take a smart and deliberate approach, and plan for future things...maybe not all of them forsaw AI or AI's impact...but i do know for sure that orgs out there do exist, do operate in a capitalisit environment, do succeed, but do not operate half-assed....i just no lomnger work for them (and regret leaving one of them lol).
So, yeah, the problems we face now - both in cybersecurity and beyond - took long to manifest, and surely won';t be fast to address/remedy.
AI companies are incentivized to continue the onslaught, with the most benevolent interpretation of mass disclosures of findings being a canonical example of "verschlimmbessern", with less friendly interpretations observing that it's profitable for a company to create the disease for which they're selling you to treatment (and never the cure).
I’m not in security, but I sort of have this (naive?) feeling about the software security industry too. They put a ton of effort into finding exploits, on the rationale that if they didn’t do it, the bad guys would. I’ve always been skeptical about that — most criminals just aren’t that smart.
You don’t need most criminals to be that smart. If there’s enough money locked up behind an exploit chain it only takes one person to figure it out and distribute the exploit to those who will use it.
Plus major nation states do employ vulnerability hunters, and it’s alleged that some of them are reducing those exploits to practice.
At a certain point either the firehose starts to dwindle or we accept that we can’t whackamole our way into security. I don’t think we’re empirically far enough into the era of intense ai driven whackamoling to say that it won’t dry up, but everyone needs to think about what time horizon they’ll wait for before they try something more systematic and systemic.
I'm quite surprised that there aren't more things of organised crime selling services to petty criminals. For example, there are a bunch of ways you can use social media or compromised IoT devices to get a list of houses where the owners are away. Selling that information for $20 to petty criminals in another country seems like a fairly low risk thing people in a bunch of countries to do.