Dependency cooldowns are unfair; we should use phased rollouts instead
18 points by quad
18 points by quad
Cooldowns work against fast-acting supply-chain attacks. But they have an awkward property: they implicitly rely on someone else installing first.
This is not true. Cooldowns work based on security scanners being incentivized (successful detection gets them marketing) to detect and find these attacks. (Not a fan of calling them supply-chain attacks because open source software without a contract in place nothing to do with supply chains.)
Not a fan of calling them supply-chain attacks because open source software without a contract in place nothing to do with supply chains.
Same tbh. I'm just trying to match the language of the linked blog posts.
successful detection gets them marketing
Shout out: Recent RubyGems attacks were found by Mend.io
Dependency cooldowns would work against the present spate of supply chain attacks because they tend to be blunt smash and grabs that are detectable by automated scans within hours to days. They don't rely on someone else getting infected first.
I removed a section about how that also selects for more xz style attacks. I thought it detracted from the core point. Should I add it back?
I was under the impression that the cooldown period has nothing to do with “who installs it first” and instead is entirely aimed at giving the maintainers an effective window to ring the warning bells that their credentials were compromised. I guess in theory it also does what others are stating in giving people time to inspect the release as well.
As others have pointed out, this proposal misses the point of dependency cooldowns.
Imagine, as a hypothetical, that a language's package index implemented a new workflow where each newly-released package went into a "held for review" state, and only security scanners could see it in that state. Then, if and when those scanners gave the all-clear, it could move into an "installable by the public" state. Or, if the scanners find a problem, it stays non-installable and the community is alerted to the possible compromise.
A universal dependency cooldown period is a different way of trying to achieve the same practical outcome: if everyone agrees to a cooldown of a few days, then all newly-uploaded packages are, effectively, only visible to security scanners during the cooldown period, and can be yanked from installability if the scanners find an issue during that period.
Security scanner vendors get press for finding problems in popular packages in popular ecosystems. I am skeptical of relying on their long-term goodwill. But, it'll be fun to see how it all plays out! Who knows, maybe Claude Mythos will save / doom us all.
That’s the thing: you don’t have to rely on their goodwill! You have to rely on their greed, and the fact that they’re directly financially incentivized to discover malware rapidly.
Working for leads is different than working for money. Vendors are also incentivized to find fatter margins.
Every other public resource in the ecosystem (registries, CI, vuln dbs, foundations, etc.) are all under-resourced. So, yeah, I'm skeptical of the long-term sustainability.
If the existing players stop doing scans, new ones will step in. This is the theory of capitalism, that rational self-interested actors compete in a way that benefits greater society.
The reality is quite different most of the time, of course, but the stars are aligned in this case: the incentives are financially rewarding, the barrier to entry is relatively low, verification is objective, and you can genuinely compete based on the quality of your static/dynamic/LLM harnesses.
Of course, things may not stay like this forever, and it's worth keeping an eye on.
To be honest, registries, vuln dbs, foundations, and even CI need continuity (and all but CI need global coordination to do well).
The cryptocoin-mining-like payoff structure (you scan until you get lucky to find stuff), and usefulness of a single well-described discovery with no continuity — those things do make chances of having stuff done over time better.
Like I said above: popular packages in popular ecosystems. I question if the incentives work in favour for The Rest Of Us.
Given that Common Lisp ecosystem used asdf-install (with an anonymous wiki as a registry, yes that bad, I don't think anyone has ever found anything fishy in the history though) for years, maybe incentives for scanning and for attacks are correlated…
Less popular packages in popular ecosystems are cheaper to attack but probably get some benefit from running the scans on the long tail having the marginal cost proportional to the scanning frequency.
Less popular ecosystems could get things that scale onto them automatically, if people do VM+IDS tests, and if they include exotic stuff just to have a definitely blackbox case for calibration. Probably not.
And instead you are arguing it's more sustainable that everyone tests in production on the same timeline?
A lot of conversations seems to assume a very particular (although unfortunately common) pipeline which conflates development, dependency upgrades reviews, and deployment. For example, it's completely reasonable to have automated renovate-style notify about and checksum (but not run) fresh versions as they drop upstream, start reviewing (with at least some human involvement and attention), but only roll them out days or weeks later (unless urgency is motivated for particular fixes).
This was always the case. Are you also finding Debian users to be selfish and unfair for not following the Arch Linux approach of always tracking bleeding edge?
More irresponsible would be exposing your users and systems to new changes you haven't vetted yourself. I would agree with you that more eyes on code earlier is a good thing. That doesn't mean all our deployments should be running the same versions from day 1.
Dependents need to pay attention and manage their codebases and dependencies proactively. This kind of cooldown as part of tooling can be helpful but is not an actual answer or solution by itself. Even for the same software, can be appropriate with different configuration values for different parts of the lifecycle. Focusing on it without talking about the bigger picture misses the mark IMO.
Are you also finding Debian users to be selfish and unfair for not following the Arch Linux approach of always tracking bleeding edge?
Debian is a project of generally good actors curating a set of packages. Relying on dependency cooldowns and automated security scanners is much closer to npm than to Debian on the spectrum of relying on other people for maintenance.
The vendors in question are private, not public, and a good number of them are "unicorns." But I think it's right to question the long-term sustainability; I don't think anything about the adoption of OSS in corporate environments could be called sustainable :-)
The point of the system is that there is no long-term!
You want contract leads next month, you want positive press this month. You want positive press, you write something that can be evaluated without goodwill entering into the equation. You pivot from all this, well, let's see who wants a piece of your former fame as a highly competent audit consultancy. (Some regulation will demand audits one way or another)
Cooldowns work against fast-acting supply-chain attacks. But they have an awkward property: they implicitly rely on someone else installing first. In practice, that “someone else” means Asia-Pacific
What? How? This makes no sense. Why would it land on a specific time zone?
It isn’t the timezone, it’s who is doing things at 00:00 UTC. That is mostly APAC people and cronjobs everywhere. Also, Monday.
OK so why not argue against tying it to 00:00 UTC? That's a dumb policy, and obviously no one should do that. But that's not the argument made in the post.
Because I’m bad at writing and I assumed that 00:00 UTC was obvious. @daily and OnCalendar=daily that sort of thing.
I'd say it's still kind of bold assumption, as
OnCalendar works pretty well with options like RandomizedDelaySec & AccuracySec
I agree, it's a bold assumption. I've made an edit that calls myself out on that. As it stands, I'm arguing from my experience of very rarely seeing jitter.
I think the bolder assumption here is that it would be someone else to begin with. Using a minimum age does not mean I stop to vet my dependencies. It just means that I am not auto upgraded.
I like the idea, but it's always going to be tragedy of the commons. It's safer to manually force yourself into a later phase. As more people do this, the chance of early detection of problems gets less likely, which increases the cost of being in an early phase. Eventually everyone will have an organisation mandate to force themselves into either the first or last phase.
Slowly “smearing” a release across timezones sounds very complicated for open source package registries to do. It’s worth keeping in mind that these services operate with little-to-no real budget, and their appetite for complexity is small.
I wish this aspect of the “original” cooldowns post was engaged with more: it’s not just that they’re a proper alignment of interests (vis a vis security scanners); they’re also free to implement, and therefore a judicious choice in open source ecosystems.
The registries don't have to do anything. The post states explicitly: "Therefore: any phased rollout logic must live on the project-side" The repositories show all available versions as usual.
That seems to require coordination from package installers, which (as a class) are roughly as resource constrained as package indices. But yeah, it’s a bit simpler there.
(I’m not really convinced on the value, though. The nice thing about a cooldown is that you can set your own “comfort level” around early exposure, with a clear step/transition point, versus each package having a unique exposure curve relative to your local time.)