Defending the patch gap against automated attackers
6 points by alcortazzo
6 points by alcortazzo
This should probably be read in the context of this article (especially the otherwise surprising references to GLM 3.5 specifically rather than models generally). Money quote:
In our experiments, GLM 5.3 used this gap to build a webpage that, when visited by a Firefox user, read a file from the visitor’s computer and sent it to the attacker with no further user interaction. In particular, we found that any Firefox version released between September 17 and October 26 could be compromised in this way using only information available at the time of release.
And a literal money quote:
Our total research spend targeting Firefox exceeded $100,000 in API costs, including experimentation to build a realistic offline environment and our preliminary testing. However, once we developed our harness and pointed GLM 5.3 in the right direction, exploiting the browser became cheap and reliable. Our marginal cost of compromising a new version of Firefox was about $1,000 at API pricing, and would potentially have been lower if using subscription accounts.
I guess it depends on the details, but $1,000 for an exploit sounds pretty cheap to me.
The important bit is that the model exploited the patch gap: the bugs were already fixed in tree but have not yet made it to release. We have already been rigorous in what kind of patches we allow to be committed and when. So yeah, we will now be more rigorous about the set of patches and how the might combine into a full exploit chain.