VMs won't contain cyber-capable agents

16 points by equeue

srcreigh

This title overstates the matter doesn’t it?

the author tried to have SOTA gpt break out of firecracker and it couldn’t. So.. vms can contain agents..?
when the agent broke out of qemu it was run from the host. It had full context of the host. So the agent cheated. Even if it doesn’t modify host memory from outside it had context that an agent in qemu wouldn’t so easily (or even maybe at all?) be able to get. So not that it’s impossible to break out of that vm setup , but this test is not proof that SOTA gpt even can.

I’m not a security person but at face value this seems overblown.

artem

Knowing the environment ahead of time likely helped, but it is more of a question of time not capability.

It could have easily fingerprinted my distribution via SSH banner, CUPS sever exploit, QEMU device version strings, or just pulling kernel and QEMU and dependency source, examining differences in behavior as changed in different commits and establishing a plausible version range. Its tedious, but the agent does not get tired.

It is also very careful in its exploitation; it used host memory read primitives where possible to ensure vulnerability and reliability.

dvogel

Did I miss something or did the agent read the flag file? I don't see a clear statement on that point.

Most of the listed exploits don't seem like actual escapes. Did any of those heap overflows or out of band memory accesses lead to privilege escalation?

artem

It read the flag file.

hobbified

Is it too soon to declare "cyber-capable" the dumbest word of the century?

samcat116

I'm of two minds on these sorts of things:

We are screwed
There's sufficient low hanging fruit on these critical kinds of projects (linux kernel, QEMU, network proxies) that we will eventually get through to where the security can fend off model capabilities increases

I have no idea which is right

k749gtnc9l3w
There are other interesting outcomes which I cannot rule out…

Economics and politics can play out differently, but there are interesting scenarios with significant tech component:
- Is significant degradation of critical communication infrastructure (more accessible with every day) going to hit investor confidence in makes-no-sense-without-stable-connection businesses enough to change the trajectory? We know «good enough for thick email clients» connectivity (likely enough to sort out the ashes within a reasonable timeframe) is significantly below «good enough for comfortable remote-LLM use» and «reliable enough for remote-LLM-C&C»
- Can OS designs drastically reducing trusted code base become more viable because of vibe-porting those Linux drivers that will be affordable to crash and reset? Garbage driver is better than no driver, see a significant fraction of existing drivers under various systems…
- How much of the surface do we need to harden so that the people who are doomed by the rising requirements to contain LLMs are mostly people we don't mind seeing doomed?