When AI builds itself

flagging this as “off-topic” as it is an unvarnished press release, and not one with any technically interesting perspective relevant to this site.

this is not anyone’s definition of “Recursive Self-Improvement” except inasmuch as a compiler boot-strapping itself to apply optimization passes to its own codebase would be considered as such.

there’s perhaps a discussion to be had in how modern tech companies have co-opted AI safety terminology as sales-by-fear tactics, but that doesn’t feel like it would be fun for anyone here.

Submitting this as it has some interesting insights from Anthropic's researchers, particularly in light of the comments surrounding the Catholic Church's AI position paper. They predict a future as such:

Organizations that use AI systems would become much more efficient as time goes on, so we could expect to see significant productivity multipliers on each person in this organization. 100-person companies could do the work of 10,000- or 100,000-person organizations. This would revolutionize knowledge work and government services, but could also be turned to harmful ends, from authoritarian surveillance of whole populations to influence operations that tailor manipulation to each individual and run at a scale no human team could match. ... The evidence we’ve laid out here suggests that we’re likely heading into this scenario. But speeding up one part of a process often just shifts the bottleneck elsewhere: overall pace is capped by the parts that haven’t sped up. In computing, this is known as Amdahl’s law, and the same logic can apply to organizations. Anthropic has already encountered one signature of Amdahl’s law: as we’ve begun to push more code around the organization, human code review has become a new bottleneck.

The Catholic Church's paper emphasizes human life and its importance, this article almost blames humans for slowing down the pace of development and research. Thankfully, they propose what I asked in the previous article: Why not stop?

Unfortunately, they then basically put forward that if they don't do it first, somehow we all get less safe:

But if a slowdown simply lets the least cautious actors catch up technologically, it could leave everyone less safe.

An interesting read to understand the thought process of their researchers.