We Must Pace the Frontier
10 points by dayanruben
10 points by dayanruben
All of it completely reasonable suggestions taken at face value, though I expect a lot of people (pro or anti AI) to assume malicious motives (market capture, power grabs, hurting competition, hurting free models, etc.).
I wish we could stop arguing about whether or not these things are marketing stunts and talk more about what AI regulations actually should be. If the risk is overstated then nothing happens. If it's not, then I'd rather we have learned something from the Cold War.
It's hard for me to take "AI regulation" seriously when all the hyperscalers are violating regulations across domains and jurisdictions and absolutely getting away with it.
So the US or the EU passes a law restricting how AI can be developed. What then? They'll suddenly grow the spine needed to enforce it?
The regulations should actually be "a machine must never make a decision". If the only story that sells that to the public is the sci-fi runaway superintelligence stuff, ugh but so be it. If the reason the slop kings are in favor is to dodge responsibility and screw with the competition and do marketing or whatnot, so be it. We should be clear though: we only want humans to make important decisions and judgments, no automation of any kind, does not matter "how intelligent" the machine could be.
What about, say, split-second decisions to avoid car accidents? I'm no LLM evangelist, but I've been wrong about their capabilities before, so maybe there are some areas where I'd want them making a decision. I strongly agree with you in principle, but I wonder if you see where the line is, because I don't.
Let's say he genuinely believes the importance and urgency of what he wrote at face value. Is there a signal he could possibly send of these beliefs that wouldn't be assumed to have such malicious motives? Is it possible for someone at the forefront to communicate this without being disbelieved?
Perhaps the answer is that he's the wrong guy to put forward this argument, no matter how qualified he is, and regardless of whether what he's saying is the right thing to do, or of how genuine he is. He's the CEO! Anything he says reflects upon the company and has to be interpreted in that light.
Even if you're the best attorney who's ever lived, you're probably still the wrong person to represent your own child in court.
Suggest somebody else? Many people have put forth this argument, usually the answer if they're not building AI is "what do they know, they're not building AI".
I genuinely don't believe a single word. At this point, I believe he would argue that his mother is an octapus, if that would help the stock somehow. That's based on past experience, not becuase I hold a grudge against Altman or Amodei (or anyone else for the matter).
However that's irrelevant. IMO regulation should be discussed regardless. But a very good starting point is force these companies to respect the existing legal framework for starters... LLMs that escape sandboxes are grounds for building seizure in other business with this amount of blast radius.
Is it possible for someone at the forefront to communicate this without being disbelieved?
If there were some signal in their actions that lined up with this rhetoric.
Motives don't have to be malicious to be self-serving. Even if you believe in the efficacy of these kinds of regulations, I think it's a little disingenuous not to recognize "everyone should go as slow as we're going" as at least having an angle.
I'm not saying Dario shouldn't have an angle, or expecting him to be a saint. I am just pointing out that we have reached a point where the most reasonable voice among those with any real power is someone advocating for regulation that would directly benefit his position. As opposed to say, a regulator. And that is very, very bad.
We’ve made clear progress in alignment — training models so that they remain safe, ethical, compliant with our guidelines, and genuinely helpful (the principles that are embedded in Claude’s Constitution).
I know this is off-topic, but — the jokes genuinely write themselves.
The main steps we can take to defend this gap are:
- Do not sell chips to anyone but me :(
- Punish anyone who tries to copy my model weights
- Pretty pretty pls stop stealing my model weights
there is a reasonable argument to be made here but god this post is so transparent and insufferable
Some kind of “speed limit” on the rate of recursive self-improvement (RSI).
Does recursive self improvement exist? Is there any documented case of it happening?
Unrelated: what's with the "spam" tags on this article? How does this qualify as spam?
I flagged spam as this post is essentially a bunch of a science fiction from the CEO of Anthropic and therefore meets the criteria of "content designed to promote a commercial service"
There are only two options: there is a documented case and it is not very effective. This is what is happening right now with the big AI studios using their own AI to develop the next iteration of their AI. Or there is a case and it is very effective, in which case we all die. So "documented cases of dangerous RSI" cannot exist before the end of the world.
Sure, but is there any evidence that it is even possible? I could fearmonger about how I COULD develop the ability to cast wizard spells and throw the world into turmoil, and there's only two options: either I will develop wizard spells or I won't. If I constantly talked about how dangerous the idea of wizard spells was, you might think I was a little bit silly...
Well sure, different things are different. The evidence that it's possible is that they're trying to do it, they keep doing small parts of it, and if the models keep getting better at the current rate they'll probably be able to do it pretty soon. Software and AI development are cognitive skills and it seems like you can get models to be pretty good at cognitive skills, maybe without upper bound or maybe with an upper bound far above human, or maybe not but we don't know. Right now it all points towards at least superhuman.
If you fearmonger about how you could develop the ability to cast highly dangerous wizard spells, and also you just worked out how to cast Magic Missile and Shocking Grasp, then it seems at least worth entertaining the notion that Fireball and Greater Wish are also in the cards.
There really is no point at all in giving the AI companies free advice. Moral suasion is not something they can take to the bank. They would argue, correctly, that cooperation is probably illegal, because it’s indistinguishable from collusion. It’s also arguably fiduciary misconduct, insofar as their responsibility is to the stockholders, not humanity. Even if none of that is true, this is: they’re not listening.
The only means of control is political. That means laws and regulation, and that means congress.
Congress moves quickly when moved. The Patriot Act was passed within a month of 9/11. If we light a fire under congress and get public control on this phenomenon, we can rein it in and profit from it as a society. If we don’t, we have only ourselves to blame for letting a few billionaires decide our fate.