AI Is an Elite Crime Spree
60 points by jcd
60 points by jcd
This feels like it is right on the line for off topic here, but since we’re seeing calls for new regulation of AI firms from the firms themselves, I think it is worth pointing out that that is absurd. We have laws for these firms. They are not being enforced.
What AI firms are trying to do is regulatory capture. They try to make the right friends in government and pull the right levers to lock in their leading position in the market. The reason they do this is because they have no moat and no secret sauce. This kind of play is as old as government regulation, the only remarkable thing about this instance is how brazenly and transparently they do it.
What’s the proposed mechanism of action here? Is the theory that they would ban open weights? If so I agree. Otherwise I don’t see how regulation would be in the interest of the large firms.
unless you mean that regulation imposes fixed costs on all players that the larger firms are better resourced to absorb. That might be true but it doesn’t prove the regulation is bad. All regulation, good or bad, sensible or stupid will impose fixed costs on all firms subject to it.
So is regulatory capture a way of saying regulation bad? Why is it bad? This “regulatory capture” argument is a favorite of the a16z crowd. They push the regulatory capture meme because fixed costs are bad for startups.
Do you have a vested interest in the viability of AI startups with small fundraising rounds? AI startups that can’t afford to pay for plausibly-safety-ensuring hoop-jumping just don’t have my sympathy. If they go extinct I can see why that’s a16z’s problem but I don’t see how it’s my problem or yours.
There was a semi recent episode of the YC podcast where the hosts admit that startups are often legal risk arbitrage. Big company A would like to produce legally adventurous product Y but they are likely to get sued because they are very rich. So instead you spin off a new company fund them and let some naive youngsters shoulder the legal risk. If they go bankrupt or go to jail that lands on them if it works out they often get absorbed. This particular podcast is very much a quiet part out loud moment. SV AI startups are often little vessels that can take a legal torpedo without dragging down a large existing company. These recent YC startups have been especially legally and morally adventurous. I do not care if they get regulated out of existence.
The proposed theory is that the big AI startups demand and get regulation that enforces constraints on them, but crucially, constraints that only they can afford to pay for. This is presented as a reasonable deal which enables them crucially to declare that only companies with these constraints may bid for government contracts. Or that products produced by companies not adhering to these regulations are too dangerous to be allowed to be used.
One of the "safety outfits" Anthropic suggests should be investigating them is METR, which is basically friends of Amodei, so they'll get a big payout too.
I know I wrote an overly long comment but, did you only read the first sentence? I addressed this theory. That’s what I mean by fixed costs. This is true for regulation in general. it does not require capture. I agree that regulation could protect large firms from small ones. Why exactly is that bad in this case? How does a low-AI-regulation environment benefit you or I? I struggle to see how this changes the competitive situation much at all, it’s not like anthropic has threatening but undercapitalized competitors. You said “only they can afford to pay” so the regulation will cost a marginal half trillion? That seems unlikely.
Is the concern that US startups will miss out on government contracts? So what? I hope they do. Startups have more motivation to cut ethical corners compared to big famous firms. Regulation being good for incumbents is not final proof that the regulation is bad. Regulatory capture has a specific meaning. it’s a form of corruption. If the regulation is sensible but happens to help incumbents, that’s not capture that’s just regulation.
The point of regulatory capture is that the entities being regulated dictate that regulation, unlike regulation imposed on them from the public or representatives of the public. This means that the regulation is less about protecting the public and more about protecting the companies.
If we take the AI companies fears at face value, they should be shut down immediately. Having thousands of well paid intelligent people working on a technology they themselves say have a 10% chance of ending the human race is insane.
Agreed! We have the same model of regulatory capture.
Regulatory capture is meant to describe a mechanism that leads to bad regulation. What I am asking for is for someone to tell me what rules they are pushing that are good for them and bad for the people. I am actually open to being convinced, I’m just annoyed that I keep hearing people gesture in the general direction of an argument about regulatory capture without actually pointing out where it is at risk of happening,
As far as I know, the regulation they are pushing either doesn’t help them or it helps them but it’s sensible.
The most capture-y thing I’ve seen them push for is banning open weights. I’m split on this one. This would help them ALOT… but it might also be sensible. it doesn’t require capture / corruption to get such regulation passed.
Curious how banning open weights might be considered sensible? Especially in a country that won't even ban guns, it seems like "good guy with a gun / open weight LLM" is a clear analogy, especially with the recent cybersecurity events.
For this generation of models it would not be sensible. If trends hold up we should expect to see Astra quality open weights models in ~6 months. So far it’s proven easy to strip an open model of all refusals. I’m not sure that means they should be banned. We will have a better idea in 8 months or so, if there is major harm that comes from an ablated model it seems likely enough that the admin will crack down. As for the gun analogy, I think that’s pure US inertia. We don’t have the right to bear more powerful weapons (tanks, missiles ect). current models are more BB gun than tank, but if trends continue that could change.
What’s the proposed mechanism of action here?
I think... They're trying to regulate the market so that Chinese LLM companies cannot sell services to western corporations and (later maybe) individuals, legally... Without going through these regulations (that they'll control tightly through a consortium or whatever, all this has been done time and again). However, NVIDIA and other HW resellers (e.g. Dell) are openly opposing that approach because US AI companies are creating their own HW now, so they need access to open models to compete.
ps. Take everything with a grain of salt here, I'm reading the same stuff as everybody else here, I might be wrong obviously.
Please somebody explain to me why these things and everything they produce are not copyleft?
Because, so far, no jurisdiction has determined that a model trained on copyleft code is itself a derivative work of that code. I think that's a shame, personally.
I think it would be the best way out of this entire problem: You can use this stuff but everything you make will be copyleft. Take it or leave it.
Whether things being generated by the model are derivative works is different question than whether the model itself is a derivative work.
Also in response to the comment above, whether you use a copyleft license has no bearing on the resulting model being a derivative work, that's a general concept in copyright. If it were to be deemed that the models are derivative work, all the stuff included in them with no license at all would be a bigger issue probably.