No, local models will not win
13 points by liberty
13 points by liberty
For me, local, open-source (not "open weight") models are the only things that would cause me to consider using LLMs. If there isn't an endpoint in which we're not renting our brains instead of using them ourselves, what's the point even for those who are enthusiastic about LLMs?
I disagree with the author about LLMs in general, but I agree with the conclusion. If the one people can rent is better, they're more likely to want to use that. There doesn't appear to be a positive end result for users and programmers without the market otherwise crashing.
For me, local, open-source
Just for my curiosity, do you consider that list to be models that are declared open source, ones that are open-source + open-data, or you really mean you will only use ones that you have trained yourself?
I ask because I have a standing rule that I don't consider software to be truly open source unless I can build it, because a code dump is not the same as my ability to affect change to that codebase. But, very similar to Chromium, if it is so incredibly compute intensive to do so that I need to rent a m500.xxgargantuan, that's really skating the line for me. I literally cannot possibly imagine how much local horsepower would be required to replicate (e.g.) TildeAI's allegedly open model
Technically, I'd allow it. It'd be far better than the status quo especially if all the other problems with LLMs were resolved. Reproducible builds would be nice here but since LLM training is probabilistic I'm not sure how it could apply here.
That said, I've compiled Chromium on a mid tier laptop. It took like 12 hours. We're talking a whole other beast here.
I guess an important question is at what level LLMs are best modified. If it's pretraining, then yeah they need to be trained on my laptop to be useful. But if they're more like game assets where they are more useful in combination with other software and can be tweaked afterwards with other tools, I'm more okay with not necessarily being able to train one myself.
In some cases, you can also choose a local model due to privacy concerns. If all inference were done by huge corporations, it would put too much power in their hands and make them vulnerable to insider attacks.
"win" is a false dichotomy - it's a bit like someone saying "local computers will not win" in the 90s and claiming that servers will always be more efficient and users always prefer faster machines.
If you said that you'd hardly be wrong... most people's "local computers" are fancy graphics terminals (using a convoluted display language called "HTML"), storing all of their data and delegating all of their compute to the big mainframe in the cloud.
But yeah, nothing ever "wins". Big becomes the new small, small becomes the new distributed, distributed becomes the new big.
lot of "business as usual" assumption in this article. No collapse of the AI industry, no collapse of the hardware supply chain, no enshittification, no foreign powers bombing data centers, no regulations or mass retaliation against the tech oligarchs who control the closed systems. The AI industry as it is now is not going to last, one way or another. Consumption patterns have proven completely irrelevant for its development, so if we are going to use local models or not is not really a matter of consumer preferences.
The argument that local LLMs will not "win" because they'll always be less powerful than frontier models is like saying that personal computers will not win because mainframes will always be more powerful
I think your comment would be covered by the author's thoughts in the "Local models are not useless" section.
The author doesn't define "efficiency", which is a problem.
Also, this claim:
almost everyone’s revealed preference is to use the strongest available model in their price range
is unsubstantiated and as @dualvariable points out, makes a likely personal experience-based (read: likely heavily biased) assumption about what people are wanting out of these things, on average.
I'm always suspicious about arguments about "revealed preference". It tends to be shaped by either advertising or external constraints.
Most people don't need a model that can solve the Riemann hypothesis in order to give them a recipe for the ingredients they have in their fridge. Or even to do a web search and summarize the results.
I think this is a classic case of the author assumes their usecase is the same as everyone else. I don't want agents that do more without human intervention. I don't trust agents to do that sort of work unsupervised. My ideal usecase is something I can assign low intensity boring tasks that it can safely grind on in the background and then I check the output of what it produces before it does anything with it.
The author stopped typing after the word "win" since there are a great deal of things one could possibly "win" and it's pretty presumptuous to imply that renting compute is the only way to win all of them. If that were true, no one would have a general purpose computer because the cloud is better, or Amazon Luna or GeForce Now would replace gaming rigs, etc
almost everyone’s revealed preference is to use the strongest available model in their price range
That is true for now, but once we hit diminishing returns it may change. At work, I do not use Fable, or even Opus sometimes, because they have become so similar in terms of capabilities. And providers also have to make the models even more proactive to get over the diminishing returns, which I don't prefer personally. For personal projects. Qwen3.6 27B is enough for most of my needs.
datacenter models are always going to be cheaper
That's a weak reason, I prefer local models because of data privacy mainly. I also like that I will always have access to it, and it works offline. And my GPU is both for gaming and inference, so it's not single purpose.
In the end, as they keep building more datacenters, local models may not "win" but that win condition is a false one. Local models don't have to be better or fully phase out cloud models. I think they are already good and cheap enough, and they'll only get better at closing the gap.
Some things that may change the equation significantly
Stuff moves so fast, it's easy to forget how just a short time ago, ChatGPT was the most downloaded app on mobile app stores, and people were confidently predicted that more people would pay OpenAI monthly than were paying Netflix.
This narrative has quietly died down, as focus has shifted towards "agentic workflows" in software development. But just as consumers are gonna look at a monthly $20 bill for ChatGPT and say "nah, it's not worth it", companies can take a look at their monthly token spend, check what they're getting for it, and decide it's too much.
Economics is like physics - it can look like you're defying gravity on the top of a parabola, but gravity and money people actually have available to spend always wins.
Very well put. They made a daily usable model with GPT 3.5 and now they are moving the goalposts for themselves for agentic, long-horizon enterprise use cases in hopes of turning a profit, but that may not even pan out due to token costs. I guess their next goal is cybersecurity arms race, which is absolutely abhorrent but what else could we expect? Casual users are nowhere in the equation.
almost everyone’s revealed preference is to use the strongest available model in their price range.
I think the "local models will win" take is that eventually (soonish), "strongest" will become less clear, and "cheapest" will stop being rental GPUs.
Current MoE (DeepSeek) are already shuttling pretty thin slices of expert weights per token while keeping the context processing and shared expert busy.
This split, i.e. hot dense rasoning loop slowly fed relevant knowledge, might be a local optimum for inference at the edge.
Fast 32GiB VRAM GPU on 64GB/s PCIe fed from 128GiB RAM is currently (ignoring RAMpocalypse) higher mid-end workstation. Would load prompt in 2s (give or take) and generate at 50 tokens/second with MTP.
That is fully sufficient for office work, coding assistance and research. Yes, you need to retrain the model every couple of months for it to keep up, but the architecture is bounded by the physics of the available hardware.
The same hardware arrangement is required for realistic gaming, so unless Steam is going under, replaced by consoles, the hardware is not going away.
And one might argue it will take exactly one high profile case of cloud provider leaking internal company data to a competitor to create serious demand for edge inference in the corporate world.
It's not my conclusion.
For me local models does everything a bigger model could do, because I don't have extraordinary use cases. AgentWorld / Ortnith-1.0 are very good at prototyping (source code), Gemma is quick to act as a "Classifier". I don't need more. Local models may soon run on a potato and be good enough for everyone while big models are staging currently.
Never a 3T model will be outsmarted by a 30B model, but if 80% of the 3T model tasks are done by the 30B, what is the use case for the 3T model? Does the 20% tasks justify all the infrastructure and environmental costs?
I refuse to pay any cent to any LLM provider.
If I pay a model, it must be open-source, fully open-source.
I think your use cases are a tiny fraction of the LLM world. And it seems the model you mentioned (Ornith-1.0) is open-weight rather than open-source?
Yes, my use cases are a tiny fraction of the LLM world. But are the other use cases different?
(It's Open-Weight, I don't pay for it, it runs on my cheap GC, I don't know any open source model that is actually use-full)
I can emulate a PS2 on a 50$ handheld these days. Current models will be available locally in 5 years time, and they'll be the 700$ HP laptop - more than enough for most people.
There is this classic book, The Innovator Dilemma that describes decades of worse technology beating "better" ones.
The TL;DR of that book: new technology always hit a point were they are good enough for the use cases of their users. When that inflection point occurs, the differentiator start being on other dimensions (cost being a frequent one), and the newer, "worse" technology, now good enough for the older technology users, start being appealing for the benefit along these new dimensions, and the mass of users start abandoning the older, "better" technology, even though it still perform vastly better along the previous dimension.
See:
The interesting point is that it is a gradual process and it is often the low paying users that leave first, leaving the impression for the legacy technology firm that margins are improving. Those firms move upmarket, serve higher paying customer that need the legacy technology better performance. But at some point, even for the higher paying customer, the newer technology start being good enough. And since that newer technology was built of smaller margin and price points, it is vastly cheaper. The high paying customers move on to reduce costs, and the legacy firm completely collapse.
The "Local models are not useless" section kind of address this, but fail to point out that at some point in the future, local model may jump out of their niche and kill the frontier model business. It doesn't matter if frontier models are better in the absolute. It matter that they are good enough in a way that local model are not.
It's pretty clear that one GPU per developer with a local model is not going to be likely. At least not for coding. But the utilization across a small dev team with local LLMs isn't too bad, particularly when people run background agents as well. I wouldn't discount that this might be a model that some people want to work with and could be economically good enough.
GPU per developer is probably cheaper than what a lot of companies have been doing!
Though running one big server with like DeepSeek V4 Flash would probably be more cost effective.
Very roughly $8k before VAT/tariffs in previous generation Chinese hardware, 4 parallel streams. I think those are 2U, 800W peak. And you'd be swimming in excess VRAM for context cache. Could easily resume some 200 sessions back.
If I had a capable GPU I'd be trying open models a lot more.
I think we're going to get small models for specialised tasks like NER.