If this is true, the hyperscalers are toast
5 points by mattofr
5 points by mattofr
The future of AI is not large language models (LLM), but small language models (SLM).
My own experimentation here is that I have a local task codified in a skill that I run every day and I've steadily seen the default behaviour of both Claude and Codex become worse and worse at it. Often they'll not bother doing any effort and ignore the skill entirely.
So now the Qwen3.8 that I can download works for my use case but also it's become as good as the adulterated big league models are.
The term “small language model” seems to have shifted quite recently. I feel like even last year, it meant no more than hundreds of millions of parameters, and normally less than that. 8B models were small LLMs, perhaps because they ultimately behaved more like small LLMs than like traditional small language models. But now, I look around, and people definitely seem to be using it for 8B models and even somewhat larger.
I am not knowledgeable enough to determine the validity of this article, however, this seems to be geared at investors ("Seriously, if you are an investor trying to figure out where to invest in the AI hype"), and may be building a narrative in a certain direction. Plus the title is click-baity.
It also seems to be pitching a paywalled newsletter geared toward investors. Off-topic? Spam? Why not both?!
While I do think that local inference is going to be a huge part of the future, I do not think that it eats into hyperscaler utilization that much. There will always be server side apps in need of inference, and that need will almost certainly grow. Plus I think AWS will happily serve you these SLMs on Bedrock for cheaper than the big names (and at much higher margins for them).
I very much expect that's what's going to be happening within a year or so. Alibaba also just released XuanTie C950 chip designed for running Qwen locally. If the capability gets to something comparable to current frontier models, that's going to be good enough for the vast majority of tasks regular people do.
And turns out that you don't even need the model to be exceptionally capable. For example, there's a recent example where implementer agent generates multiple shots and then another role select the best one. Turns out this works better than Claude doing a single shot attempt while DeepSeek pricing still makes it vastly cheaper. https://github.com/llm-as-a-verifier/llm-as-a-verifier#self-verification-terminal-bench-21