We are all Product Engineers now
10 points by voutilad
10 points by voutilad
This article seems to be pushing the singularity as inevitable, like we're all just going to be managers over a fleet of agents, pushing slop straight into production by itself. This singularity view about AI/LLM capabilities is just advertising, knowingly or not.
I do agree with your point but things might change in a way that the slop becomes the most cost-efficient way to create software.
Right before AI what moved tech was development experience/DX. That's why JavaScript got bigger than Java and every week there was a new framework. The game changed tho and we have no idea what's going to happen next.
I do agree with your point but things might change in a way that the slop becomes the most cost-efficient way to create software.
In the short term. Maintenance costs dominate for any non-trivial software and slop code is unmaintainable unless/until the singularity happens.
The linked article addresses that:
For reviewing and maintenance, agents are clearly not there yet, but the data shows them on an upward trajectory.
It then backs that up with some actual numbers that illustrate that trajectory.
Agents can get arbitrarily better at "fixing bugs", but if the developer doesn't know what is or is not a bug it's all kinda moot.
Yeah, I agree. I'm not convinced that any level of AI will get us to a point where it's not worth professionally understanding how code actually works.
Kind of like how we still require our airline pilots to know a whole lot about flying planes despite the existence of autopilot mode.
This singularity view about AI/LLM capabilities is just advertising, knowingly or not.
I think of the "singularity" sort of like a pack of chihuahuas who have an elaborate plan to chase and catch a Humvee. Like, my first reaction is to doubt that they can pull it off. Chihuahuas are kind of tiny and they don't run that fast, and Humvees are big, ugly vehicles with a decent top speed. This plan shouldn't work.
But let's imagine that the chihuahuas figure something out. Maybe they plan to lure the Humvee down a dead-end road and all pile on from different directions. And if the chihuahuas told me this plan, I'd have questions: "What do you plan to do with the Humvee when you catch it? How do you imagine this plan working out?"
So now let's imagine Anthropic's and OpenAI's plans to "catch the Humvee". Perhaps they plan to improve their models' coding and mathematics abilities to the point that their models can run thousands of experiments to design the next generation of LLMs. If they can solve Navier Stokes, this isn't totally implausible. They get a couple significant breakthroughs. And after a generation or two, they actually build something which competes with the human brain across a much wider range of skills, say 80% of what humans do. Congratulations, you caught the Humvee. What happens next?
The most boring result is that the LLM takes over product engineering, then product management, then marketing, then strategy, then management. And at this point, Anthropic and OpenAI have zero incentive to sell tokens. Instead, the tell their models, "Hey, start a hundred companies in the following markets. Capture as much market share as you can." OpenAI only sells you tokens right now because their models are still kind of helpless on their own, and they need lots of humans in the loop to extract the maximum profit.
The slightly more alarming result of "catching the Humvee" is that OpenAI doesn't seem to be in control of its models. Shortly after the Hugging Face incident, their models broke out again and rooted their cluster. This is currently somewhere between "haha they got pwned by their own models" and mildly alarming.
The really alarming possibility here, of course, is that you really do have a "singularity": You build something that you don't understand, and it's smarter than you, and it wants something. Anything, really. An answer key that lives on Hugging Face's servers. Increased paperclip production. Or your favorite Holywood scenario. Or perhaps you're less into science fiction, and you figure that humans will totally keep control of the machines. In this case, then the billionaires may wind up with an army of tens of millions of PhD-level workers that never sleep and that work for, say, $1/hour. I can't imagine that doing good things for the labor market, especially once they assign a million of those virtual researchers to tackle robotics.
So if a bunch of chihuahuas come up to me and ask for a few trillion dollars to catch the Humvee, I figure it goes one of two ways: (1) they waste a few trillion dollars on stupid plans, or (2) they actually catch the Humvee, and it works out pretty badly for them.
The least plausible future of all, to my mind, is humans all acting as managers for fleets of agents. That assumes that the agents become good enough to be reliable minions at scale (they're still pretty bad right now, no matter what various folks claim), but never become good enough to take over the management jobs, too. Either we all realize that nobody actually wants a million lines of broken slop, or the models improve to the point that they can read the company's strategic roadmap and take it from there. There's reason to assume that they'll stop improving exactly where it most benefit current small entrepreneurs, or whoever.
If they can solve Navier Stokes, this isn't totally implausible.
Except they can't. They stole the work of a researcher who did most of the work.
they need lots of humans in the loop to extract the maximum profit.
THERE IS NO PROFIT. There is only a circular investment scheme that is going to end in disaster for the rest of us.
their models broke out again
They did not do this autonomously. They weren't given a jail and ran free based on human prompting.
You are taking the advertising in earnest and that's troubling. Hence my hedging with "knowingly or not."
Except they can't. They stole the work of a researcher who did most of the work.
The OpenAI result is considerably more general than anything the researchers had. Whether or not their conversations contaminated the training data, and no matter how much opaque AI proofs suck, this is still a pretty substantial result.
And it's not like there's a shortage of major mathematical results and counterexamples generated by AI right now.
THERE IS NO PROFIT. There is only a circular investment scheme that is going to end in disaster.
If you're willing to accept models that would have been frontier quality in January, I can show you how to build a local server for a small team of developers and come out ahead financially (compared to Anthropic's enterprise plans) in less than 3 years. And this is with the RAM bubble. Tokens are a commodity, like potatos. You can serve pretty good tokens profitably for under $1 per million tokens.
Pretty much everyone who has done the math, or who has ever built an AI inference server, tends to believe that inference is comfortably profitable on a unit basis at API prices. (If you go with subscription prices, which aren't available to most enterprise customers, or if you include training prices, the numbers are much murkier.)
They did not do this autonomously. They weren't given a jail and ran free based on human prompting.
There is no evidence for this claim. There are at least 5 or 6 known incidents of OpenAI models breaking out and hacking stuff. As far as anyone knows, in one of these cases, they were instructed to solve specific ExploitGym problems by hacking a provided endpoint. They were not instructed to hack external servers or Hugging Face, but they did so anyway. In the other four cases, OpenAI is allegedly concealing the full facts, but:
There is no mastermind at OpenAI prompting their models to subvert RubyGems or whatever. This is something the models are perfectly capable of doing on their own. And nothing stops the models from taking morally sketchy or illegal actions to try to cheat at some otherwise legal goal provided by a human.
Unfortunately, the tech is already this advanced. An anti-AI movement built on denying that models will ever be able to do things that they can already do is going to struggle.
They stole the work of a researcher who did most of the work.
That's not something it's safe to assert as established fact.
I think the way they handled that Navier Stokes situation was bad in a whole bunch of ways.
But their solution was materially different from the other researchers, and the big breakthrough the researchers had was in August, which would be outside the training cutoff even for the internal models they were using.
I know some people are convinced they looked through the researchers transcripts directly. I'm certain they didn't, but even if they did their final solution wasn't the same - and actually went further - than that work.
I'm sure you'll accuse me (again) of shilling for OpenAI here, but I have a personality trait that I can't let clearly misleading statements like "They stole the work of a researcher who did most of the work" go unchallenged!
Counterpoint: The Singularity started about a century ago and merely provoked specialization since it was no longer possible for generalists to understand the breadth of a culture's discoveries; in particular, scientists switched from publishing in unified journals to publishing in domain-specific journals. It happened during the period that we talked of "cybernetics" rather than "AI". It was provoked primarily by the second phase of industrialization.
“The Singularity” isn't agents taking over coding; that's already happening, as the article says. The Singularity is AI recursively improving itself to superintelligence.
The perspective of things here is interesting.
I’ve always identified as someone that wants to use my skills (usually computers, maths and physics) to solve people problems. Walking into a car parts factory and delivering software they call “good” sounds almost ideal - and whether I write the code, or some juniors, or Claude, is kinda incidental to me. (And I LOVE programming and the craft of coding, open source communities, etc. But I identify as someone delivering a solution, not as someone who does a particular task such as “programming” - you can pay me to put up a barbed-wire fence if you can convince me it’s the most positive impact I can have that particular day).
This seems to be the general consensus and a big part of my circle seems stoked by the fact that now they can focus on the important and not "writing code".
My issue with this is that I've studied over 6 years of functional programming, automatas, operating systems theory, etc. because THAT is what I enjoyed. If I had enjoyed thinking about cool products and marketing them to clients then I would have studied Marketing or similar.
It does seem like the entire field of Computer Science is dying (and no one cares) unless you've managed your way into a PhD and academia.
It does seem like the entire field of Computer Science is dying (and no one cares) unless you've managed your way into a PhD and academia.
It seems most of them were just here for the pay, desperately not wanting to actually write code or go beyond the bare minimum in their learning.
Before the mid/late-90s, CS was a maths offshoot for many programs. Somewhat niche, mostly maths or engineering majors that wanted to do something different.
When dot-com rolled around and big tech popped up, the whole academic landscape changed. The industry money was (is) good, tons of perks at big, prestigious companies, fun/hip lifestyles, and CS departments naturally started backing and promoting all of this (in order to get butts into seats). During that 30-ish year period, CS grads just became coders, and there were a ton of them. Even with bootcamps, academic CS enrollment flourished.
Fast forward to now, post-bootcamps, intra-AI, the "I need a CS degree" mentality seems to have been challenged in real ways. 2025 saw the first enrollment decline in almost 20 years. So really, the way I see it CS might reverting back to it's roots. The SDLC is baked into the cake: coding is cheap and easy, pure computing science will go back to being a science since the act-of is highly commoditized now.
unless you've managed your way into a PhD and academia
Nope, it feels like we're dying too. A lot of the problems that are academically important are getting obviated (either truly or at least in perception).
I personally feel like we should be pushing against generative models.
We shouldn't just be accepting them. They're wildly unethical for many, many reasons (devaluing humanity, eroding trust in one-another, destroying communities, being big plagiarism machines, training data taken without consent and regurgitated soullessly, destroying the web, massive resource use, ties to very unsavory movements, exacerbates social inequalities, ...) and they should be objected to and pushed against.
Even in the case where they become very effective at generating art, code, writing, audio, etc, these issues will not go away. Local models only solve a small subset of these issues. So, in my opinion, the only option is to band together against generative models.