Reviewing AI Code Is Not A Viable Argument (2025)
47 points by Diana
47 points by Diana
Worth noting that speed doesn't have to be your only goal. You can do things like "oh there's a bugfix here, let's spin this off into a preparatory commit that can be reviewed in isolation". Or "I don't like this type structure because it permits some invalid states to be representable; let's go fix that". Or "okay, I think the testing story doesn't quite give me enough confidence, let's spend some time prototyping more advanced techniques like PBT/fuzzing/formal methods". These are all things I'd have clubbed into the same commit or left as a TODO tech debt comment in the past. None of them are related to speed, except in the most general sense of correct software saving you time down the road. The marginal cost of doing things right is now (stunningly) lower.
LLMs are very open-ended as tools, and the value you get out of them mirrors the values you bring to them.
(edit: there are also risks to building too many half-finished prototypes that never see their way to production. So it's not all good. But I think overall this is a huge boon for rigor-focused engineering.)
I often find myself emphatically agreeing when I read ai takes like this. However another voice in my head mentions that this isn’t what’s happening in most companies. While it absolutely can increase rigor, more often then not I see it being used to race to the lowest quality MVP possible. Perhaps that reflects more on the company culture rather than LLMs but it’s still disappointing to see. I hope the industry will get its head on straight and produce higher quality software.
Using coding agents to build higher quality software - as opposed to cutting corners and churning out faster slop - is definitely a decision companies need to make.
Need to? I'd like them to. I hope we can show that there are advantages to those who do. But I've worked with clients who were rotating intern shops churning out slop long before AI and... they have products. They make money. I don't like it as a craft, but in the market sometimes it works.
They won’t. And they aren’t.
I feel there are certain intrinsics when it comes to attention and care that remain the same even as code velocity increases.
These types of decisions happen at the individual level. I've never worked somewhere that actively encouraged/required quality.
I work somewhere where quality is not just encouraged, it is structurally non-optional. The structural requirement for quality (because we ship appliances to customers without traditional observability) was one of the reasons I picked the place I work at.
The problem is that the companies who could most benefit from this are the kinds of companies that would never do it. Slop is not an AI thing; it's a management thing. AI just makes it a lot cheaper to produce.
Yes, this is very distressing to me. Feels like the inexorable logic of capitalism.
Perhaps native, but I believe that we'll be able to make a stronger economic case for good software eventually. One can dream :)
I can spin a clanker with a single prompt in my historical projects, and it will start finding real bugs without even trying hard. I know because it keeps happening.
Guess what? Humans are sloppy too. I am sloppy. Mistakes happen. LLMs just bring it upfront because they work so fast and are quite a bit dumber in many respects. Mistakes compound, so if you just naively let clanker make code changes, you're going to have a bad time real fast.
But it's lazy to think that just because naive use of the tool is flaky, the tool is not useful.
My current go to slop-flow is actually getting 5 specialized reviews, including architecture, maintainability, reliability/security, etc. automatically after every change being made, and I have a whole design documentation system to keep things organized and agent's decision making improves significantly Is it perfect? No. Is it better than naive approach? Sure it is. Can it be done better? I hope. And that's also the fun part. A new weird tool in my toolbox to play with.
Note that I only talked of producing and reviewing code here.
I said nothing about using a stochastic text analysis to find bug patterns
You said:
My problem with LLM Coding Assistants is that I cannot see, in the face of the scientific evidence, how they can help someone write code better or faster.
Helping to find common bug patterns is absolutely a way these tools can help someone write code better and faster.
Fix code yes. Write? I don't think so. But at this point we are deep in Wittgenstein semantics
They write code that is of average quality. Which means they write code better than 99% of humanity. Considering that most of humanity cannot write any code.
I would argue that code that doesn't exist is actually the highest possible quality, therefore LLMs are actually dragging the average down
Less code is better if there’s a more (or equally) maintainable way to provide a desired capability without, or with less, code.
However, I would argue that no code is strictly worse than crap code if it means that the desired capability is not met.
We are obsessed with maintainability. If someone vibe codes a piece of software that solves their personal problem, maintainability is not a concern. Their problem is likely ephemeral.
Notions of maintainability assume the software is a permanent fixture. We are entering era where software could just solve one problem one time and then not be needed anymore.
The losers in this scenario are SaaS companies. Winners are regular people who are empowered to solve their own problems.
People using LLMs don't even see code, they see software, so in a way it doesn't exist. Most software people vibe code will be thrown away eventually after the little piece of software solved their problem and is no longer useful.
Ephemeral code. Code in motion.
The issue with agentic coding is with the kinds of mistakes they make. It's one thing to have little bugs hiding in corners of the code, but another thing to make choices that are completely misaligned with the goals of the project. (e.g. making your code 10 times slower just to solve an edge case no one cares about)
I am also skeptical about whether it makes the whole process - development, review and finally deploy faster; I find that sometimes I might generate something faster with a prompt, but then it takes more time to validate and understand to the extent that I find myself wondering - wouldn't it have been faster to just write by hand in the first place? And arguing that it depends on the case... judging also takes time and energy, that I would rather spend on something else.
But, as long as the person submitting the PR takes ownership I am agnostic about their LLM use - it simply does not matter as far as quality, correctness, consistency and all other metrics we find important are concerned. If it is better/faster for you to implement with the LLM, do it, if not, do it on your own - the ownership stays the same, it is on the human author.
As far as I personally am concerned, I love AI for learning and extending & deepening my understanding - but I do write most of the code manually, since I still find it more productive - all things considered, not only the typing part.
I got a dialect of Clojure working on top of Chez Scheme in under a month. It's passing the full Clojure test suite, and already runs many popular Clojure libraries from the JVM, and it performs around 1~2x JVM Clojure in most cases. There is absolutely zero chance that I would've been able to build a project of this scope in my spare time by hand.
Note: This piece was written nearly a year ago
Well, that’s clearly a problem, then.
I think pretty much anyone who actually is optimistic about LLM’s and coding would agree that the last six months and the last three months specifically have significantly improved the utility of these. Do you actually use the latest for-pay cloud models? Do you have controls in place that prevent entire categories of failure, such as the principle I developed out of having worked in the auditor space, MFIC? https://gist.github.com/pmarreck/b30aa3ca69cb70a5526f8a63ab8c8d7e Do you have tooling that helps keep already-developed code and project structure in context such as https://github.com/pmarreck/dirtree and https://github.com/pmarreck/codescan ? (Note that these are also useful to forgetful human developers or those who are or have become unfamiliar with a codebase!)
Anyway, either you make it work and enjoy the benefits, or you rebel and continue to write bespoke hand-coded things (while ALSO frequently writing bugs and security holes, btw) while others (perhaps writing occasionally flakier code, but far faster) pass you by.
Lastly, speaking as someone who spent 2 man-years working on a million-LOC Ruby on Rails codebase for Desk.com that is now retired, corpo code is ephemeral, which lines up nicely with LLM-produced code.
In your case, working on Erlang internals, it makes admittedly less sense to trust it at all, but I don’t see the argument against letting it write at least things function-by-function that you then review- you’d obviously know what’s subpar, but I bet you’d be pleasantly surprised SOME of the time.
You can work with the knitting needle or the loom. Perhaps “both” is ideal, depending on the context.
Upvoted because I know you care about code. 😉
I think pretty much anyone who actually is optimistic about LLM’s and coding would agree that the last six months and the last three months specifically have significantly improved the utility of these.
And people have been saying exactly that for at least two years now.
Many of the people who are saying "the last six months ... have significantly improved the utility of these" are not the same people who've been making that claim for two years before that.
Do you have examples or is this just conjecture?
I am an example of this. A year ago they were useless for me. Today they are very useful for me.
So much anecdata. So little actual data when one tries to objectivise it. Does that mean openai and others don't try to do proper analysis, or that that they try but bury what they find?
I replied to a post asking for examples that people saying AI two years ago were good is not the same set of people saying it is good now. That was the original dismissal in the thread I'm replying. That because they were saying it two years ago saying it now must also be hype
did you try to objectivise with studies when git was adopted? Or it only matters now because it's not real programming and you don't agree with it?
When I switched to git, it was objectively better than what I was using at the time, which was CVS. Making a new repo with git was a single command (git init) versus several commands on a server and the client. The fact that you don't need a separate server was a plus in my mind ("No code is the best code" or something to that effect).
And no, using git isn't programming (it can be used to track anything that is text based), but it solved a real issue I was having. LLMs don't solve any issues for me.
That's fine, they solve issues for me. But since apparently our experiences don't matter because apparently we need peer reviews for some things but vibes for other things are fine and expected (basically your post, since YOUR experience with git is what you are talking about and what made you switch).
Yes I did, actually :)
You need a peer-reviewed study to know if something is useful for you? if it helps you? if it makes sense for your workflow? Or is this just a high-horse form of cargo culting?
Can you square hand-washing for doctors or checklists for pilots? Both were very controversial when they were first proposed, and it took a lot of studies to prove their effectiveness for the respective careers. I could see the argument that both processes prevented needless deaths (patients for doctors; pilots and passengers for pilots) and thus, such studies were required to get them adopted.
To bring this back to my previous comment, my adoption of git had nothing to do with it being better per peer-reviewed studies and more being more effective for me in programming. LLMs have yet to improve my programming and there have been studies that show that LLMs are problematic (loss of critical thinking, loss of skill, etc.) but that might be because it's hard to prove how well LLMs improve a programmer's output because we can't fork() a programmer and have them do a project with an LLM and the same project without an LLM at the same time.
No. If you read a bit more, i said multiple time that peer review is not the methodology of science. It is a modern part of it.
But yes, I am sorry, I need more evidence than my own feelings. Because we know, for a fact from history, that abandoning the methodology of science and engineering for personal feelings leads to deep problems.
I recommend studying the history of bloodletting and the arguments its proponents have. Or for the opposite case, hand washing
I don't need studies to enjoy using/doing something or even recommend it to others. "I use this, I find it cool, maybe you'll find it useful" or even "you really should try, it's great" when I'm more excited about it, I'm, you know… human? But the conversation around agentic coding is more along the lines of "luddites are absolutely ridiculous for not seeing this for the game changer it really is, and I am absolutely certain my testimony is worth more than any data anyone might have collected about it; maybe it has terrible externalities but it's too powerful to ignore and be left behind".
I enjoy drinking beer and wine, and I don't feel it affects my life negatively. Should I recommend drinking alcohol to everybody for this reason? Should I ignore what studies say about alcohol consumption because it does not match my personal experience so far? I assume you'll agree I should not.
it's not real programming
I did not intend to gate-keep here, I am a sloppy amateur programmer. I don't even have the anti-AI absolutist stance you might think, I don't care much about intellectual property, and I believe most concerns about AI apply to computers or even technology in general (and these concerns are not ridiculous IMHO). Some people even pay me for "doing AI" for medical image processing and "digital health" stuff. I don't even think anythingLLM should be thrown out the window, it seems hard to ignore LLMs in the field of natural language processing, although I'll admit, I don't know much about it.
I suspect a disagreement on utility of scope. Utility hasn't been binary.
At any given date (surveying 2023 to 2026), LLMs remain useless beyond a certain scope. That scope has been increasing steadily. LOC to be sure, thanks to "context", but more in cyclomatic complexity, architecture, and, the big one: holding multiple goals.
I suspect those who have found them useful all along this ramp were already comfortable with "pair programming" or "XP" practices, with experience pairing across increasing competency levels. This informs how much structure needs to be in place for "continuation" to fill it in, and how much idiomatic training is needed for a refactor to be successful.
(I'd argue recent "whole codebase" transforms such as rust|zig translations which are shown to do plausibly by sliding a narrow scope across the codebase, play into the advancements in harness loops more than depending on competency of the models.)
Absolutely! I know some people that found use before last winter. It just didn't fit my workflow at all and they slowed me down greatly.
I personally have never been comfortable with XP and pair programming, but I've spent the last 15 years understanding how my brain works, what are my strengths and what are my weaknesses and I've found a way to help me with my weaknesses (memory, recall, hypefocus periods with absolute inability to kickstart working in others).
seems my reply never made it.
i don't work on internals. I work on corpo code too. I am the one that clean up behind LLM users.
note that all you just said doesn't invalidate a single line I wrote. Quite the contrary.
Once or twice per week I will take prompt ideas from my notes, open up arena.ai in a private window, and see what the latest models have to say. This is the main way that I learn the tells of chatbot writing. The latest models are still bad at writing, sorry.
Please consider how your first paragraph is structured. Evidence which only works on people who are primed to accept it is no different from a mentalist's hypnotic script which only works on the suggestible; activation of certain memes is not the same thing as validation of evidence. The continued inability of chatbot enthusiasts to demonstrate empirical objectively-measurable improvements, while continuing to manifest personal or subjective gains which sound good to them, should be thought of as a sort of auto-confidence trick which they play upon themselves.
In the realm of how many lines of code the frontier models are writing, I'm personally baffled that none of them have taken a serious look at how to produce "negative" contributions (in that the model would suggest removing code). ("The Best Code is No Code At All" - Jeff Atwood.) For example, if a codebase is bloated with a lot of overhead through layers of abstractions, the frontier model being able to observe and untangle the complexity by proposing removing lines of code would be quite novel.
It's also entirely possible that they are capable of doing this, but I have yet to witness it first hand.
They “can't” really do this when editing an office document either. (As in "can" but it seems unnatural for them.)
When you ask for a new change, they preserve the old in a kind of "new this, not old that" pattern, where "that" is what you wanted removed. Makes sense insofar as they are continuation machines, adding new tokens to prior tokens being their whole bag. (By contrast, it's delighted to delete whole files or omit commented-out code swaths.)
That said, emphasis on synthesis, concision, simplification, etc., help steer into them into returning less, for both prose and code.
My problem with LLM Coding Assistants is that I cannot see, in the face of the scientific evidence, how they can help someone write code better or faster.
It's hard to take pieces like this seriously when your own personal experience - and that of so many other credible, experienced developers who you spend time with - has shown how they help someone write code better and faster, time and time again.
"Show me the peer reviewed study". I don't need to. I've seen enough already.
"But the METR study says developers over-estimate the impact of AI tools on their own productivity". I'm happy for you that could find one study that reaffirms your priors.
"Show me the peer reviewed study". I don't need to. I've seen enough already.
That's a bafflingly low standard of proof for someone who calls themselves a researcher.
That is the standard of proof we have for pretty much all other programming practices. We didn't adopt git or rust because there was a peer reviewed study proving that it would increase productivity.
Plus, if I could find a peer reviewed study claiming that using git or rust was bad for my productivity then I would ignore it, because it would be wrong. Most studies of programming practice are pretty terrible anyway and obviously won't generalize from 10 undergrads with an hour of experience in a tool to a large company of experienced professionals with years of training.
While we're at it, here is my not-peer-reviewed study of the peer reviewed literature - https://www.scattered-thoughts.net/writing/qnd-review-of-ppig-1989-2015/
We didn't adopt git or rust because there was a peer reviewed study proving that it would increase productivity.
git and rust also aren't known to cause delusions(0). LLMs cause the user to be an unreliable observer due to their psychological effects, so we need an objective measure of productivity improvements since self reports are unreliable. This is a unique problem to language models that traditional tools do not have.
The interesting finding in the METRE paper, which model advancements cannot fix, was not the productivity measurement, but the delta between the engineer's perceived productivity and the measured productivity.
Feel free to take that into account when evaluating my work from now going forward.
I do not think peer-reviewed academic papers are the gold standard in helping us understand how software engineering works. The incentives in that system produce weird, hard to follow results - at a glacial pace.
There's a big difference between academic papers that help understand physics or biology - where you really do need rigorous research experimental processes and benefit from peer-review - and fields like software engineering where you can do the work yourself.
How many practicing software engineers do you know who read papers about software engineering more than once or twice a year, if that?
How many practicing software engineers do you know who read papers about software engineering more than once or twice a year, if that?
That argument would have more weight if software was well engineered in general; do you considered it is? What was considered a supercomputer when I was young is barely enough to render most websites, even when actual content is effectively just text. I think we can do better.
I think software today compared to software ten or twenty years ago is enormously better, and lets me get way more done.
Do you think software engineers don't read papers because they are lazy and bad at their jobs, or do you think it's because they've tried spending time reading papers and found them not to provide good enough ROI on that time?
My hunch is the latter.
I exist in the glacial space you describe (though, blessfully, not in human aspects research). The interpretation that I and many colleagues hold is that software has, in general, become more fault tolerant (in the sense of avoiding catastrophic failure), but simultaneously more defective. The incentive structure in software development is currently far more favourable to develop quickly and fix later strategies, largely due to a lack of accountability mechanisms for the companies selling software or software services.
I would tend to agree with this assessment. End user software in my experience has definitely gotten worse, more sloppy, harder to use, and with odd glitches. This especially has been the case over the last 5 years or so. Aesthetically I guess it's prettier sometimes, but in terms of UX and performance it's horrible. I can't even run Twitch on my phone anymore for Christ's sake. A video streaming app should run perfectly fine on this hardware.
I think it's a secret third thing: most software engineers are not skilled at technical writing or technical reading. There are hundreds of pages of good insights, much of it within the past century, which should be required background reading for our discipline; most of it isn't taught or is taught incorrectly. This isn't a moral judgement; I'm not saying people are "lazy and bad at their jobs", but that they have not trained particular essential skills.
Yeah, and it's not even just technical reading - absorbing information from academic papers is its own skill, universities have to teach it.
TBF, people are more likely to read papers if they are working on infrastructure. That infrastructure is typically where we have invested make more resilient platforms. The benefits then trickle up to the rest of the software built on those platforms.
E.g., at my current job, the authorization platform is based on some papers put out by Google. It seems to work well enough for our company, which is quite large.
My hunch is that it only applies to a very narrow aspect of programming (which is usually not mentioned). As an example, I never found a good definition of "unit test" that can apply across a wide range of languages and when I ask for a definition, I'm scolded for asking. As such, I've always found it hard to apply to the stuff I write in assembly or C. Unit testing seems to be much better for a language like Java or Javascript, which I don't use.
I have to avoid the term "unit test" in the same way that I avoid "TDD"!
Unit tests carry all sorts of baggage around only testing a single "unit" (whatever that means), which inevitably leads to a growing pile of mocks.
I prefer to avoid mocks as much as I can - I like tests that are as close to answering the question "does the feature work?" as possible.
So I'm stuck with "automated tests", because I don't want to get into the weeds concerning if something is an integration test or a unit test or a smoke test or whatever other terms people want to use.
Those approaches do have specific subgenres. Your idea is close to the classicist/Detroid style testing. As opposed to the mockist/London style. It's a shame those labels are not commonly used. Same with Beck's version of "unit of functionality" vs corporate Java "public method is a unit".
Agree with not letting LLMs "unit test" that turns out like testing does the lang work.
Recently, occurred to me to try two of Opus' favorite new words in this context, and… they worked. The concepts must be really important to it:
Avoid testing inside the code, test at the surface, and always for whether the code does what the user needs.
Avoid testing inside the code, test at the seams, and always for whether the code does what the user needs.
Depending how one words it, Claude generates varying balances between surface and seams. If one doesn't like one of those flavors, add it to the avoid as well.
Still working on how to get any testing mileage out of "honest caveats". Trying "Avoid telling me about honest caveats, write failing tests for those conditions instead."
The definition from this 1969 paper is pretty good:
Unit test: testing, outside of the the system, of a part of the system that may have less than a complete function.
Fair enough, academic publishing has a lot of issues, and to be honest I know next to nothing about the academic field of software engineering per se (I'm an amateur!). However I don't think personal experience is worth more than (imperfect) attempts at collecting more data than mere testimonies and other forms of subjective experience. In the field I know vaguely more about (human health), practitioners (clinicians) don't have to read papers to benefit from whatever researchers are finding out, it trickles down to them in various ways. It isn't perfect, but it's basically all we have besides superstition. IMHO we would have better healthcare overall if personal experience was drastically more critically scrutinized by doctors and patients (us!). Which is why that "I've seen enough already" really made me choke a little. Maybe drawing a parallel between medicine and software engineering is ridiculous though. :)
A good researcher would recognize existing studies seem to be leaving a lot out and critique them accordingly. Blindly accepting studies is not what a researcher does.
own personal experience.
You mean, the same that doctors use to explain why they don't wash hands?
Or the one they used to defend bloodletting?
Science literally was built on the bodies of the people we killed by trusting "own personal experience".
Note that I demand at the end to be proven wrong. Show us, in away that is robust and methodologically sound, where it differs from what we have found with robust empirical research.
I am ready to change my mind. But, I am sorry, "own personal experience" will not cut it. Nor a dozen anecdotes that explain nothing but just tell us we are wrong.
It is how we killed thousands of people in the past. Science and engineering allowed us to make the life of billions of people better. Sorry that I will keep defend them, even if it hurts your own personal experience.
This is not about the METR stuff. It is about a pattern over centuries of this happening. We developed science and engineering to counter that and it works. Sorry, I will not throw it away because you think you are better. I am too humble for that, and I need to sleep at night.
What was the last peer-reviewed paper you read about the craft of building software that positively influenced you and helped you become a better software developer?
That is a good question. Hmm... Probably research on TDD, I will bring it up once I am back from travel. Mostly research on TDD at this point shows no real impact, hence why I do not push it hard.
That's a fun example because it goes directly against my own personal experience, too!
I'm a firm believer in automated testing because I've seen directly how much more productive it makes me - because I can ship a new feature for a project I haven't touched in years without having to manually check that I haven't broken any of the existing, documented behavior.
I'm quite confident I could find holes in any paper that attempted to prove otherwise.
(Test-first development, on the other hand, has failed to convince me. I've tried writing the tests first and found it to slow me down without improving the quality of my code... until the last six months where I've found forcing the coding agents to write their tests first appears to produce less hallucinations and code that's better factored and more likely to pass review - anecdotally though.)
Note that I did not say "automated testing". I said "TDD". There is a huge difference and if you don't know it, that is, actually, quite scary.
That's why qualified I qualified "test first" development in my parenthetical at the end.
I know from experience that "TDD" is an ambiguous term - some people use it to describe red/green test-first development, but it's often used as a label for any kind of automated testing.
So in my reply I deliberately used the less ambiguous terms "automated testing" and "test-first" - precisely because I do understand the difference.
[in my experience] it's often used as a label for any kind of automated testing
I'm a fan of your work and believe you make this claim honestly but... this seems absurd. Who is using "TDD" to describe all automated testing in general? That seems like an abuse and misunderstanding of terminology rather than a legitimate difference in opinion.
I've worked on and with several teams that claimed to do TDD, by which they meant "have an automated test suite" + "sometimes include a regression test in bug-fix commits".
In the same way that REST has drifted to mean "RPC over JSON+HTTP", and Agile has drifted to involve a lot of up-front design with flowcharts, the meaning of TDD in a broad set of industrial settings has drifted away from writing the test first and then coding strictly against the test.
To speculate on the reasons: TDD as originally described (the strict sequencing of test -> minimal code -> refactor -> loop) works so poorly that when someone hears "TDD is great" and then they try it and it doesn't work they assume it's a problem of their understanding. They experiment with and adjust their process until landing on some variation of (think -> code -> test -> loop) and then decide they're doing TDD.
In my recent round of TDD clarifications, one surprising experience is that folks out there don’t agree on the definition of TDD. I made it as clear as possible in my book. I thought it was clear. Nope. My bad.
Sound like a classic example of semantic diffusion. I've fallen victim to that one myself.
What common term or acronym would you use to describe automated tests that might or might not be written test-first?
This is a major source of frustration for me. I dislike that the acronym TDD, in its classic definition, embeds test-first which I think distracts from the value of automated testing generally.
In my own writing I avoid saying "TDD", so I'm stuck with "automated testing" (the word "testing" by itself overlaps with manual testing) instead.
So any time an online discussion mentions TDD I feel compelled to clarify if they mean test-first. Which is what I did here.
If I could wave a magic wand I would redefine TDD to mean development that incorporates automated testing, and enforce TFD for the less useful test-first development pattern.
Thanks for clarifying. However, I'm not sure that your quotation supports the claim that there are folks out there using "TDD" to refer to automated testing in general. In my view, TDD refers to a development workflow, which describes the points in the development lifecycle that involve writing/running automated tests. Different people can and do disagree about what the specifics of this workflow are, as your Kent Beck quotation illustrates, but invariably it refers to a kind of development workflow. On the other hand, automated testing is closer to a component of a project, which implements automated validation of the other components.
To be clear, I do appreciate that you're being explicit about what TDD means--I just remain uncertain that anyone defines it as encompassing all automated testing.
What common term or acronym would you use to describe automated tests that might or might not be written test-first?
I don't have one. I just say "automated testing", or even just "testing"--qualifying "manual testing" explicitly when needed. To me, automated testing is something that most serious projects practice and view as useful (with some notable exceptions, of course--I recall the SumatraPDF developer doesn't see much utility in them), so I rarely need to argue for its inclusion. TDD, on the other hand, is far more controversial, being a particular development workflow.
If I could wave a magic wand I would redefine TDD to mean development that incorporates automated testing, and enforce TFD for the less useful test-first development pattern.
Again, I can always appreciate the goal to be explicit, but I'm not so sure that TDD is a poor choice of term. Certainly TFD is a clearer one, but I wouldn't be in favor of repurposing TDD to describe the use of automated testing. I write a lot of automated tests, but I would not characterize my process as test-driven; they are supplementary.
I'm not @Diana, and it's not a (or several) peer-reviewed paper(s) but in my case, it was two books. The first is Writing Solid Code (and it's not lost on me this was published by Microsoft). It was C-centric, but there is solid advice there about API design (like design-by-contract).
The second was Thinking Forth. Get past the Forthisms, and it's about thinking about the problem to be solved. Today the advice to think might be dismissed as taking too long ("TIME TO MARKET!!!!!!") but I've found it works well for me.
And here's a bonus third book, The Mythical Man-Month. It's less about coding per se and more about software project management and why there is no Silver Bullet that will make programing trivial. Even with LLMs, you have the communication overhead over design to contend with.
Oddly enough I was revisiting the Mythical Man Month just last night. It's impressive how much of that advice from 1975 is directly applicable to building software with coding agents today. The section on "conceptual integrity" in particular.
Amusingly there's only one idea from it that I think coding agents subvert: Brooks's law itself, which says that adding more developers to a project makes it later.
The reason that law holds so firm is the communication overhead that each new developer adds. I suspect that coding agents dramatically reduce this effect, because an agent is such an effective tool for answering questions about how the system works already.
I've seen anecdotal evidence (yes, it's always anecdotal these days) that adding a solid engineer with deep coding agent experience to a project really can help that project deliver value faster.
I just can’t get behind the idea that this is a serious level of expectation to have when crafting your takes on programming, ie, that you need to be able to back them up with a peer reviewed paper. It’s simply not how I’ve formed a “take” on any other programming topic in my entire career, and I don’t think many other people have either. We are mostly professionals here with a lot of professional experience, and it is on that that I think most of us draw to come to our conclusions about programming, not the scientific method.
I agree with you!
I do not see peer review paper as a standard. That is not how science was made historically either.
Buuuuut. Strong methodology, based in science and engineering, with good reproducibility and on a diverse and large enough sample?
Yeah I think we should hold ourselves to that standard.
There are some things that can be vague and worth evaluating. Then there's "I got the result I needed in less time than I'd take to type it at full speed". Then there's a whole class of problems where the review is just unnecessary, like "correctly transform known inputs to known outputs" (parsing, compression, various encodings and transformations) - everything interesting about it can be evaluated externally without ever seeing the code.
So at some point the burden of proof really flips. "You claim I could get this result without LLMs faster than I can type it (if LLMs are not an improvement)... how?"
I few counter points beyond "personal experience", reviewing code written by LLM which you describe in detail on how exactly it should be written should take a lot less time to review than asking the same thing to some intern, LLMs are really good at writing code, and in a way it's like writing code by describing it, a faster way of writing code without needing to write every single character, as someone who types really slow this is quite a bit useful alone, also I get basically instant feedback rather than waiting on some intern. Also, at least before the coding agents era, I used to review a lot less code, but now a lot of my work is just reviewing code, and therefore I've gotten a lot better at it, I do not believe the hard ceiling is at/around 400 LoC / hour, that too based off some historical peer-reviewed paper, keep in mind code written by recent models is a lot easier to read line by line over some interns sloppy work.
Can you show me any strong study (no need to be peer reviewed) that support your argument for said difference? On the theoritical model of how these things work, there is nothing to support your hypothesis.
Doesn't mean it is not true, but you would need evidence (strong and with good methodology across a significant sample, possibly reproduced) to make it a plausible argument.
No study, but I'll chime in with more anecdotes (everyone loves those).
I think a key thing to help overcome review fatigue with LLMs is to participate in the planning phase - either plan the code yourself, or work with the coding agent on a plan before it writes any code.
Reviewing a thousand lines of LLM code with no additional context is miserable.
Reviewing a thousand lines of code where you knew in advance that the plan was to modify file X to add a new JSON endpoint that queries for data Y and returns that data using format Z, plus add a new test to exercise that endpoint ... is far quicker and less painful.
I mean that sound great but you do not address the point made in the post and in research...
Why do you all manage to do something that all empirical research say is impossible?
Are you lowering your standards unknowingly? Or is there something fundamentally easier to review in LLM code? A hidden quality?
"all empirical research say is impossible"
I think you're leaning way too hard on that idea, especially since you haven't yet cited the "empirical research" for your 400 lines of code per hour of review number (I had a look and couldn't find any clear candidates for that myself.)
Saying that something is "impossible" also isn't how research tends to work, because it's trivial to disprove an "impossibility" by finding just one counter-example.
I'm not sure there's anything I could say or show you that would help here. All I can offer is more anecdotes.
Here's one: I have a collection of 224 tools that I have vibe coded - that's classic vibe coding in terms of not reviewing the code at all. I've been collecting those for a couple of years now.
You could reasonably expect that these would be a disaster - a huge pile of buggy, unmaintainable crap that provides no value at all.
And yet I genuinely can't remember the last time they had a bug that prevented me from getting value out of them - and I've been modifying them to add new features (again, pure vibe coding) without any difficulty at all.
The majority of them were written on my phone!
I do not consider these to be production quality software. My real projects are all still built using plenty of hands-on code review. But the fact that I've been able to build 224 small projects without that level of review does make me question how important a robust review process really is for this kind of code.
Since plenty of others in this thread have been sharing anecdotes, I'll just do the same.
Are you lowering your standards unknowingly? Or is there something fundamentally easier to review in LLM code? A hidden quality?
My personal experience at my day job is that the LLM proponents are absolutely lowering their standards unknowingly.
I actually think it's not even just as simple as "lowering their standards unknowingly" but more of a combination between that and "becoming lazy unknowingly."
It's been absolutely shocking to me the amount of times co-workers who are heavily into LLM tools have spoken at length about how of course they are carefully reviewing the output of LLM's in all of their day-to-day work ... but then I can regularly and easily find ample evidence in their work that they've basically been rubber-stamp approving things. When you mention this stuff to these people, they brush it off as if it's just a one-off.
At this point in time, I am not of the opinion that you can reach these people anymore.
I am on the fence with LLMs strictly for code generation (using them a lot in general), but the arguments presented here are pretty bad: for me, it does not matter who the author of a PR is - the same quality standards apply; human or machine.
What matters more is the ownership - if someone prepares mostly LLM-generated PR, they must take the ownership and responsibility for it; they should do their best to, as previously, slice PRs into a context & size for other humans to be as digestible as possible.
Nothing changed there - code is still the ultimate specification of software that we, software developers, must own and understand.
What matters more is the ownership - if someone prepares mostly LLM-generated PR, they must take the ownership and responsibility for it
100% agree with that. A computer can never be held accountable. That responsibility is uniquely human.
I personally endeavor to apply higher standards to machine-generated code, not merely the same.
"Show me the peer reviewed study". I don't need to. I've seen enough already.
The strawman here is asking for something that would convince them - because your own personal experience isn't enough to convince others, and shouldn't be, of course. And your response is that you're already convinced so you don't need any more evidence, which is a complete non-answer.
I hate to say this, because it sounds insulting and I don't mean that you yourself are in a cult - but this is actually how members of cults communicate. They take challenges to their beliefs personally and defend them but have no idea how to convince others.
I would (seriously) like to see a software development process where every part is supported by peer-reviewed studies. This seems like a rather unfair standard when everything else we do, from product management to architecture to task management to source control to CI/CD to observability practices, is just stuff that we feel like seems to work better than other stuff.
I agree. A post I think of when this topic comes up: The Underwhelming Impact of Software Engineering Research (April 2022).
Actually a lot of this stuff you mentioned is supported by peer reviewed and good methodology studies!
I don't think the initial claim in this post has aged well: AI agents did make a lot of mistakes a year ago, but it's not clear at all that this is still true. Like a lot of people, I saw an inflection point with "frontier" models around November of last year.
It's true that LLMs change the volume of code and thus put new pressures on our review capacities, though. But our responsibility as engineers is to push back and ensure that review loads match our actual capacities.
I’d argue that they’re still making significant architectural mistakes - as of my last tests, they remain unable to identify when shared behaviour could be extracted and reused, for instance, instead merely adding more new code. In some ways this is worse - the new code often works, so it’s likely to be merged without cleanup.
I’m not really following the logical train here at all. I don’t feel I need to review AI generated code any more or less than a human. I give them the same review because my bar is the same for my approval either way.
Edit: never mind, I missed that the answer I looked for is already there. Section “The Limits Of Reviews” says “it seems that a maximum of 400 LOC/H is a good maximum speed acceptable for efficience”. That’s measured in LOC/hour, not LOC/review as I initially thought.
You summarize these conclusions from research on code review:
A review that last more than 1h is too long.A review that has to be effective cannot be more than 400LOC at a time, in that time.
And when calculating how many LOC a developer can review per day, you translated those conclusions into this input:
For every 400 LOC written by a LLM Coding Assistants (at best, less for code that are hard to review), we need one professional senior developer to spend 1h of his time reviewing the code.
But that doesn’t necessarily follow from those conclusions. If a senior developer could review 400 LOC generated by LLMs in half an hour, and therefore review 800 LOC in one hour, that would be consistent with those conclusions. I don’t see evidence that 400 LOC must take at least 1 hour to review.
Was there such evidence in the code review research, and you simply forgot to mention it?
Still relevant after the correction above:
I’m also curious exactly which papers on code review led to those 1h and 400 LOC numbers. If you didn’t save the names of those papers and are only summarizing from memory, then you don’t have to spend lots of time searching for them again – I ask just in case you already have that information.
1h for 400 LOC smelled like BS to me so I calculated what mine would be for work.
In the last 6 months I've reviewed 173 PRs totaling 139k lines changed. For a normal 40 hr work week we can see the rate of review by assuming what % of my week I spent reviewing. If I spent 25% of my time reviewing, that's ~540 lines/hr, 15% -> 900 lines/hr, 5% -> 2700 lines/hr.
I'd guess my real throughput is 500-1k lines/hr, somewhat close to the (not) cited stat.
Note that what you count here is probably what the research call "lightweight review" which is not considered approximately comparable to the standard and results that "code review" mean in the research.
For the kind of review you mention, we do not have any support in the literature for it helping reduce bug count significantly. Doesn't mean it doesn't, just we could not find it.
We do have evidence that it helps with knowledge sharing and knowledge distribution, which we have support for being important for long term well being of projects.
I don’t do “lightweight” reviews or “heavyweight” reviews, I just read the code. That was even a generous calculation considering I only counted work, and I code & prompt far too much outside of work.
The insinuation that I’m half-assing it is honestly bullshit and disrespectful. If you spent half as much time trying to code with an LLM as you do reading these studies, you’d have a much more grounded understanding.
I am not insinuating anything. This is the tern of art in the literature for what you describe. There is no negatice or positive quality ascribed to it by the name.
I have it somewhere, i can go check.
Note that most of them are more at the 100 to 200 loc/h limit, but they tend to be older, when programming languages were a bit more prone to bugs.
Ping me in a few days, rn i am travelling.
I personally tend to use LLM as "just another refactoring" tool. If I have a codebase where I would want to refactor common usage patterns that I myself have identified, writing a prompt and getting a refactor-base that you can then iterate upon is a blessing.
Same thing if I need to bootstrap a project, I could write all the boilerplate manually, or I could ask it to create a project tree structure minimal enough for me to iterate upon (like using a snippet-manager in whatever editor you have).
Sure, I could fiddle with my LSPs, try to find as many $language_specific tools to adress the problem, but as a glorified awk/sed LLMs for me certainly provides faster iterations when it comes to that.
Like with many things in my opinion.. there are the "correct" usages of a tool, and.. "yeah sure, you could but do you really want to?" sort of thing. Not sure if that makes sense outside my head, if not I could possibly elaborate after a few more cups of coffee (just woke up).
If Donald Knuth is impressed by Claude then maybe just maybe it's worth looking into. I meet a lot of people who are nowhere near the level of Knuth yet they find no utility while he does, and he's 88 years old!
I really dislike appeal to authority as an argument for something. I would prefer to see evidence for efficacy and safety rather than believing that something is good because a subset (some with authority, some without) consider it to be so. There is a lot of evidence against these tools being sustainable, ethical, or long-term helpful versus a modicum of evidence for them being useful for limited application.
The curious should look into WHY he was impressed. I was appealing to curiosity, not authority. Those who don't know Knuth might want to look him up. Those that do know him but don't know why he was impressed, should look it up.
The ethical problems with this technology is who owns it and how it's being built out. Lets acknowledge we are talking about capitalism and the problems it creates.
I would like it built by democratic oversight and available to everyone as a public service, and of course locally run where it can be, but that's just me.
At least the Chinese are building in the open and giving it away for free. Kimi-K3 is really good and weights should be released soon.
Regarding utility...
I've talked to sooo many people who, just months ago never saw a line of code and have no idea how to program make software that solves their personal problem that others neglected to solve. We might actually be entering the age where people will use computers what they are meant to be used for, computing! Instead of just consuming computer versions of old media.
I don't think people in tech understand how elitist they sound about being dismissive of "vibe coding". I've been programming for 30 years. I'm a "hardcore" programmer who writes hard software in C++. I find a LOT of delight seeing people who don't know how to code find joy in making software.
I have read Knuth's take on all this, but I found it entirely unconvincing in the face of all the known problems. Your or others' personal experience is a good signal for your feelings about using it (which is not something I'm discarding outright), but not efficacy, sustainability (in the business and practices sense), or any other criteria that my concerns relate to.
EDIT: I just realised that I have somewhat already had this conversation with you last week on your Abject submission. I was wondering why I had already heard about Knuth in this context recently...
Sorry for the wall of text. I do remember our conversation.
You haven't yet met anyone who has never programmed build their first software because of Claude, or some other coding agent?
My central claim is that llm coding IS bringing value to regular people who don't know how to code. This is very recent, last 8 months. It's empowering regular people to find joy in building software and to solve their problems. That's a great thing! We should not leave the power of computers exclusively to programmers!
If E. W. Dijkstra was unimpressed by the entire artificial-intelligence programme then maybe — just maybe! — it's worth ignoring artificial-intelligence products. Quoting a famous 1984 lecture, EWD898:
… Alan M. Turing thought about criteria to settle the question of whether Machines Can Think, a question of which we now know that it is about as relevant as the question of whether Submarines Can Swim.
So let me understand, you are comparing Knuth's opinion on the current wave of AI, which he has access to and has experimented with - with Dijkstra's opinion in 1984 about a technology that was completely non-existent at the time and where the idea of what is happening nowadays was reserved for science fiction. And you are trying to prove the above argument absurd and the best quote you could find is someone in 1984?
And then, there's the quote. What Knuth said is that they are useful and you are arguing against they can think. So the comparison to me seems like a category error that doesn't apply at all to what Knuth is saying or the rest of us here are saying.
LLMs are not intelligent and don't think. That doesn't mean they aren't useful. Does a databse think? Obviously not. They are basically a compressed database of knowledge. Or another way to think of them are text compressors. They are so far the best text compressors man has made. That's incredibly useful. So useful they are able to decompress a prompt into code (which is what LLMs do when they write code).
I've been building machine learning systems for 20 years and understand the math. I don't have magical thinking about what these things are, unlike the AI cheerleaders and AI doomers.
They have an obvious utility, just talk to anyone who has never coded before talk about, with joy, a program they made that solves some specific problem of theirs.
Will machines "think" like us? No, never. Will we build machines that can do closed-loop learning? Yes, we will, if global warming and ecological collapse doesn't destroy us first.
just talk to anyone who has never coded before talk about, with joy, a program they made that solves some specific problem of theirs.
This is such a low bar it is less a celebration than a damning indictment of the state of modern computer technology. Computers should have enabled people to make programs for their home computers for forty years by now! Hold on- I hear the Old Ones murmuring something about "magazines" containing something called "BASIC". Uh, and the Slightly Less Old Ones chanting "Excel"? Hm.
Though, I wouldn't have thought this current iteration is nearly as empowering as those if it's a simple case of asking the appliance to make a program for you. It just seems like another category of product to consume. And if it isn't so simple, these people would be using Python if it had a fraction of the advertising budget the AIs do.
Yes this is a damning indictment of the state of modern computer technology! The 80s led to commercialization of computers and businesses took over and they need to extract value so instead of empowering users they rent them power.
I am one of the old ones! I actually learned on basic as my first language.
While basic and excel ARE empowering, they are nowhere near as empowering as LLMs. Why? Because it will take a user YEARS of dedicated work to do anything remotely sophisticated inside these boxes. Better was something like HyperCard and Sketchpad.
What computers great is they compute and computation solves problems. LLMs provide a high level interface that allows a user to directly manipulate the machine with their words. I am shocked there are people using terminals who don't understand the value of that.
English (or spoken language in general) is imprecise. Ask a genie "give me a million bucks" and you might just end up being trampled to death by a thousand thousand male deer.
There are professions where jargon develops to help members talk more precisely about topics than general language might allow. I'm thinking of law, medicine and even math, among others. Such language might even contradict common meaning of words, but it exists to help people communicate subtle topics in their profession without confusion (or getting trampled by deer).
instead of empowering users they rent them power.
Precisely! I'm glad we can both look at the same trend and come to the same understanding. Except when the same rent-seeking institutions that are driving the trend turn around and apparently throw us a bone, I don't look at it and think O, how fortuitous!
LLMs provide a high level interface that allows a user to directly manipulate the machine with their words.
It seems we'll have to agree to disagree, particularly on the meaning of the word "directly".
Imagine, if you will, a magic chisel, that will fashion any wooden trinket you can describe. You don't even have to supply the wood or sharpen it! Which is nice because you don't even know you can learn how to sharpen a chisel. The means of production is sitting right there on your desk and all you have to do is call up a foreign megacorporation to seize it for you. To me, that sounds less like empowerment than mere convenience.
I called out BASIC and Excel (and Python) because they show that natural language is simply not the barrier to programming for non-programmers. Every tool has a learning curve and everyone intuitively knows they need to learn how to use it at all before they can use it for their purposes. It's more a matter of being shown the tool exists for you to use in the first place.
Once upon a time we used to have regular releases that were (externally) tested. That became effectively neither, with continuous deployment, questionably ineffective “reviews”, and testing on users and customers. Developers are encumbered with testing, leaving the unfortunate user to discover the rest.
Now with AI we need a different way. Surely the alternatives are remove the human from the loop and accept the consequences, or revert to an older style software development lifecycle?
@pushcx Does Lobsters have a process for updating post titles with the year it was written, if it was in the past?
The author said "Note: This piece was written nearly a year ago", and that's crucial context to have, given the speed at which the LLM landscape is changing.
I think the whole piece is even more applicable today than it was when I drafted it.
I would argue any change since then in the landscape has only reinforced its point.
The note in the article is clear, and the release of this article is actually within the last few days. It is a choice of the author to release it now, in this context, so it doesn't need a label IMO.
I can't view that link, I get a very brief popup saying something about being unable to access, which disappears before anyone could possibly read it (I can only assume OpenAI is vibe coding ChatGPT.)
But, I really don't find the part you pasted very compelling at all as a response to what I read to be OP's central point, that it's difficult for humans to effectively review LLM work, despite many users of LLMs still saying that they "review every line." I feel like many people glaze over when reading AI writing because it seems like it makes sense and uses relevant words, but when you try to dig into what it's actually saying it totally falls apart. It drives me a little crazy that people post things like this as if they weren't truly garbage writing. Like this list of points which feels like it's formulated to look maximally pithy but literally says nothing:
I honestly have no idea what point any of this is supposed to be making but it definitely has the vaguely dismissive tone of someone who thinks they're winning an argument.
I think that you should disclose that you are employed by OpenAI to work on Codex when you argue for LLM-assisted development.
To be clear, I don't really agree with the parent article and in some way I agree with your points, but I think it is poor form to talk about this kind of stuff without indicating you work on something closely related. (Let me also say that I didn't look you up because of this comment--as a Codex user I glanced at the contributor list one day and saw a familiar face from Lobsters. I do apologize if you're no longer at OpenAI.)
I suppose it would be more efficient to just have claude read this and post a response. Better yet, we could just kick off the claude vs chat flame war in the comment section of every submission to lobste.rs the minute it's posted. Then we could finally relieve ourselves of having to read the opinion of lobste.rs users and get straight to the good stuff
So basically the answer is "if the standard is too high, we should just reduce our standard". That only works if the new standard fit the market.
I would love to see research to support that hypothesis! I think I even call for that at the end of the piece.
Could I recommend you to learn to summarise, as it seems your LLM are not good at summarising.
In the past, I've been told that it is unkind to inform people that it is cheap and easy to learn for oneself how to write, including how to communicate technical concepts, how to disagree using both rhetoric and dialectic, how to choose registers and phrasing which enhances or mutes emotional overtones, and how to perform basic analysis via close reading.
