The Tower Keeps Rising
46 points by facundoolano
46 points by facundoolano
This is definitely one of the better articles I've read recently on the topic of what's happening to the software engineering industry, which I've been finding a bit unnerving.
When I look at some vibecoded scaled-up projects the codebases become Babel not because nobody can communicate, but because nobody needs to. Every developer has a tireless translator that can explain a corner of the tower and make whatever local alteration they ask of it. The changes keep landing, even as the architectural language that would let the humans reason about them together disappears.
I am way more tolerant of coding AI (especially slightly less capable local AIs) than many people on Lobsters. But I disagree with this statement whole approach to working very strongly.
My experience with vibe-coded software, in the original sense that "no human has looked at the code", is that most of it is terrible. It is often bug-infested. It is often 300,000 lines of Go slop for something that could have been 20,000 lines of crisp Rust. It corrupts its data storage. Frankly, it is sometimes so bad that people in group chat start (quite sincerely) talking about whether the author needs a wellness check or something. I have tried a bunch of it, and most of it was shockingly bad. Like "80s Unix GUI" levels of bad, or late-90s Windows drivers from fly-by-night hardware companies.
The only real exceptions to this are:
So as someone who actually uses AI for certain kinds of projects, please forgive my profanity here: If you don't read the LLM output, you won't know when it's a steaming pile of shit. By definition.
But I disagree with this statement very strongly.
Can you be specific about which part you are disagreeing with? Your comment does not seem to address that quote at all.
Sorry, that was very poorly phrased on my part. I disagree with that whole style of working, and the idea that it is in any way viable for most projects. Not with the author's summary. Thank you for asking!
For context: I'm the author. I intentionally did not make a judgement if this is a good or bad thing, or if this is going to continue working. It's primarily an observation that with agents you can continue to make progress even when people on the team maneuvered themselves into situations where previously they would have needed to talk to each other.
At Babel, the loss of common language stops construction whereas in AI-assisted engineering, construction can continue after shared understanding has already collapsed. The lack of an immediate failure is what makes it curious and a bit disorienting. The tower does not fall, and so we do not notice what was lost. It just keeps rising.
So far.
The tower has already begun to lean even before "AI". Most for-profit software past ~2012 has been getting worse at such a pace that normal, non-techie users are noticing it.
And Simon isn't alone. Just a year ago, I think the memetic shape was by and large that something along the lines of "agentic engineering" is what people could or should do, and, though I think many people are hesitant to admit it, I think most people using these tools are tending towards vibecoding and not agentic engineering, just as Simon himself found himself pulled.
I found that most of my existential dread towards LLMs obsoleting my craft or my love for the craft dissipated when I drew two lines for myself:
For one: Don't bother with LLMs in my free time. I don't use either coding agents and I opt out of search stuff, unless I'm so desperate to figure something out that I deem even the confabulation machine's input worth a shot.
Simply by not focusing on them as much, it's way easier on my mind. And some might say "oh, sure, just stick your head in sand, that'll solve everything", no, it of course doesn't, but realistically what more could I do than not engage with the tech in my free time? I have zero interest in going after random devs whose projects end up on the slop hit lists, I already wrote my spin on the "I hate what LLMs are doing to the environment and people's jobs" blog post, and I cannot affect big corpos.
And for two: At $DAYJOB, where we are strongly encouraged to use AI, I only ever use it for code reviews and as an errand boy, that can answer stuff from the codebase I could find myself, but can't be bothered to. All the code I write myself, because if I didn't do that, I no longer have an intuitive understanding of what's going on and at that point I am unable to do my job to the level I'm expected to by others and by myself.
I find that it's a decent addition to linters, because there are simply things a deterministic machine cannot figure out, while a pattern matcher recognizes as incorrect. Sometimes it says really stupid stuff, but because the code is mine and I largely understand it, I don't blindly rely on its inputs.
So by the time another human is involved in the loop, I'm reasonably confident that my own intuition and the machine figured out the obvious stuff (i.e. clear logical errors and false assumptions) and they can focus on the more involved parts of the code review. All done without wasting each others' time and disrespecting each other by letting the machines talk on public channels (be that a forge or a chat) instead of us.
The company recently announced hard limits on the tokens we can use. I'm sure people who relied entirely on vibes must be getting antsy that they'll need to go back to using meat-based inference, but personally I am completely at ease, because my workflow barely costs a euro or two a day, and the rest is all my colleagues and me.
Simply by not focusing on them as much, it's way easier on my mind. And some might say "oh, sure, just stick your head in sand, that'll solve everything", no, it of course doesn't, but realistically what more could I do than not engage with the tech in my free time?
I don't think there is anything wrong with this. I don't think "stick your head in the sand" is the right analogy. I think a better analogy is "I don't like to work at an office, so in my free time I stay away from offices". If you don't enjoy something and can avoid it, I think that's sensible to do.
The real people losing here are the people who try to min/max everything in life. Who treat everything like a job to "become the best at". If you like coding, then code in your spare time. If you don't? Keep it as a job. It's actually good to be inefficient in your spare time on occasion. Otherwise your free time becomes a second job and you never rest.
And some might say "oh, sure, just stick your head in sand, that'll solve everything"
Of course, this is actually you saying it to yourself on behalf of a third party that isn't present. I've been working on catching myself doing this because my life gets worse when I make choices based on what those imaginary critics might say.
There is also some "masochistic epistemology" here; we're supposed to face the truth even (especially?) when it hurts. To avoid painful information can seem unvirtuous. But attention is finite, so you'll be better off spending it on something that makes you happier, more fulfilled, and willing to do it all again tomorrow.
I really like this bobsled analogy. I don't think it is as inevitable as the author suggest that users of LLMs will take the full ride. However I think this describes the behavior I see in the industry better than any other analogy.
But large software projects have never been limited only by how quickly an individual can produce code.
I would say the opposite. Strict opinions is what makes the “right” answer for the project clear. When teams move away from code owners for systems, concepts blur and velocity slows down a little at a time until everything is so muddy no one can make progress without weird hacks. A few sideways commits putting a square into a round hole that another team depends on and your system cohesion bleeds. It doesn’t happen instantly, just little paper cuts here and there, but eventually everyone is mired in it but can’t why.
I feel that some vibecoded software changes somewhat randomly and unexpectedly.
My suspicion has been mature vibecoded codebases will be the programming equivalent of the Microsoft Word phenomenon, “I moved my image 5 pixels and then everything shuffled in obscure ways.”
I know that it’s pretty useless to complain about moderation, but still
Merging a response article with the article being responded to in practice means moderating the response away. The article being responded to has left the front page long ago. And even if it wasn’t, one could easily believe to have already sufficiently read the comments without checking back to discover that a response article has been merged.
I’d much prefer response articles not getting merged with what they are responding to. It seems fair to merge two posts that report the same underlying even the same days, and situations similar to that.
(I saw the response article on the front page before the merge. I only found the merge after checking the moderation log, because I was curious why the post disappeared from the front page so soon.)
I agree, I'm not a fan of this merge either :/
I can't say I'm that pleased either. I recognize that moderation is hard, but my article was barely about the article this was merged with, even though it opens with a quote from it. It quotes several other articles more significantly, and does other things.
Oh well :\
When I look at some vibecoded scaled-up projects the codebases become Babel not because nobody can communicate, but because nobody needs to.
This is hiding so much potential organizational dysfunction. Since this isn't a basic need like food or shelter the complete sentence needs to define which goal is free of this need. I suspect this is how the author meant it:
... but nobody needs to communicate in order to find a solution they are satisfied with.
But that's not where software development stops! Hopefully we can all agree that this would be nonsensicle:
... but nobody needs to communicate in order to commit the business to their chosen solution.
So the useful discussion to be had relates to all of the steps between "has seemingly working code" and "the business is ready to move forward with the proposed solution." In my opinion, how one feels about those steps being omitted is a direct reflection of one's maturity as an engineer and/or the maturity of the engineering practice within your place of work.
Very well written as always. One thing did stand out to me:
You're left sitting there, parsing whether or not you're going to be rude even to ask if this is LLM generated
IMHO I don't need to bother. If someone sends me slop I can say "this is bad and I'd don't like it" even if no LLM was used in any part of the production.
If this is a new contributor, unknown to you, then yes replying to a PR noting quality issues and passing the ball to the contributor seems reasonable. Though even there some people may have a different opinion; I think it was the Zig project that had the position of wanting to be able to mentor new contributors into potential active contributors (hence wanting to deal with human not LLM and hence the anti-AI policy) (but I may misremember).
But if this is a colleague, or someone online who you haven't met in-person but have interacted with significantly online, then you have a relation with that person and you would, most likely, prefer not to be rude if at all possible, in the interest of maintaining that relationship. Regardless, you do kind of need to know how the contribution came to be, because then you can discuss, and the outcome of that discussion may have significant impact on the relationship (and that impact is desired). So do you take the risk of being rude, or do you remain conflict-avoidant and relationship-preserving-by-default? Some people prefer the one, others the other.
I guess that means I don't disagree with you, although I do normally have a tendency towards conflict-avoidance. But this is, I think, where that remark came from.
To me, there still is an important fundamental difference.
If I receive an excellent contribution from an AI or I receive an excellent contribution from a human, either way, that's very useful.
If I receive a poor quality contribution from an AI or I receive a poor quality contribution for a human, either way it isn't useful to me.
BUT... If I receive a power quality contribution from a human, I can interact with them, teach them, and they will be significantly better next time. Right now, that is not true for contribution from an AI: if I interact with it, and teach to it, it doesn't really get any better.
That is a difference that is meaningful to me. Perhaps it will change, if we figure out how to make an AI that can "learn".
And they said one to another, Go to, let us make brick, and burn them thoroughly. And they had brick for stone, and slime had they for morter.
Yikes, yeah, it makes sense in retrospect since they not kneweth that Go to was considered harmful.
On a more TFA-engaged note, though, I get it. Perhaps Kowloon Walled City would be another interesting/fitting analogy here in addition to Babylon.
The Tower of Babel is when you vibecode too much, nobody knows what's going on, and your startup runs out of money without ever shipping a product.
The Kowloon Walled City is when your team vibecodes a project, but you understand just barely enough to keep it from collapsing, and you actually end up with a bunch of internal customers. Then you hand off the project to another team, who is so horrified by what you've built that they tear it down and rewrite it from scratch.
I'm overly annoyed by the misspelling of Fawlty here.
Trite "AI is speeding up what was already there" take; but if you think about it, almost none of our tools help building understanding as teams. Of course, it gets worse when the rate of change is higher because then nobody understands anything anymore. I'm also sympathetic to the article about not understanding all of your codebase, you'll always only hold a portion of the puzzle in your head.
Looking at both angles, I think there's a place for AI to help people here. When you're building the mental model and the code, it would be nice to have Architectural Decision Records (ADR) originating from the questions you asked when using it or it interacting with you to expose the thought process. If you save it as a repo artifact, it becomes a way to interrogate the code base interactively. That'd be one way to bridge people's understanding and it's a more active way of interacting with the code as well. I guess is just good documentation practice anyways but less annoying to write down.
There's also the fact there's probably a sustainable rate of change a project can bear (depending on the team, history, management,...) and AI doesn't help you here, you can only accept that writing more code, faster, is not gonna help you.
I need to order my thoughts on this subject but there are historical bits and pieces that make sense to me: the eXtreme Programming approach of being close to the customer to have a tight feedback loop, great documentation practices to help other engineers, DDD,... all process oriented answers to knowledge sharing where the goal is dependent on the process actively being lived AKA "friction is the point".
This is only tangentially related to what you wrote, but here it goes anyway.
I found the AI generated system overviews that https://deepwiki.com/ produce quite helpful to read before diving into new codebases. It definitely doesn't capture the "why" of a codebase (not automagically at least), but I found it surprisingly helpful with the "how".
In an ideal world every project would have exquisite hand-crafted documentation for everything, but since there's often not time to do this I found deepwiki useful when looking at it as "the next best thing".
Adding something like ADRs to the repository could potentially help generating better deepwiki style system overview documentation.
Your mileage might of course vary, since we're talking about AI generated system overviews here...but I've found it quite useful so far when it comes to teaching myself or other engineers about unfamiliar codebases.
I'd be careful with that site. Very careful. Of course, the two obvious caveats apply here---I might have been holding it wrong, and that was seven months ago. But no, I don't trust it.
Fundamentally, I see two major questions that will influence how things go:
Fable is already good enough that I hope to never work with a poorly-behaved outsourced "body shop" again. I like to imagine I'm a good programmer, and Fable is already better than I am at greenfield projects. Almost anything I can build from scratch in a week of crunch, Fable can build in two hours for $50. Things that take me 6 months? I can still beat Fable, because I'm building a deep model of the code and problem space. But that's a difficult value proposition to explain to corporations, which follow incentive gradients pretty blindly and love a shiny demo. And as previously noted, the models will almost get better for a while.
What really bothers me is that I don't think this is just programming. It it were only programming, I'd be sad, but sometimes professions die. What really worries me is the next potential AI breakthrough or two, the one that allows models to take over a greater fraction of knowledge work—or anything which gives a big boost to robotics, and exposes manual labor.
I don't think we get utopia. Massively devaluing the physical and intellectual labor of a big portion of the population would not be a good plan.
The "bobsled analogy" is perhaps better known as the "slippery slope". A logical fallacy.
A year ago, vibe coding was 0%. Simon Willison has published several articles talking about his careful adoption of coding agents and the increasing trust he places in them.
From this, the author claims "there's only one place to go".
Perhaps it would be nice to ascribe some taste and judgement to Willison et al.
The bobsled analogy is not a slippery slope argument. A slippery slope argument claims that a hypothetical first step will lead to another inevitable outcome. The bobsled analogy is describing a process of dispositional change being observed repeatedly.
I've seen myself eat one potato chip and then be unable to stop eating potato chips enough times to know that I should never eat the first potato chip. I think it is pretty easy for us to agree that my conclusion about my own chip eating behavior is not based on a fallacy here.
This specific quote caught my attention.
Ka-Ping constructs a model voting machine, and decides to see how hard it would be to verify that we know it behaves correctly. To push that exploration to its furthest, Ka-Ping Yee and David Wagner try an interesting experiment:
David Wagner and I decided to insert three bugs into Pvote to see if the reviewers would find them. We inserted what we thought would be an “easy” bug, a “medium” bug, and a “hard bug” to find, and chose each bug individually in such a way that an insider could conceivably exploit the bug to influence the results of an election. [...]
We decided to insert all of these bugs in a 100-line region of a single file, lines 11 to 109 of Navigator.py, and told the reviewers to look in this region. [...]
None of the reviewers found the “hard” bug.
After retrieving the PhD thesis and the Navigator.py code in question, I can't say I'm surprised at these results... but I also don't think they generalize well to good production code.
This code looks like typical research-grade code in the bad sense of the term, failing to apply solid software engineering principles, for example:
Ironically, the first method is called "goto" (:
The function where the "hard" bug was inserted (as described in the PhD thesis appendix) is the following. Such a soup of incomprehensible logic. Fortunately, modern programming languages like Rust make it easier to structure and break down the code in terms of maintainable abstractions.
def review(self, group_i, slot_i, cursor_sprite_i):
group = self.model.groups[group_i]
selections = self.selections[group_i]
for i in range(group.max_sels):
if i < len(selections):
option = group.options[selections[i]]
self.video.paste(option.sprite_i, slot_i)
if option.writein_group_i != None:
self.review(option.writein_group_i, slot_i + 1, None)
if i == len(selections) and cursor_sprite_i != None:
self.video.paste(cursor_sprite_i, slot_i)
slot_i = slot_i + 1 + group.max_chars
return slot_i