Multi-agentic Software Development is a Distributed Systems Problem (AGI can't save you)
18 points by kirancodes
18 points by kirancodes
Gwern argues against "complexity" being a barrier for AI, Kiran argues for multiple agents being (inherently) a certain class of problem. Being distributed has nothing to do with being complex (in a complexity theory sense).
For the future: Please expand on why you think a different piece of writing is relevant. Simply posting a link is low effort and could be considered trolling.
Being distributed has nothing to do with being complex (in a complexity theory sense).
https://dl.acm.org/doi/10.1145/800135.804414 serves as a counterexample.
Right, communication complexity exists. Sorry for being imprecise, I should have said: in this argument by Kiran, multi-agentic systems being distributed has nothing to do with Gwern's object of ire, which is arguments against AI in terms of computational complexity theory.
I know you got a kick out of linking a really old paper. Let's engage with the texts, though, shall we?
The linked paper states that we need approximately log(n) bits of information to be exchanged for an input size of n, if we want to compute a boolean function distributed over two people (deterministic 2-way complexity). Great, that would definitely be a point for Gwern, log(n) is not a big, interesting increase in complexity.
What does the FLP paper referenced by Kiran say? You can look at it here: https://dl.acm.org/doi/10.1145/3149.214121
In this paper, it is shown that every protocol for this problem [distributed consensus] has the possibility of nontermination, even with only one faulty process.
So if a protocol does not terminate, would you not agree that it is irrelevant how big-O complex its algorithms are?
So if a protocol does not terminate, would you not agree that it is irrelevant how big-O complex its algorithms are?
Well, that's why you focus on productivity/coalgebraic reasoning instead.
In fairness, this is one of those things which is only "well-known" in PL theory.
Yes indeed, the idea of consensus protocols is in essence about the whole system staying productive while tolerating failures in distributed computation (be it faulty results or nodes going offline).
So ... we're not really in disagreement, then?
I mainly lean towards the opinions held by Lamport, Turner and McBride that most of this complexity is extrinstic, if that helps clarify things.
Gwern is not a reputable computer scientist. His main fallacy is thinking that classical space-and-time analyses do not apply to AI because AI is magic. Previously, on Awful, I've put more complexity-theoretic LLM analysis into a single paragraph than he brings forth in that entire essay. Also, previously, on Awful, we've analyzed his ideological refusal to understand the basics of chaos theory; he believes that AI magically overcomes chaos theory, you see.
When you say AI, do you mean something like the traditional AIXI definition, or something more modern, such as https://arxiv.org/abs/2510.18212 ?
I think it is really helpful on my end to have definitions nailed down before engaging in a productive discussion, which is why I am being so pernisckety here.
What is "AIXI"?
It's a formalism developed by Marcus Hutter (aka the guy behind http://prize.hutter1.net) in order to analyse certain formal properties of machine learning algorithms.
Mind you, it was developed prior to the AlexNet revolution in 2012, and it's not computable, so it's only somewhat useful as a North Star.
When I say "AI", I mean a field of study. It's the same field of study denoted by "robotics" or "cybernetics". There's no particular architecture or approach that you're going to be able to find which can somehow subvert the numerical limitations inherent in chaotic systems, though, so it shouldn't matter much. Previously, on Lobsters, I've pointed out that "AI" is nebulous for most people, but it is a concrete history for machine-learning practitioners today.
I agree that there are numerical limitations inherent in chaotic systems, but I suppose that my concerns are that given enough capital (whether intellectual via hiring of "star ML researchers" to add yet another weird trick to an internal stack, or material via provision of more compute), you can achieve a transformational impact on the economy?
More concretely, I believe that the arguments in https://www.versobooks.com/en-gb/products/636-the-people-s-republic-of-walmart also apply to solving multi-agentic software development i.e you can have your cake and eat it too in practice, despite the formal bounds of chaos theory.
Then you will be very frustrated when no finite number of sensors allows you to perfectly predict a game of pinball, let alone an economy!
I don't think you need to perfectly predict either the game of pinball or the economy though in order to achieve transformational economic impact.
You just need to be able to distinguish between ergodic and non-ergodic events!
https://arxiv.org/pdf/2001.10488 is relevant here because Taleb has actually done the elbow grease regarding working out the epistemological implications of statistical assumptions used in mathematical finance.
This is a great read. Thanks for sharing.
On the consensus front I’d argue it’s not the goal - rather the goal is meta-stability.
If you replace "agent" with "human" does the explanation, formalism or conclusion change in a meaningful way?
Humans possess general intelligence (citation needed), we have more than half a century of practice and research into software engineering, and still teams of highly skilled humans regularly struggle to produce working software that fulfills the needs of the users.
right exactly, the point is that agents getting smarter doesn't help. The problem is that a lot of people are conceding to the fact that we may need to consider a world where agents are smarter than humans, so existing evidence on human performance alone is not convincing. My point is that irrespective of intelligence, these problems will persist and should be considered.
The problem is that a lot of people are conceding to the fact that we may need to consider a world where agents are smarter than humans, so existing evidence on human performance alone is not convincing.
I missed that nuance, probably because I consider a world where agents are consistently smarter than humans feels remote to me.
Right, just to clarify I don't hold a strong opinion about whether that's possible or not! I just find a lot of the questions that I care about, such as designing for consensus etc. are being ignored by people claiming that AGI will solve it, and the blog post was more to provide a structured argument why irrespective of what you believe about model capabilities, these problems are important to tackle!
To be fair, multi-human software development is notoriously inefficient.
Off Conway's law alone you get O(sqrt(N)) output from N contributors, though reality is likely even worse. The reason you hire more than 3 people is to reduce the bus factor, and to set up a pipeline of juniors to replace senior developers when they leave.
Accepting the risk of not doing so is a big part of why small startups are able to compete effectively with larger, more risk-averse organizations.
Consider some differences, though:
I'm not saying that's enough to overcome the challenges. Merely saying the situation is fundamentally different.
i think you, like the first commenter referencing the gwern post, are still too-focused on intelligence. those things might be valid reasons why humans are not perfect at building software in the large; that does not mean that the distributed systems problems laid out in the post won’t still be hard limits on multi agent work, just as they would be for humans even if the two things you mentioned weren’t true.