A Language For Agents
8 points by hongminhee
8 points by hongminhee
The interesting bits in this article for me are the managing of (side-)effects and fast tooling(type checking and testing), because it's something that a lot of mainstream languages also struggle with.
For outside-of-work projects I use Unison, and even though you lose greppability, you gain really fast developer experience(test caching, no build step, abilities to handle effects), I'd imagine an agent to be able to take full advantage of that. But it seems that plain-text beats it anyway; an exact hashing containing the dependencies seems to be less useful to an agent for now?
I'd imagine languages that caters significantly more to agents(in its current state) than humans to turn out similar to serverless platforms. Can get messy, but easy to get started with and get money from.
I hope that agents get better tooling for editing files. That files can't be edited line-wise without context affects a lot of languages, and goes far beyond programming langs. Instead I wish code transformations were done with an interface similar to ast-grep.
And I guess sed has to be denylisted entirely.
Instead I wish code transformations were done with an interface similar to ast-grep.
It's surprisingly hard to actually do reasonable edits with ast-grep. Turns out, you need to read stuff into the context anyways to understand it, and LLMs have an easier time doing local text edits than to then map what they see to an AST.
When generating code in a language and the model was
What extra resources does it use in (2)? Does it eat from the context size? Or is it something different? (some kind of "working memory" that's different from "context" maybe?)
What kind of differences should I expect between (1) and (2), for the same task?
It will be a tragedy if LLMs create yet another roadblock for creating and adopting new languages.
That's hard to say in general because it's quite a complex and nuanced question. For a really long time I was assuming that the cost of in-context learning is so high that it would punish any innovation in programming languages. The reason I came around on the other side of it is that most of the token wastage does not come from throwing docs into the context, but the agent making detours on a task.
For instance you can build yourself a tiny programming language as an experiment that works very well with a a specific model and excels over one that the LLM has been trained on, because it's better constrained and the tooling is more token efficient.
If you want to make a massive detour from any existing language this will be tricky. But for instance you can do something like "This is a language like JavaScript but it has no classes and all functions are pure" as a starting point and walk from something familiar. Then if the LLM actually emits code that is in violation of this, you still parse it down and indicate with errors what the agent should do. This is a very basic example, but there are many way in which you can bootstrap yourself from familiar syntax up. There are always going to be questions of where the innovation should happen vs can happen.
Did you (or OP) happen to ask an LLM what it prefers in a language? I did, and interestingly references many of these points.
I was honestly surprised at the reply, because (like this post), it requests as much explicit information as possible so that reasoning can be done locally. One thing it stressed was having an Effect system, which makes a lot of sense - the biggest gap that I've had with using LLMs so far is that it just can't see much of the implicit context of a system.
I thought maybe as long as there was an LSP, or some tools that it could call out to, the language shouldn't really matter. But, it may be the case the local reasoning (which is always valuable) wins out in the end.
Much of what I’ve said boils down to: agents really like local reasoning.
What's been true for humans is now true for agents!