Language Models as Thespians
9 points by jstrieb
9 points by jstrieb
It might be more accurate to compare LLMs to script writers, but comparing them to actors is easier to understand and discuss.
I think it’s not only more accurate, but shines more light on how AI assistants can generate usually-correct statements even though they don’t “know” anything.
When you’re talking to, say, ChatGPT, you’re not talking to the LLM. You’re contributing dialogue to a story that the LLM is writing. These story generators are usually tuned to produce one kind of story, slice-of-life fiction about the characters User and Assistant.
If Assistant provides genuinely correct responses, it makes a better story, so the story generator usually learns to write correct lines for it. But correctness is only ever an instrumental goal, so sometimes other goals get prioritized (e.g. Assistant needs to be portrayed as smart and helpful, so it shouldn’t say “I don’t know”). And other times, the subject of an episode is just too complicated for the robot author to understand, and it resorts to taking “creative liberties”.
But correctness is only ever an instrumental goal, so sometimes other goals get prioritized (e.g. Assistant needs to be portrayed as smart and helpful, so it shouldn’t say “I don’t know”). And other times, the subject of an episode is just too complicated for the robot author to understand, and it resorts to taking “creative liberties”.
I’m out of my depth here, but is this really a fair characterization of LLMs? You say they don’t “know” anything, but how would they then determine that they can’t produce a truthful answer, and choose to take creative liberties instead?
From what I’ve seen, LLMs seem pretty indifferent to veracity.
how would they then determine that they can’t produce a truthful answer, and choose to take creative liberties instead?
They don’t. It’s creative liberties all the way down: it’s often said that they eventually start hallucinating, but that’s not true: they hallucinate from the start, they just do a better job when they can extend the text with a high probability item out of a pool of improbable choices (one 0.9 in a sea of 0.2s), instead of having to choose one out of lots and lots of 0.4-0.5s (numbers may not be representative.)
It’s still the same mechanism of “No idea what I’m doing, but that word looks likely to be a good next choice.”
I’m out of my depth here, but is this really a fair characterization of LLMs?
There’s really no accurate characterization of LLMs that aren’t just plainly describing what they are, namely:
LLMs are fundamentally just complex statistics. All the improvements to them are just there to make those statistics more correct. Words you don’t want to show up are made less likely to be selected. Words you do want to show up are made more likely to be selected.
We’re back to Plato’s Republic Book X, when Socrates asked how it was that Homer wrote about so many things—war, diplomacy, ship building, the gods—without actually being an expert in any of them. Maybe Homer was an LLM?
If you want a comparison, an LLM is someone trying to bullshit their way through an interview. I’ve had interviewees know enough to sound plausible, but not enough to be consistently right. However, they’re terrified that if they admit they don’t know, they won’t get the position, so they make stuff up and hope you don’t notice.
LLVMs don’t have motivations, but they have the same tendency to bullshit.
I enjoyed the “finding the most useful metaphor” angle; as a fellow “designated computer guy & self-proclaimed skeptic” it’s a question I expect I’ll be pondering for a while still, so always appreciate reading people’s take on it.
Only bit that gave me pause:
The web developer persona prompt generated better output than the plain request.
Did it? 😐 The plain prompt produced
while the webdev-persona prompt produced
Granted, (a) I’m no marketing expert (b) the persona’d prompt aimed for “simultaneously trendy and timeless” and “memorable”, so the persona output surely is better depending on the metric 😶