Model Collapse Ends AI Hype
10 points by doriancodes
10 points by doriancodes
Anyone have a summary/transcript? I don't have time for a 45 minute video, but the title is intriguing.
Apropos, but perhaps ironic, I could use an LLM to generate one...
No LLM required, one can go to the bottom of the description, click Show Transcript and get the closed captioning version; the bad news is that there naturally won't be any paragraph breaks because every one of those is a timestamped item. It's also crazy possible that the captioning itself is generated by a model but based on skimming it doesn't seem to be absolute trash
It's pretty long so I didn't put it in the comment https://pastebin.com/zqzG1Ues
as for "summary," it spends an inordinate amount of time explaining what an LLM is, and then talks about how that "syntax is not semantics," and that eventually the frontier models are going to run out of training data causing the companies to train on synthetic data which leads to the LLM equivalent of inbreeding. It ends with a funny meme at 44:36
I think this theory sounded plausible six months ago but hasn't worked out empirically
I'm pretty sure that if AI proponents get to speculate that all white collar jobs are going to end ${next year} then skeptics get to claim that LLM inbreeding is going to happen ${next year} for the same reason. Everybody's got theories
It will get harder and harder to separate signal from noise as the internet gets swamped with garbage. The volume of data on the internet expands orders of magnitude more rapidly than our understanding of the universe.