How does Pangram work?
14 points by gnyeki
14 points by gnyeki
Key sentences:
Perplexity analysis has a lot of problems. It’s not a good measure of the thing it’s trying to distinguish, because human sentences are frequently just as canalized as LLM prose. So predictable or structured human writing is often falsely flagged by perplexity detectors
That’s how we train Pangram: for every element of our dataset of commercially licensed human-written texts, we generate an AI double that is as close as possible to its source. The doubles are the same length, same tone, same topic, but AI-generated. We then train the Pangram model on those human-AI text pairs.
our dataset of known human text is drawn from 2021 and earlier, before AI was loosed on the internet.
Project idea: collect text pairs, train a classifier like pangram and use it inside a browser extension that flags if a page is certainly AI generated.
You mean like this one?
More generally, though, I was going to install the Pangram extension but it requires a $20/month subscription....
No, open source and importantly with the classification running locally. (Firefox has local translation models, a classifier should be less resource intensive than this.)
I wonder how much Adversarial learning AI companies use and how quickly it'll become obsolete