GPT-6 Astra Solves a WWI German Radio Cipher
12 points by rajtilakjee
12 points by rajtilakjee
This article would be a lot more interesting if it included the prompts, transcript, and generated code from the session with the LLM.
It is irresponsible to misattribute these accomplishments to the AI agents. Every time, at least one person is involved in prompting and guiding the process. The headlines should be "[human] solves [challenge] using [tool]." The article author's use of anthropomorphizing phrases like "Astra's hypothesis for" and "Astra felt compelled" do not leave a good impression on me. I agree with @simonw that without seeing the prompts this is uninteresting.
I totally agree that attributing the title to an LLM doesn't make sense, but I also have another question:
Could they do the same with GPT-3 or a weaker LLM in general? If yes, then is Astra just an accelerator? If no, then where is the line between the tool is "effective enough" to worth a bold credit and not?
I disagree with the black and white stance here. I do agree that we sometimes do a bad job of this. Some recent advances in math have been headlined as "AI Solves ..." when there's a human driving. But sometimes a human writes a prompt and the AI really does the work autonomously. I don't think it's anthropomorphizing to say that my dishwasher washes dishes or a car wash washes cars. If a model is prompted to solve a problem and one-shots it, I think it's fair to say the model solved the problem. And if a model generates a hypothesis, I think it's fair to call that the model's hypothesis. That doesn't mean it's "thinking" or human-like. I also think it's totally fair to call an AI-generated idea uninteresting until a person stands behind it and takes ownership.
Astra's hypothesis for why this particular message was previously unsolved is that “TRUPPENVERSCHIEBUNG” was used as the key starting on December 9, 1918 - whereas, as noted above, this message was transmitted earlier, on November 27, 1918. The reason for this discrepancy is unknown.
I don’t know how such keys were managed (how/when distributed, how often rotated, &c.) but it sounds like a tired transmitter might have just accidentally used the next key in the book by mistake?
I presume this also makes the solving of the cipher rather less impressive—that it was probably just trying all the known keys and found a plausible match. It wasn’t solving a cipher, it was decoding a specific cipher text for which the key was not known.
Yeah, the title makes it sound much more impressive than key reuse.
Could be a good avenue to check for some other encrypted messages though.
To me an interesting example of storing all encrypted messages for a long time and then some day you’ll be able to crack and decipher them
So..what??? I don't get this type of posts. Still not AGI and never-ish will be, AI;DR as always.
I don't get this type of comment.
I like old ciphers and while I would probably enjoy reading about it more if it had been a (team of) people, I still think it's interesting to read about.
I've so far refused to use LLMs for anything generative, still write code by hand, and am terrified of what lies ahead. But I still think it's interesting to see what these models are used for and how well (or not) they work.