Claude Says
33 points by lr0
33 points by lr0
I agree with the overall sentiment. One nitpick though
AI is the reason our software stack is overengineered to the 10th degree
I can confidentially state that this is not the case. Software for a long time now had been over engineered in many areas for a variety of reasons. Including, but not limited to, (un)intentional technical gate keeping, abstractions over abstractions over abstractions by companies looking to or selling the idea of "accelerating" development (llms in many cases extending this trend),and much more.
The situation we are finding our selves in now is not a cause. It is another symptom of a system that has been screwed up for a while.
I feel like "over-engineered" is a misleading term here. It implies that it was a deliberate engineering effort, when often it was just cancerous growth. No engineer would think "Oh, the Golden Gate bridge is aging, let's build a brand new smaller bridge with less capacity on top of it to resolve the problem!"
The worst software constructs I've seen were a product of ignorant, rushed and/or disempowered developers. So, the opposite of engineering.
Imho it can be both as I have seen what you sketch out for sure. But don't discount the over engineering done by highly technical people who (unknowingly) like making things complex. I also mentioned technological gatekeeping, I have seen projects first hand sold to management as being complex and needing specific knowledge in an attempt to keep "rifraf" at bay.
"Oh, the Golden Gate bridge is aging, let's build a brand new smaller bridge with less capacity on top of it to resolve the problem!"
I distinctly seem to remember instances where bridges have been upgraded with an extra layer below on top.
At the end of the day we can nitpick about semantics around engineering all day.
But my overal message is that we have been converging on this overly complex mess from multiple directions for quite a while now.
And speaking of overengineering bridges, the Tacoma Narrows Bridge was not overengineered when first built. It did not end well. But that's bridges. This is software. Was the Apollo Guidance Computer overengineered? Given how harrowing the Apollo 11 landing was, I don't think so.
Yeah exactly -- prior to LLMs, software was already bad, mostly due to bad incentives:
I think LLMs can and do magnify those problems, but LLMs can also help you do the opposite, if you want.
Often I am 80% confident that something is wrong, to me the appeal of Claude is that I can quickly check, and be certain that it is wrong and how it is wrong.
I tend to agree that it is annoying if people state "Claude says...", however I am not sure what a good alternative is:
The third bullet point. We (humans) have done that for thousands of years before LLMs, how is that not a good alternative? It's how one builds knowledge, expertise and expands their abilities.
Before I get hit with an arbitrary argument like "why would I waste an hour on learning about an obscure problem that I won't ever encounter again" -- it doesn't matter. When researching certain topics, you're not only researching them specifically, but also improving other soft abilities like critical thinking, perusing different sources, fact checking, etc. Even in a perfect world where the technology wasn't developed with complete disregard for the environment and the society, I still wouldn't outsource my thinking to it.
I still wouldn't outsource my thinking to it.
Huh, yet you want to outsource the thinking to me: I should research the "obscure problem", provide human feedback to you, improve my skills etc. Perhaps I should then also fix the problem or rework the implementation because I already know it so deeply?
I'm not sure where and how I outsourced my thinking to you, but yes, I ask people questions about domains they're familiar with and/or competent/interested in. They do the same in return. Human-to-human collaboration is much better than human-through-human-to-LLM collaboration.
You can't know this but I recently had the situation, that during code review I pointed out how design and implementation of a feature are flawed. I checked if the flaws are real, suggested a better approach and got stuck improving the design and low-quality (maybe slop) code. I definitely had the feeling that my coworkers outsourced part of the thinking, that should have been done during design, to me.
I hope that explains my previous comments a bit.
learning about an obscure problem
Nope, it doesn't even count as learning: 80% sure usually means that all plausibly-retainable knowledge is already learnt, and one night's sleep later after looking up the lawsuit-proof evidence one will not be any wiser.
When researching certain topics, you're not only researching them specifically, but also improving other soft abilities like critical thinking, perusing different sources, fact checking, etc.
This assumes that the person pointing out is not being pulled out of researching something they are not yet sure enough about what's going on to be able to vet generated output.
Used to be you could say : "I searched and found this : [link]"
This was sort of helpful, because searching took at least a little effort, established some sort of common baseline as articles or stackoverflow responses normally establish question context, as well as not implying that the responder really knows what they're talking about. Still, it was not entirely polite, as your interlocutor could have, and probably had, searched themselves.
"Claude says" seems strictly worse, but I can't express why very well in communicable terms. I would rather people simply didn't, and just said "I don't know, ask claude", rather than proxying it for me.
The real issue with both cases is: it takes knowledge to vet an explanation, be it linked or generated. «I found this and it explains in a way I endorse with more detail and polish than I care to write myself» is a pretty big value add on top of searching — or on top of prompting (but for people who read faster than type, this is cheaper than to provide an explanation personally). However, one cannot reliably signal how attentively one reviewed a text. So the often most efficient use of expertise, looking at a few available explanations and picking the one that actually makes sense, becomes impossible to use properly.
the "claude says" phrasing is as fitting as the link example; In both cases, the searcher needed to gather an understanding of what is relevant and what to search. The result is then provided to you without having to do such contextualization, gathering, and filtering.
The "claude says" approach instead offloads the gathering & worst-case the filtering. But the contextualization still occurs. If both parties have the same context built up, "ask/search it yourself" makes more sense. If not, "ask yourself" discards the context/evaluation the searcher developed to ask/search relevant queries and is strictly worse.
Often I am 80% confident that something is wrong, to me the appeal of Claude is that I can quickly check, and be certain that it is wrong and how it is wrong.
I find that this is most appealing when it's my own idea that might be wrong. My local LLM is a surprisingly capable critic of my ideas. And it is very good at finding the one nasty corner case that will send my entire design back the drawing board. Of course, I'd find that corner case eventually, but it might be 12 hours of work later, or perhaps when I'm working on version 2 and I find out I committed to a bad API. Like any outside technical criticism, I'll decide that half of it is probably wrong, and filter the other half through my own knowledge.
But none of this excuses either outsourcing my own thinking, or just parroting Claude at my coworkers.
use it if you must, then use the terms and sources it provided as a starting point to search and integrate information so that you build your own understanding, then explain that understanding
Honestly I'd be fine with exactly what you said... "Had a gut feeling I couldn't articulate, Claude said ... in response to that, here's where I agree/disgree with that response."
I think that last part probably means some of your point #3 too, as if you couldn't articulate your thoughts initially, it seems unlikely you'll be ready to meaningfully critique the response right away either.
What a narrowminded way of thinking! Of course it's stupid to copy-paste something without thinking. But if I copy-paste something and send it to you, it probably means there is some value in it for you?
If I copy-paste a search result from page 6, would you also rather just have my search term? Or, if it comes from a book, the name of a book that I got my knowledge from?
If you say "just give me the prompt instead" it means you don't understand how LLMs work. Unless, of course, you are as content with a lottery ticket as you are with the prize.
That's actually a really good point and now that I think about it... being able to steer Google or DDG or whatever to actually find the thing I'm looking for has been a pretty well-honed skill of mine for a while. I'll often share both: the link I found with an excerpt + quick explanation, plus how I found it if there were specific search keywords that needed to get added to surface the result.
And that aligns well with how I often end up using LLMs: it's not the first prompt that gives useful results, it's the 5th prompt after steering the whole session in a specific direction.
Exactly. Look, if you're gonna send me an AI-generated text, can't you just save us all the hassle and give me the prompt you used? I won't be happy, per se – I did want a text by you-the-human – but if you don't wanna take the time to digest your thoughts into a form that I can read, then at least let me have the essence. But I gotta say, if you don't have the time to digest your thoughts, are you sure you really have time to communicate with me?
A keyword list is almost better than a keyword list turned into long-winded prose by some AI.
The same counts for anything created using GenAI, be it images, code, writing, audio or whatever else. Do not send me anything created (partially or fully) using it. I'm only interested in what you have created.
One day people will notice that «search bubble» also applies to just sending the prompts…
(I'd say the useful option is of three parts: prompt, the output, and how much vetting of the output has happened; OK if the last part is literally precisely zero, then prompt is enough)
So I've heard a lot of people say this:
can't you just save us all the hassle and give me the prompt you used?
For context I do not use LLMs for writing prose that I'll share with anyone, but do use them as a research assistant, code reviewer, part-time data analyst, and occasionally for writing code (which does not get committed without significant review).
Here's where the "just give me the prompt you used" thing has never made sense to me: I essentially never one-shot anything. If any LLM output is going to leave my machine in some form, it has gone through multiple rounds of iteration and review. High probability that it has also had access to the large volume of private notes that I've accumulated over the last 2 decades.
A keyword list is almost better than a keyword list turned into long-winded prose by some AI.
100% agree with that and honestly lol that has been how I've often answered questions in the past. No big prose trying to explain something, just "look up 'frequency-domain decomposition'. It's from civil engineering but it's probably a good technique for what we're doing." and then being available for follow-up if need be.