LLMs reward expertise
5 points by gnyeki
5 points by gnyeki
Sometimes I wonder who exactly Sean is talking to. Some of the setup feels like building a strawman, especially for what I assume is his primary audience: professional software engineers. The idea that being in expert in a particular domain isn't lost on experienced engineers - this has been pretty much universally agreed upon when talking about what makes good, usable software. ("Solve your own problems", etc.) Obviously the same would be true in using an LLM. I get the idea that sometimes blogging should be "stating the obvious", but I can't help but shake the feeling it's being presented as some kind of deep insight only won after years of struggle.
I'll admit I was also just immediately turned off by the first sentence:
In the 2010s, if you had technical gaps (say, you couldn’t write CSS), you had to either rely on a skilled colleague or just hope that the answer to your exact problem was out there on the internet.
Those were the only two options? There was no specification or reference material of any kind that one could potentially read and learn from and synthesize a solution to the problem from first principles? If the only problems I'm tasked to solve are like the ones generally described here, perhaps I deserve to be automated away by an LLM.
I don't think that the ostensible audience, professional software engineers, is the actual target audience. I think that Goedecke writes for executive- or corporate-level people, executive directors, vice presidents, etc etc. I think he writes little vocabulary lessons for that level of business person, someone who needs to know (or maybe feel like they need to know) the hot new vocabulary. He writes for executives who want to think they're "not just business people".
Very much agree, LLMs are great at handling implementation details and doing boring work, but you have to know understand the problem domain to use the tool effectively. If you're unable to evaluate the outputs it's producing then you're guaranteed to end up with garbage.
I gotta say, it does feel pretty good when the model assesses the codebase, asks a good question in your domain expert terms, you think hard for a second, and guide it to the right path (which it executes well).