Summary - Rust Project Perspectives on AI
18 points by intarga
18 points by intarga
This whole schism pains me. I am myself so incredibly torn.
All of the wild externalities of building and using AI systems: the environment, the parasitic profit from open source labor, the economy… I could go on.
And yet, I have seen impressive security improvements atwork. The sheer amount of bugs we could find and prevent is mind blowing.
Then I go back to all the harm. And yet… :)
I am truly and utterly torn.
For you, what makes this different from other technological shifts like the Internet? Because you could point at a similar set of opportunities/concerns. (I ask this as someone old enough to have lived through it!).
For me it's the magnitude of the change, even compared with something as big at the Internet.
I don't think I [edit: was 'we', but I mean in my lifetime] have ever seen a popular technology that is so inherently centralizing. Blockchain is, despite protestations by its boosters to the contrary, but it was never this popular even at the peak. Cloud could be but in fact turned out to basically just be different flavors of renting stuff you could do yourself.
If we believe the hype, LLMs provide a unique opportunity to make it actually impossible for individuals on their own to build software at a speed and quality comparable to those empowered by big companies, which to me is... well, I've been in trouble for saying "evil" before, so let's say "counter to my values". I do not want to live in a world where Anthropic gets to decide who is allowed to be a software engineer. (And that's not to mention that if you don't believe the hype, it's in some ways worse. And there are tons of horrible negative externalities.)
Frontier models are hard because of the capital required, although even there we already have several viable players. Ironically the nature LLMs makes it relatively easy to swap them out... unlike things like search or social media, which I’d argue ended up deeply centralized.
That said, “not worse than other examples” isn’t really a justification.
I do see counterpoints. e.g. my non-technical niece recently built a decent app with AI, which was genuinely a joy to see. It's not a sufficient counterbalance to your concerns, but gives me some room for optimism.
Great question... It seems to me that the web and email and such can exist while keeping most negative side-effects at a minimum (power draw remains, sure). But LLMs, even those that show amazing use for security analysis, can't ever. Or can they?
I find fascinating how, even in a summary of a diverse position, the choice of words, tenses and semantic structure heavily bias this report to a positive stance, aligned with its author, even if we strip the part that are explicitly announced as partisan.
It is exactly the reason why it has been so hard to discuss this.
In particular, i find fascinating that the author himself talk of "feeling empowered" to do things in which he acknowledge poor knowledge, without the flaw in that logic coming to his mind.
Even less offering proof that his feeling translate to actual reality.
It feels like hand washing and blood-letting time for software. Also Ironies of Automation is something I recommend. A more complex and empowering tool demand more skilled operator to control it, not less.
Feeling empowered and confident in doing things you are not knowledgeable about in complex system should be an alarm, not something to rejoice and support!
"feeling empowered" to do things in which he acknowledge poor knowledge, without the flaw in that logic coming to his mind.
Please elucidate for those of us who also don’t see the flaw.
I point at it lower. If you do something you are not equipped to validate if it is actually done right or not, then you cannot decide if what you did is right.
And yet, acknowledging that he is not equipped to evaluate if it is actually working, he also acknowledge it makes him feel like it work and make him do it more.
It makes him ignore his own knowledge of his limits.
"Thanks to my chainmail vest, i feel empowered to run in battlefields. You can also shed your fears coming from survival instinct. Get a chain mail vest tomorrow and shed your legitimate fears"
It’s a fair point if you can’t evaluate the output, but often you can even if you can’t write it. Both by reading and researching what you’re seeing, and by testing. And of course humans can work together to test or review different parts of what is generated.