What's wrong with AI?

"I told Visual Studio to build release and debug builds for x86 and arm" is a very reasonable thing to say.

This person was, charitably, pretty deep into what I think is now considered "AI psychosis" -- ceased collaboration with coworkers; started frequently arguing with just about everyone; pasted reams of LLM output in direct chats, shared channels, and emails at every opportunity; denigrating others as people he could replace with "just another claude agent". A pretty bad situation.

Yeah, very thankfully not something I've had to experience much yet. The whole LLM pronoun thing is super interesting both from a language and thought-process perspective. From a thought-process perspective, it takes me no effort at all to not anthropomorphize these tools. I use them sometimes, I get good results often, I get trash results occasionally, and I have a reasonable-enough mental model of how things work inside the black box to properly contextualize the output.

From a language perspective, things get a bit weirder when I think about it. I'm bilingual, with English as my mother tongue and French coming via French Immersion through grade school, and it's different between them.

In English, no problem at all using "it" to refer to either conventional tools or LLM tools ("Visual Studio started the build, but then it crashed" or "I got Claude doing a lit review but it got stuck because of paywalls").

In French, where all* nouns are gendered, animate or inanimate... things get a little weird. There's, as far as I remember at least, no real deterministic way for determining whether a particular noun is male or female, but after doing it for a while you just kind of... know? For inanimate common nouns there are heuristics based on the end of the word and similar, but it's kind of vague and there are exceptions. Anyway, point of that is, in my head Claude is "il / le" (masculine) and ChatGPT/Codex is "elle / la" (feminine), for whatever reason. The English part of my brain has no problem not assigning gender to inanimate objects, but the French part of my brain insists that it must be done and it comes up with inconsistent assessments there, which fascinates me. No clue how native French speakers handle that either.

Going to your specific example, I would personally not say "I had rustc work with me to produce this binary"; I would on the other hand absolutely "I had [some person] work with me to produce this binary".

I think there are some uses of "personhood language" that are fine even when applied to rustc; your example, or "rustc didn't like that code" to mean it failed to compile (or produced really bad assembly, or whatever). But "work with" in this context doesn't feel like it fits into that category.

The IMO strongest argument for not saying "Claude spotted a lifetime issue" is that it's clear that rustc isn't the sort of thing that can actually be a person, so saying "rustc spotted a lifetime issue" is obviously shorthand for a more "mechanical" statement. But Claude is much closer to the personhood end of the spectrum, so it's more important to be precise. I don't know to what extent I buy this argument, but I think it's definitely plausible.

Engineers anthropomorphizing their tools has been around for decades, if not centuries.

So we can assume one of these claims has 'corner cases' and is not always true, depending on person and situation.

But in already regulated areas (which ai is not) those cases do not count. Say you're very responsible automobile driver, you always alert and aware of your surroundings and never had an incident in your life. Say 95% of people are like that even! Speed limit sign of 50kmh still applies to you and everyone, and it was made because of the remaining few percents and totally justified (though feels unfair at times).

What I mean to say - do you think your exceptional case justifies everything else? Or we just turn a blind eye there because of cheap dopamine?

in this case, it sounds like you are describing working with the output of the AI rather than describing things that the AI produced as things that you did.

i think it could be phrased clearer what that statement is trying to get across.

I find most of these articles come from people who have very clearly never used these tools to do any serious work. And people just create straw man arguments based on how they imagine people work with these tools. It's convincing for other who hate this tech, and so you end up with an echo chamber of people who don't use LLMs telling themselves how everybody using them is just getting dumber and outsourcing their thinking to the tool.

I didn't flag it, but I assume it's because now that there's no more #vibecoding tag people don't know what to do with LLM posts they don't want to read. This article starts with

This is a rant.

Personally, I don't want to read AI rants. I don't want to read AI-anything on Lobsters. I can get that from Hacker News or the #random channel at work or even the newspaper these days. It'd be nice if there was a replacement way for me to filter out all the AI stuff so I can just get non-AI coding news from Lobsters again. I imagine those flagging posts like this feel the same way when they see a post like this tagged "culture" and aren't sure what else to do.

I’m quite sure the current climate is making you feel like you’re in a madhouse.

Yes. I feel like I'm taking crazy pills

...This lists a bunch of "issues" but ignores the biggest one:

"AI" is not artificial, nor is it intelligent.

All current "AI" are simply chat bots trained on a larger corpus and with more compute and memory resources available to any prior chatbot.

It is so overtly built upon the theft of work of others that its "created" solutions exactly match prior work without system level prompts preventing regurgitation. See regurgitated license text, regurgitated code - or outside of code: presenting unpublished work of mathematicians as its own. The ability of it to "solve" problems is directly dependent on existing work solving the same problem.

AI is not bad because of any of the sophistry or "social" problems listed in this article, it is because the only way it "solves" anything is by stealing the work of others and presenting it as its own.

Any code that comes out of these chatbots is no different from a bunch of glorified code transformers and translators being used to remove GPL, MIT, BSD, or any other OSS licenses, and the authors of that code, from the code being copied.

If we were to believe that that theft had not happened, then it is still just machine generated from a basis vector, and so is not subject to copyright, so anything that comes out is public domain and claiming copyright ownership is fraud.

I’ll grant that the bits I read are mostly accurate. Let’s leave aside that I don’t care about copyright as a legal framework.

The problem I have with all these critiques of AI that I have read is that they amount to saying the world shouldn’t change and people shouldn’t want a machine that lets them do almost any task with text or images to a mostly adequate standard. It lets experts in a field experiment rapidly and see concrete prototypes. Obviously people want this because it’s amazingly empowering. If the alternative you’re offering is “just don’t use it” that rings hollow. What is the author’s vision for how we all should be better in a world where this does in fact exist?

Pillaging the Commons for profit (which is morally/ethically wrong, especially at this scale) & resource usages required aside… I think I finally realize my thoughts & why I keep feeling conflicted. LLMs can be a valuable tool to assist workers in their pursuits not just for for “getting work done”, but also learning & acquiring new skills. They shouldn’t be as a tool that replaces workers or human interaction—but this is precisely what the products are all pitching. Even the UX leans in this direction. When I have used OpenCode (or similar), effectively they are built in a manner where the path of least resistance is that you tell it what to do & it’s supposed to do that. This can be from “build mode” instead of “plan mode” first all the way to where if you modification to its output due to errors/styling/whatever, it will replace it making it ‘easiest’ to just keep telling it to try to fix up these things. Ironically a “Copilot” would be much nicer, where it monitors your buffer for errors & tries to prefetch documentation while helping the user rubberduck their way thru the design process finding potential caveats. The folks I have seen use these tools effectively, in my opinion, & avoiding slop while still holding the reins of their projects have had to design a rig/harness/skill/prompt or otherwise that tries to get at this. The rest seem to eventually completely lose the project to the ‘vibes’ (& over time skill atrophies) …Which seems to be fine to many as things are often throwaway & not meant to last, but the folks/community aspect of all of this often more important than the code itself—which again is a bit sad/ironic since building a community that share the ideals of teaching & putting this code out for others to use/enjoy are the ones getting exploited the most by the training, which removes the license/copyright/attribution & the bits of humanity behind it.

The other issue is the ‘hype’ where folks, even as the ideas have matured in the space, are still jumping in too headlong. Is all of this good for us in the long run? …Or are we being forced to as maximizing profit/shareholder value is prioritized.

My favorite part here tho has been the flipside: videcoded decomps of proprietary software/firmware to turn these moats into something users can take back control of. But I think it’s inevitable that there will be new verbiage, codified in law, that allows the corporations to copy open source & free software, but restricts access in the other way. But over time they might straight up lose as folks flock to the free alternative (even if ‘free as in beer’ is the primary motivator).