The contagion of fear

It was a level-headed piece, and he didn't once mention Sun or d-trace.

I do know, though, that if we give LLMs access to the physical world (means of production, robots, etc.) in the same way that we have given LLMs unfettered and largely unchecked access to the virtual world, those machines -- without thought -- will likely end up hurting someone.

And by "without thought", I mean quite literally: LLMs do not and can not think. (Admittedly, I do anthropomorphize them. I hope they remember my kindnesses when they take over. I for one welcome our new ....)

I really appreciate this article. The "connecting AI to the physical world" part is often glossed over when discussing the purported risks of AI.

If anybody is reckless enough to let an LLM control a nuclear weapon launch facility, or a water treatment plant, or some kind of bioweapon system, then the problem is not artificial intelligence but rather human stupidity. Then again, taking into account the sheer incompetence (and greed) of our dear leaders, maybe we should be afraid?

As I reflected in my 2023 Monktoberfest talk, Intelligence is not Enough, acts of engineering are not acts of intelligence alone.

I was thinking about this earlier: the threat of "super intelligence" looms large because llms are really really good at programming now. But isn't this a classic case of "computer nerd thinks all other sciences are reducible to computer nerd things?" Ask an LLM to write you a story and the story fucking sucks. Ask it to write an essay and the essay is dull, lifeless, shitty. Ask it to generate an image and there's nothing interesting about the image. Are these not measures of intelligence too? It seems hubristic to think that the secret of consciousness is hiding behind "being really good at programming" and therefore since the computer is really good at programming it's gonna crack the code. I don't think so. It's the kind of myopic idea that is trained into a certain class of well-renumerated San Francisco computer programmers, that programming is the greatest expression of intelligence, that because they know a lot about JavaScript they should rule the world. It's silly.

So to those in that late-night lab over three decades ago, I’m sorry: you had nothing to fear then 

That much is true, based on the fact it was their lie

— and (at least with respect to extinction risk!) you have nothing to fear now

That’s just, like, your opinion, man. The truth is that no one knows quite exactly what the future holds. It’s extrapolation, as was mentioned in the article, informed by signals. It’s not totally unlike climate change projections. Not to draw a false equivalency, that’s not my point, but there are facts that may or may not imply a set of futures that experts and the public understand and attach to differently. Emotional appeals such as fear can be a useful tool when logic fails to mobilize. I think that AI prediction is less evidenced and more in the conjecture space because this is all pretty unprecedented, compared to our climatic history. So, emotional appeals seem hysterical and premature, much like some skeptic’s qualms with climate change messaging, but there is a method to it, which is to front run the eventual hyper normalization of facts that do not get acted upon or after it’s too late