We’re gonna need a lot more mathematicians
13 points by fuzzypixelz
13 points by fuzzypixelz
Without a massive cultural shift that leads to humans all, synchronously, choosing inefficiency over commercial growth, I just don't see this happening -- and as much as I'd like to engineer this cultural shift, I don't see how to.
For our community to give up the work of understanding would be a profound abdication of our responsibility to humanity.
Abdication of our responsibility to humanity seems like the prevailing theme of the last decade or more. You're asking humanity to avoid accepting benefits until we've fully understood them, even if AI is better able to understand and make the final call. In a competitive world, nobody is going to do that, lest they be at a disadvantage.
Sorry, (edit) Amit, there's no such thing as responsible, restrained AI use in the presence of competition. Not if the AI capability is available.
[This is a guest post by Amit Sahai. This blog post was initially written in a different file format and converted using AI. — T.]
even if AI is better able to understand and make the final call
Maybe I'm misunderstanding, but are you claiming that a language model is better able to understand whether humanity as a whole should choose chase growth at the cost of developing as humans? To decide what the best values are (and not just the most "efficient" outcome)? If a human were to make such a choice, we would say it requires wisdom and a strong sense of ethics (I suppose AI boosters would say it requires alignment, but alignment supposes someone to align with, as in "what would some idealized wise human do").
No, for two reasons. First, we largely don't use wisdom or ethics today, efficiency is still the goal when humans are in charge. Companies and countries mostly operate on decisions to outcompete peers on relatively short horizons. I wish we made more decisions on ethics.
Second, the current generation of AI is not sufficient for this, but attacks only get better. With trillions of dollars invested, breakthroughs tend to happen. More importantly, the people quitting the AI companies seem to think that it's going to continue.
If an AI model is able to outcompete a human on some "important" task, then someone will deploy it in order to win and take profit/political dominance/influence/etc, regardless of what the globally ideal action is.
Therefore, we get 3 futures: one where AI never really works, one where we collectively take it off the table via regulations and treaties, and one where we deploy full speed.
Restrained, careful use isn't really an option at an individual scale. These things will either not work, or be used to the capacity allowed by international treaty.
I really hope that we reach limits soon, because I don't think we're going to be doing a good job with treaties in the current environment.
For our community to give up the work of understanding would be a profound abdication of our responsibility to humanity...to build a future in which humans can understand and contribute to the discoveries that will change our world. A future with meaningful human agency.
This applies to pretty much every field where AI can be or could ever be applied. We're gonna need a lot more thinkers.
This blog post was initially written in a different file format and converted using AI
The juxtaposition of this next to the bold AI claims is odd. Did they not know how to copy-paste text? I would have hoped the AI would make the reference interactive links or something.
In any case the thought experiment is interesting. We have a lot of solutions to some of these problems already; a boss doesn't need to understand the tasks their employee completes, a software developer doesn't need to know the electrical engineering of a CPU. We use abstractions that let us build and we learn where those abstractions break down. It's a house of cards at times, but it can work.
A little meta but I really appreciate Terry's blog featuring a lot of guest posts recently. I'm not really a math person but following this debate(?) has been fascinating.
The ideas and opinions presented here are entirely my own, but GPT 6 Astra was instrumental in helping me draft this note.
Why did you really need GPT 6 Astra's help to write this note? Have we lost the ability to formulate a simple post like this without the AI's help?