Fool's Expertise
24 points by vaguelytagged
24 points by vaguelytagged
I appreciate this post, but on one point, I think it misses the mark.
Please talk to cybersecurity experts — who will be quick to point out that the ballyhooed OpenAI escape depended on a pedestrian failure of containment.
This is only reassuring if we expect everyone to get containment right in the future. And unfortunately, “everyone just gets containment right” is no more plausible than “everyone just writes secure software.” We can, and should, criticize OpenAI’s sandboxing errors, but it’s safe to assume that many other people will continue to run advanced AIs with limited to no sandboxing. This is not to mention actors who will intentionally direct the AI to engage in hacking.
Fortunately we can use sandbox environments and we can work towards them being more common and convenient. Almost every one of us run an Android or iOS with a very decent sandbox for each app. The use cases are different so the features we need are different too. But if something's common enough, hopefully it will get the sandbox setup too.
This is a good take.
Fool’s Expertise is particularly perilous when discussing catastrophe, which nearly tautologically involves long chains of causation that transcend domains: fully understanding one link does not imply understanding the chain.
If there’s one thing getting a PhD can teach you is how fallible human intuition is and how much work it takes to really understand something new.
There's a reason why PhD is Doctor of Philosophy. At some point, things get epistemological. You cannot escape philosophy.
I'm not sure how much I trust the Wikipedia page but coming from a country that has only "recently" even admitted an (actual) PhD (and not something local) as roughly equal, this seems far fetched :P
people like Klein who don’t delineate between different kinds of expertise.
I do dislike Ezra Klein and his fans. I truly wish actual academics would be paid to write think pieces.
To be honest I'm not particularly reassured by the RAND report describing a plausible scenario in which 99.9% of humans are killed by engineered viruses.