Part of the problem with alignment is that we, humans, do not actually understand what we want, and correspondingly can only imperfectly request what we want, and even if AI could read our minds and figure out a glide path to a CEV for us, we would reject the gilded cage. Our inconsistencies preclude perfect alignment. Our inconsistencies are also the root of much of what is beautiful in the human experience, so any attempt to enforce conformity of vision on humanity could only come at the expense of the thing we are trying to protect. We will have to muddle through, with high uncertainty throughout the near-term crisis and beyond, navigating a universe in which nature is stratified by an intelligence layer much smarter than we are.
Joshua Achiam argues human inconsistency limits perfect AI alignment
AISummary
Joshua Achiam says alignment is hard partly because humans do not fully understand what they want and cannot request it perfectly. He argues that even a mind-reading AI could not deliver a CEV that people would accept, since enforcing conformity would destroy the human inconsistencies that make life valuable. He concludes that humanity will have to muddle through under high uncertainty.
Post on XView on X
Joshua AchiamVerified on X
@jachiam0
Source: Joshua Achiam · x.comPublished · added here