That's why alignment is such a huge issue - the immediately obvious and tempting way to "solve" that problem is to give the machines control.
The child-parent analogy stretches to describe it. A young child has an extremely fuzzy picture about what is going on - how did this house get here, why is there food in the fridge, why is the food wrapped in this clear thing, why does Dad disappear for a few hours every day, etc. At a young enough age they're not even thinking about it.
Yet it all happens anyway and the child is provided for. Even in the best case of perfectly aligned AI, that's our future if we allow it to happen (I hope we don't...)
The child-parent analogy stretches to describe it. A young child has an extremely fuzzy picture about what is going on - how did this house get here, why is there food in the fridge, why is the food wrapped in this clear thing, why does Dad disappear for a few hours every day, etc. At a young enough age they're not even thinking about it.
Yet it all happens anyway and the child is provided for. Even in the best case of perfectly aligned AI, that's our future if we allow it to happen (I hope we don't...)