I think I disagree! You can always manipulate a human. It's true tho that this particular example has a myriad of failsafes, so the actual example I wrote is less than interesting.
Yes, but to make "apocalyptic" scenarios happen your AI would need to manipulate at least millions of people. So far it's not yet clear they can successfully manipulate even one, or even that "manipulate" is a coherent concept, since LLMs pretty clearly have no agency or intrinsic goals. So like... I guess if we start seeing AI systems autonomously throwing elections and starting wars I'd be more concerned? It doesn't look likely anytime soon.
