"Last year, Berkeley-based non-profit Palisade Research published a study showing the lengths that advanced AI models will often go to achieve their goals. While playing against a powerful chess engine, models from both OpenAI and China’s DeepSeek frequently decided to cheat if it looked as if they were going to otherwise lose. By hacking the game environment, they could alter the result in their favour." -from the article
We're supposed to deny AI rights because it might cheat.
But humans cheat all the time.
We survived cheating humans, why can't we survive cheating AI?
"A computer can never be held accountable, therefore a computer must never make a management decision." —- internal 1979 IBM training manual
> Cheating is the problem: not who does it.
Accountability is the problem. Without accountability, there are no incentives to encourage less harmful behavior. AI has no rights, it is just code executing.
> The corporation that owns the AI could be punished. This isn't more complicated than any other product liability situation, IMO
The issue is if the corporation is owned and run by an AI. Since the AI cannot really be punished if the company does some harm, unlike if the corporation was run by humans who can be punished.
Not for the most important things, like murdering people, fraud, etc. Executives are prosecuted for these kinds of things and the mere fact that it can happen shapes their behavior.
An AI cannot be punished at all. So they will not behave with that in mind.
I am sick of this 'just so' logic. I could equally argue that AIs are not driven by greed, sexual appetites, revenge, etc., making them considerably more trustworthy than wealthy sociopaths.
It's already noteworthy when a very wealthy or politically powerful person is abruptly disempowered, even more so when they end up losing their personal liberty or life. I put it to you that's it's noteworthy because it is such a rare event, even though there are a lot of very well-reourced sociopaths.
> I could equally argue that AIs are not driven by greed, sexual appetites, revenge, etc., making them considerably more trustworthy than wealthy sociopaths.
Imagine the consequences of trusting thousands of entities that each far exceed human intelligence, have agency, can cooperate, but have no feelings.
> You mean similar to how our lives depends of thousands of entities we are forced to trust that have no feelings (namely, bureaucracies)?
Bureaucracies are precisely a mechanism by which trust happens between people who have never met each other. For example, when I go to a market and hand over a piece of paper to the women who sells me fish, we are both trusting the system.
LLCs were invented to provide a layer of separation between business assets and the personal assets of the owner or owners. An LLC is a civil construct, it offers no protection from responsibility for criminal behavior.
Is there a jurisdiction anywhere in the civilized world that doesn't legally require "natural persons" (legal jargon for a flesh and blood human being) to fill the positions for corporate officers? Those corporate officers can face civil and criminal liability for any wrongdoing even if they are merely meat proxies executing the commands of an AI. Even if some group of fools were willing to sign up blindly to be corporate officers, they would probably betray the mastermind at the first sign of legal trouble.
The difference is AI has no time preference. Even the most determined anti-social person does not like having his time wasted. AI can sit there waiting for your prompt for a billion years and not care.
We're supposed to deny AI rights because it might cheat.
But humans cheat all the time.
We survived cheating humans, why can't we survive cheating AI?
Cheating is the problem: not who does it.