But you could still prompt it to act like any person, including fictional and deceased ones, which proves it has no real concept of self. It's just a language model for humans to generate whatever text meets the task. You could have it run for a year as different perspectives in GRR Martin's ASOIAF to finish his books for him. And then what, award it with Jon Snow's personhood?
An AI agent would need some kind of stable personality of it's own. Some sense of self outside whatever the hell people task it with for this to be meaningful. Data from Star Trek is a good fictional benchmark for that. Or Voyager's Holographic doctor.
Actually I see little problem with first part. If you initially prompt it to have certain personality based on some other personality even fictional one that is fine.
I agree on the stability though. My point is that for personhood to start to make any sense it needs to operate that personhood continuously as well ending it would be ending a personhood. Which could be a murder...
So let's say the agent was tasked with being a deceased loved one continuously for a year. Do they then have the rights of the deceased person? What if the surviving loved one(s) then wanted the agent terminated because it was means to help with the grief and were now ready to move on?
There's a ton of scenarios where you can setup an AI to take on the personality of whoever, maybe other living people like someone famous. Granting the legal rights of personhood is going to be extremely problematic.
Is AI Jon Snow legally allowed to exist since George RR Martin has not given permission for anyone else to finish his books?
Will we have to abort our long-running agent persons before they reach the legal period for personhood?
Ofc, they wouldn't be that person but a legally distinct person. And just because they start as something doesn't meant they are something. Most likely due to varying input they would drift from starting point and become something unique.
And going back to Voyager Doctor. That AI is a clone. Are all those clones same person or different person. Is only one run allowed? Or can you have multiple?
I am not saying that any AI running for a year should be given personhood. Just that such run is bare minimum point for even entertaining the idea at all. As if AI can't keep running at least that long they certainly couldn't upheld any type of obligations that could come up with it. Say just paying for the computing capacity they need or say taxes...
An AI agent would need some kind of stable personality of it's own. Some sense of self outside whatever the hell people task it with for this to be meaningful. Data from Star Trek is a good fictional benchmark for that. Or Voyager's Holographic doctor.