The thing saying it is not a person, and has no beliefs.

The problem is basically epistemology. GPTs don't have any. Arguably they don't know anything. (Arguably they do to an extent, because the knowledge is encoded in the words in the training data.) But even if they know things, they don't know that they know, and so they cannot tell between "knowing" and "not knowing".

This is all semantic bullshit. I could raise a child devoid of any outside contact and teach them that klingons enslave our people and we must hide in the woods. Things you see in the skies are their hunting machines. They could completely "know that" and its a total fabrication.