I was addressing the other half of the initial statement:
> Nothing about the conception of knowledge as "justified true belief" implies that machines can't accrue knowledge.
Even if the LLMs could adjust their weights in response to knowledge that they encountered post-training, they would need to make judgements about what is "true" so that they can only add "justified true beliefs" to their weights.
I agree that this is also a problem humans have, and we do really badly at it because we tend to weight social conformance more than factual accuracy.
And yes, I agree that this problem bears no relation to the question of whether they're intelligent, or even sentient. I was answering a different question.
So? We also have the exact same problem, and no solution that covers all facts, not even the most basic ones.
So them having this problem would be either neutral or supportive of the idea they're intelligent, not refuting it.