They are weights for matrix operations, so in principle you'd think they would be deterministic. In practice it's more complicated than that.

Not to be snarky or dismissive, I mean this genuinely: ask an LLM about it. I currently have a headache so I'm not up to explaining the technical details, but they are interesting and worth reading about.