For a model that returns probabilities over caller-defined options, I'd check calibration separately from accuracy. On held-out labeled questions, NLL and Brier score measure whether it assigns probability to the right answer; ECE or reliability plots can show whether decisions made at, say, 0.8 confidence are right about 80% of the time. Fit any temperature on a separate calibration split and leave the final test untouched. The split should be by underlying document or scenario, not by individual paraphrase, or near-duplicates leak across it.
I tried this with Bobcat, a Jev-style model I trained on Qwen. A single temperature lowered development ECE from 0.021 to 0.010; its sealed final ECE was 0.012. That is calibration against our own labels, not a measured claim about Jev's calibration. On TypeSafe's 20 published workflow examples, we can compare probability assigned to a reference answer, but that reference is model consensus, not ground truth, and the sample is too small for a general calibration claim. Details and the evaluation code: https://huggingface.co/sanghwa-na/bobcat-1.1
I'm curious to know why you'd say that. I'm asking because I've been prescribed it since 2021 to manage depression and chronic pain from spinal disc herniations. I've had moments of high strangeness with it, much to the effect of what John Lilly described in his writings. I do intend to stop it once I've established a better quality of life via alternative methods.
It’s a sane default to label power as evil in a society driven by greed, usury, and capital gain. Power tends to corrupt, particularly when the incentives driving its pursuit or sustenance undermine scruples or conscientiousness. It is difficult to see how power is not corrupting when it becomes an end in itself, rather than a means directed toward a worthy or noble purpose.
We know that the current administration functions like a cabal of sex-trafficking mobsters, so none of this is surprising; strong-arming is the norm, not the exception. I expect this to get ugly, and I hope Anthropic has the financial and legal resources to respond accordingly.
Not a useful analogy because nihilism is a human pursuit, not an OS one. The aggregate history of humanity does have meaning; it embodies information, cultural transmission, a lineage of ideas, which are all artifacts of human interactions that often shape the trajectory of civilization and the choices we make as individuals.
