True, but how useful is that granularity, really? If you combine the fact that the test has multiple items assessing the same skill, and the fact that they employ (reasonably) sophisticated statistics (Item-response theory) to come up with a probabilistic assessment of the student's true ability given the observed answer pattern, would something like this really buy you a whole lot more?
It's probably not too useful. Ideally it would reduce variance a bit, but probably not enormously. People don't really understand their own level of certainty, anyway, so that would be a confounding factor.
I wonder if it would be a better compromise to allow multiple answers. If I've narrowed it down to two, I guess one and then I have a 50% chance of getting it right. Instead, let me just put down both answers, and if I got it right, give me half a point. I think you'd get the same ultimate outcome as penalizing wrong answers, but with a bit less randomness, while maintaining comprehension. Maybe!
