Which is totally not possible with judges, right ?

I feel like a lot of arguments being made here fail the A vs B test. Any argument that purports to provide help with choosing Judges vs Algorithms needs to apply differently to Judges, and differently to Algorithms.

How about: with Judges we simply won't know (for sure) what influences them. Are they racist ? Who knows. Do they prefere to let people with jobs out (realistically: yes, but we don't know for sure). Do they ...

With algorithm we can literally test, by presenting them with artificial cases, lots of them, and see how they judge. With a judge, you can't.

> I feel like a lot of arguments being made here fail the A vs B test. Any argument that purports to provide help with choosing Judges vs Algorithms needs to apply differently to Judges, and differently to Algorithms.

Out of curiosity, is there a name for this "fallacy", if it is one, since to me it mostly seems like the other party is failing at some basic level of critical thought.

I've been dealing a lot with arguments of this nature at work, and it'd be great to have a name to it. Pointing it out in the verbatim sense ("ok, but that's true of <your counter position> as well") becomes tiring quickly, and honestly, just causes the person to move on to the next fallacious claim.

Well, humans are already capable of dealing with it. The judges know that prisoners know what is expected of a good prisoner. The decisions are already being made with that in mind.

Contrast this with evaluating a programmer's performance. Everyone knows that lines of code written, number of tickets closed, number of fixed bugs or lines of documentation written do correlate well with performance. But the minute they are revealed to impact performance reviews, those metrics becone trash. Until you can find viable instruments, you shouldn't ever put those into a model and expect to have good predictions. If your model is not explicitly equipped to deal with endogeneity (like structural equation models), it will fail when faced with it.

If you think a judge is influenced by things that are unrelated to the case, you should appeal to the court above (which you can readily do in Continental Europe, but I don't know about Common Law).

> Well, humans are already capable of dealing with it

If that were true, algorithms wouldn't be able to outperform those humans on the metrics that matter.

About your faking metrics issue, the trick in this case is simply taking the metrics that matter and feeding them into an algorithm. Problem solved.

Take metrics:

1) will suspect face justice if released

2) will he reintegrate faster if released

Anyone criminal who wants to game those metrics, well I for one will be applauding that !