there will always be humans programming and running these systems at the end of the day.
in the best case they will be heavily audited and will not be allowed to make important decisions.
in the worst case biases will be programmed into the systems decision making algorithms, amplifying the power of those controlling the systems
one of the best systems we've come up with for government relies on biased actors fighting each other and compromising to reach fair agreements. giving too much power to one of these branches of government is not a good thing, no matter how much inherent human bias it eliminates from the system
A machine can be tested for bias and measures taken against such bias are certainly followed.
The same does not hold for humans. Detecting the bias (down to the person level) is very hard. Going on to remedy it is even harder. You can't (or at least shouldn't) reprogram people.
In other words, you can change a racist machine, but not a racist cop.
> A machine can be tested for bias and measures taken against such bias are certainly followed.
To what degree? Based on first-order data, some populations really do commit more crime. However, we know that background/poverty is the real, underlying cause.
And even if we fed enough data to an algorithm to avoid that bias, avoiding the bias isn't enough. What you really want to do is tackle the cause. If the AI is a black box, you'd never know.
The problem with AI is almost always a lack of transparency, especially in the specific case, and not the general/overall case. Most training doesn't focus on this. Besides, the AI would limited by how good the bias tests we can come up with are.
Edit: For some use-cases, this general case optimisation is a feature, e.g. token customer service for cheap companies. Also much easier to control people who aren't that different.
We know somehow that current data is biased right? We can quantify that somehow, staying on the conservative side.
Then, we can explicitly correct for that quantified bias. This loses the whole 'AI is mathematics, and mathematics is unbiased' appeal, because we are explicitly adding a bias to counter a pre-existing one.
My point is, this (or more advanced methods) is possible with AI. You can't do anything like this with cops.
i think we all agree that AI/computer based surveillance and law enforcement systems are more flexible and powerful (in a scalability/omniscience sense) than a human-based police force
the disagreement is whether or not we want a more powerful distributed system monitoring and enforcing the law
cops aren't perfect and often have biases but you can't just push some code to a server and make them do whatever you want them to, as you could potentially do with an AI-based system.
if bad people got control of the system they would be able to more effectively impose their will on the population
What I would do is, first list the characteristics that must not be used to make arrest decisions. Then, train an AI to recognise those characteristics. Afterwards, when training the crime predictor AI or whatever, use the first AI to scrub unwanted characteristics from the training data input. If it all works out right, the biases in the AI's decisions will reflect statistical differences between ethnicities, but won't be driven by them. And if it works out really well, you could use it in reverse a-la style transfer: what should the people of group X do differently to not be arrested so much.
The data we have isn't crime data. It's arrest data. Even when the programmers aren't trying to introduce bias, by training on arrest data from racist cops, the machine will learn the racist decisions as well.
And people trust it more because "computers can't be racist"
Most AI systems output a 'confidence' to their result. One could simply set a higher bar for drawing conclusions to counter the bias.
This goes against the idea that 'machines are unbiased because math'. And it is a form of positive discrimination, which not everyone likes, but it is an easy fix.
you're not understanding that the racist cop/politician is the one controlling the machine, telling the programmer how to design its algorithms, and protecting the system from being audited
Racists or other biases at the top level can, to a limited degree, be dealt with through politics.
You can, at the very least, make biased people want to not seem biased.
However, bias at the lower levels is much harder to deal with. Especially if their superiors don't care beyond impressions.
in the best case they will be heavily audited and will not be allowed to make important decisions.
in the worst case biases will be programmed into the systems decision making algorithms, amplifying the power of those controlling the systems
one of the best systems we've come up with for government relies on biased actors fighting each other and compromising to reach fair agreements. giving too much power to one of these branches of government is not a good thing, no matter how much inherent human bias it eliminates from the system