Exactly. The crux of prejudice is that it makes people form an opinion before they have facts a.k.a. context. A lot of things are not what they seem at first and because people respond to well to "Occam's Razor," they choose the laziest explanation and penalize people they don't know.
A real-life example: my friend was dating a woman whose brother got in an accident. After receiving painkillers, he became addicted to them (an old story at this point). As he weened himself off of them, taking responsibility, my friend would drive him to get methadone a few times a week. A system that uses information from this one would label my friend an addict because he was so often seen driving to or parked at the rehab clinic. When in fact, my friend was a charitable, stand-up guy helping out a soon-to-be family member recover and become a stronger, productive member of society.
Just like credit scores. Their models aren't perfect, but the threshold for useful (for their benefit) is much lower, and profitable lower still. Since you're the product being sold and they're OK with many defects, and given that there are no financial consequences for you getting rolled up in a bad / inaccurate / predatory model, they don't care and aren't incentivized to fix it (and in fact in many cases are incentivized financially to keep it broken, since "hits" like that can be a selling point).
A real-life example: my friend was dating a woman whose brother got in an accident. After receiving painkillers, he became addicted to them (an old story at this point). As he weened himself off of them, taking responsibility, my friend would drive him to get methadone a few times a week. A system that uses information from this one would label my friend an addict because he was so often seen driving to or parked at the rehab clinic. When in fact, my friend was a charitable, stand-up guy helping out a soon-to-be family member recover and become a stronger, productive member of society.