# Garbage In/Garbage Out in criminal justice AI systems **URL:** \ **Category:** In My Humble Opinion\ **Created:** [January 22, 2019, 5:28pm UTC](https://boards.straightdope.com/t/garbage-in-garbage-out-in-criminal-justice-ai-systems/828211 "2019-01-22T17:28:46Z")\ **Posts on this page:** 5\ **Page:** 2
**Author:** ![Jragon](https://avatars.discourse-cdn.com/v4/letter/j/e19b73/32.png) [@Jragon](https://boards.straightdope.com/u/Jragon)\ **Post date:** [January 23, 2019, 11:35pm UTC](https://boards.straightdope.com/t/garbage-in-garbage-out-in-criminal-justice-ai-systems/828211/21 "2019-01-23T23:35:47Z")
Hello, yes, I’m an AI researcher that specifically works on systems that can explain why they made the decisions they made. I also have some background in anomaly detection work (read: predictive classifiers failing or giving false/positives or negatives). This is really an awful idea. All it really does is magnify social biases and causes people to “trust in the unbiased computer” as opposed to dealing with large systemic issues in the criminal justice system. The biggest issue comes down to training data, you could: 1. Select real crime statistics, in which case it inherits all the biases in the criminal justice system present. For instance, it could predict person A has a high chance of recidivism because they’re from a majority minority neighborhood (that’s assuming they don’t straight up use inadmissables like race which um… they have in the past). This doesn’t take into account that “recidivism” is only going to register if they get re-arrested, which is naturally going to happen for POC and other groups that get unduly arrested more in the first place. 2. Make up fabricated data based on crime statistic from some model you’ve developed. Congrats, you’ve just inherited the biases of whoever made the data set. 3. Use a modification of (1), but somehow label individual instances as “good” or “bad” or tag their “level of correctness” somehow. You’ve basically just now made 1 with the problems of 2 as well, congrats. Half the time we make a computer vision software it fails because there’s a plant in the background or because someone is black. Criminal sentencing is far too sensitive to overfitting to **extremely** subtle social biases to be usable in its current form. Yes, these techniques are maximizing or minimizing some function, but fundamentally overfitting or fitting to data produced by a heavily biased system is a problem. The system just cannot do anything more than reinforce the system it’s trained to replicate, or conform to a specific person or team’s vision of that system. Overall, things likely won’t change much if we trust an AI system to produce these results, and there may even be cases where a **very** racist judge is less racist because of the prediction (of course, the opposite is also true). But it really just adds an air of fallacious validity to our flawed-in-the-first-place system. ---
**Author:** ![Taber](https://avatars.discourse-cdn.com/v4/letter/t/ac91a4/32.png) [@Taber](https://boards.straightdope.com/u/Taber)\ **Post date:** [January 23, 2019, 11:40pm UTC](https://boards.straightdope.com/t/garbage-in-garbage-out-in-criminal-justice-ai-systems/828211/22 "2019-01-23T23:40:41Z")
> [@SamuelA](#): > > You should perhaps re-examine your goals. Let’s suppose there is a clear correlation between class and recidivism. I mean, I don’t assume there is, but I would believe it if the data showed that, because obviously richer people have more ways to stay out of trouble. > > But if you’re the system designer, what do you care about? Ethically, anything other than “to minimize the number of victims of crimes, weighted more heavily towards more serious crimes, with the budget I have” is a policy that **leads to more crime victims**. > > Sure, it may be \*unfair \*to jail people of a lower class longer, and it may also be unfair because due to a long history of discrimination, some minorities have a much higher probability of being in the lowest class. > > But remember, any policy of trying to be “more fair” **by definition** means you trade off for less effectiveness in preventing other citizens from being a victim of a crime. > > What matters more to you? Personally I see only one valid choice, but if you can argue another way and get your peers to agree, it’s possible to set the optimizing criteria on an AI system your way. The wrinkle is that uneven enforcement inflates crime statistics among groups facing stricter enforcement. Having the AI codify that isn’t the most effective. ---
**Author:** ![SamuelA](https://avatars.discourse-cdn.com/v4/letter/s/c77e96/32.png) [@SamuelA](https://boards.straightdope.com/u/SamuelA)\ **Post date:** [January 23, 2019, 11:53pm UTC](https://boards.straightdope.com/t/garbage-in-garbage-out-in-criminal-justice-ai-systems/828211/23 "2019-01-23T23:53:21Z")
> [@Taber](#): > > The wrinkle is that uneven enforcement inflates crime statistics among groups facing stricter enforcement. Having the AI codify that isn’t the most effective. I agree that is a confounding variable. Doesn’t mean you should just abandon using AI or numbers and go by the gut feeling of a judge or parole officer, though. ---
**Author:** ![doreen](https://avatars.discourse-cdn.com/v4/letter/d/858c86/32.png) [@doreen](https://boards.straightdope.com/u/doreen)\ **Post date:** [January 24, 2019, 12:15am UTC](https://boards.straightdope.com/t/garbage-in-garbage-out-in-criminal-justice-ai-systems/828211/24 "2019-01-24T00:15:08Z")
> [@Taber](#): > > I’m not sure if there’s a real solution - this all seems bad, but I agree that it is probably better than humans doing it. That’s really all I’m saying- and it’s really all that most people in favor of these tools are saying. From the outside, it’s easy to see that there would be a problem with your hypothetical “parole officers are harder on Blacks so they violate more often so they are denied parole more often.” I would bet my bottom dollar you never imagined the following absolutely true situation. An officer completed a re-assessment of an offender. The offender had been doing well , and the tool recommended less intensive supervision. The officer wanted to override the tool and keep him at the same level of supervision. When asked for the rationale, the officer replied “He’s not living up to his potential”. That is absolutely not supposed to be a factor - but I’m sure it would have been an unseen factor if that officer was making the decision based on his own judgement. ---
**Author:** ![Typo\_Negative](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/typo_negative/32/484_2.png) [@Typo\_Negative](https://boards.straightdope.com/u/Typo_Negative)\ **Post date:** [January 24, 2019, 1:10am UTC](https://boards.straightdope.com/t/garbage-in-garbage-out-in-criminal-justice-ai-systems/828211/25 "2019-01-24T01:10:20Z")
> [@EscAlaMike](#): > > The goal with most of these AI assessments is to take the prejudiced human equation out of the system as much as possible and to reduce recidivism. In a law and order culture like the US with a “lock 'em up and throw away the key” mentality, these systems are actually very progressive. They reward treatment and encourage rehabilitation. It is difficult to reconcile this with the fact that system that is using it is anti-science. They are not interested in justice, just convictions. **How bad science is undermining America’s justice system** [https://www.msn.com/en-us/news/crime/how-bad-science-is-undermining-americas-justice-system/ar-BBSCBwX](https://www.msn.com/en-us/news/crime/how-bad-science-is-undermining-americas-justice-system/ar-BBSCBwX) They still tout bite mark and blood splatter analysis as science. > [@](#): > > Over the past decade, dozens of studies have warned of the inaccuracy of commonly used methods of forensic science. > > Some of the methods, including bite-mark comparisons and blood-spatter analysis, are now considered by many researchers to be scientifically dubious, responsible for dozens of wrongful convictions and deserving no place in the criminal justice system. > [@](#): > > The divide between science and justice has widened in the nearly two years since the Justice Department under then-Attorney General Jeff Sessions ended an Obama-era partnership with independent scientists to improve forensics. In 2017, Sessions disbanded the National Commission on Forensic Science, which included researchers, scientists, lawyers, evidence examiners and law enforcement officials, and replaced it with a Forensic Science Working Group, which is housed within the Justice Department and is led by a career prosecutor. [Previous page](https://boards.straightdope.com/t/garbage-in-garbage-out-in-criminal-justice-ai-systems/828211.md?page=1)