Pretrial risk assessment tools under fire for bias

Two recent reports cast a disparaging light on the use of pretrial risk assessment tools to determine whether someone facing criminal charges should be released while awaiting trial.

Both reports say the use of these tools, which use police and arrest data to determine risks involved in releasing someone, takes away individual liberties and may be unconstitutional. And in most cases, transparency is lacking, because companies creating the tools will not release information on what goes into the tools to make these determinations. They call that information proprietary.

Arnold Ventures, the creators of one of the most widely used pretrial assessment tools, called Public Safety Assessment, or PSA, stands by its product. It says the tool changes the process that otherwise allows rich people who are a danger to society to pay their way out of jail, while poor people with low risk to the community cannot. Money bail, the company believes, is a major factor in the injustices that keep so many behind bars, not pretrial assessment tools.

The nonprofit group EPIC, or Electronic Privacy Information Center, has released a new report that calls pretrial risk assessment tools a strike against individual liberties and says they may be unconstitutional.

“We do not condone their use,” said Ben Winters, creator of the report. “But we would absolutely say that where they are being used, they should be regulated pretty heavily.”

Report: few counties monitor results

Another recent report by The Appeal, a nonprofit that produces journalism on how policy, politics and the legal system impact vulnerable people, says that while one-third of the counties in the United States use such tools, few monitor the process to determine if they work as intended.

Pretrial assessments use algorithms to determine risk of releasing someone prior to trial, based on whether they might be a flight risk or are likely to commit a crime while out on bail. The idea behind them is to increase pretrial releases and decrease missed court appearances and pretrial recidivism. Sometimes, though, the results may be the opposite, these experts say.

“When used pretrial, these assessments can make the difference between whether someone is released or detained, with or without excessive bail, and whether their sentence is short or lifelong,” the EPIC report states. “Experts evaluating risk assessment tools have found them unreliable and biased, and critics argue that the use of these tools in the criminal justice system is unconstitutional.”

The report is entitled Liberty at Risk.

Pretrial assessments use police and arrest data, which could allow them to “exacerbate biased outcomes,” the EPIC report states. “When the algorithms identify individuals who are more likely to be stopped by a policeman, rather than individuals who are more likely to commit a crime, the algorithms perpetuate systemic bias. The Department of Justice wrote in 2014 that ‘the length of a defendant’s prison term should not be adjusted simply because a statistical analysis has suggested that other offenders with similar demographic profiles pose a greater risk.”

The Appeal report says that while Manhattan has been using pretrial risk assessment tools for five decades, other places, like Arizona, Utah and Kentucky, adopted them after Arnold Ventures created the PSA in 2012. Those who advocate their use say they are less biased than human judges and help determine who can be safely released pretrial, which reduces the use of cash bail and pretrial incarceration.

Arnold Ventures, as an example, says it believes that “moving away from a system in which a defendant’s liberty is determined solely on instinct or bail schedules to one in which judges have access to research-based risk assessments can help to improve judicial decision-making, increase public safety, promote the fair treatment of all individuals, and ensure the responsible use of taxpayer funds.”

Money plays a large role

Alison Shames, co-director of Advancing Pretrial Policy and Research for the Center for Effective Public Policy, said money bail plays a much larger role in injustice, pretrial, than any assessment tools. Her organization, funded by Arnold Ventures, works with jurisdictions to determine what their system of release determination is based on and how they can tweak it to make it more equitable for those facing pretrial retention. All pretrial assessment tools are not created equal, she said.

“Assessment tools are just one part of that. We really strongly believe it is a comprehensive approach that any jurisdiction should take,” she said. “Assessment tools are just one piece of a bigger puzzle.”

Arnold Ventures is very transparent about the information fed into its software to help determine a score for those facing trial, she said. PSA does not use demographic information, for example, to determine that score. Age is the only such factor it uses.

PSA gives judges a researched-based approach to help them determine who is a flight risk or more likely to commit a crime while on pretrial release, Shames said. “Otherwise, it gets based on where you live, what color skin you have, what your attitude is in court and who is in court to support you.”

Loss of liberty right now is caused primarily by financial releases, Shames said. “What we know from research is when the PSA is implemented along with other improvements, it will often lead to a decreased use of financial conditions, so an increased pretrial release rate.”

The Appeal, according to its report, did not find solid evidence that pretrial assessment tools are working as intended. “Only nine jurisdictions interviewed for Mapping Pretrial Injustice reported that their pretrial jail populations had decreased after their adoption of pretrial risk assessment tools.” Most jurisdictions do not track the impact of risk assessment tools on their jail populations at all.

Stanford Law School recently released its Risk Assessment Factsheet with a set of key questions on how pretrial assessment tools were designed, deployed and evaluated so stakeholders will have that information. It provides “a straightforward, standardized mechanism for stakeholders to use to conduct such audits and comparisons of any pretrial risk assessment tool they choose and when completed includes detailed answers “gathered and assembled by our team and then verified.”

Winters, an Equal Justice Works fellow for EPIC, said pretrial risk assessment tools have become a hot issue in the past couple of years due to their application in high profile cases.

“Every state’s laws are different and that is another reason we wanted to do this report,” Winters said. “It is really complicated because it is so different state by state and even county by county. We wanted to get an idea of where (pretrial assessment tools) are used and what they look like.”

The need for transparency and using subjective factors based on demographics are important, he said.

“Some of these tools include factors in terms of criminality of your family or social network,’’ he said. “It requires a lot of leaps and ends up de facto criminalizing based on your age, race and whether you are poor. Those are the factors that help determine level of risk which gives a veneer of objectivity to decision-making. That is the real difficulty with their use.”

The report was created so members of the public and defense counselors have a better idea of the technology being used, but also as a beginning to policy discussions, Winters said.

“The spaces we are moving next are advocating for where these tools are used,’’ he said. “It is a really hard conversation because a lot of times, adoption of risk tools comes with the goal of bail reform, but it does not always achieve that.”

When courts are asked to stop using these pretrial risk assessment tools, they want to know what the alternatives are, he said.

“The answer is that you have to fix the system that is operating and have legislation that is for presumption of release, to mitigate the collateral damage it reflects,’’ he said. “We advocate for proper regulation of them and that they have to be transparent, validated independently and regularly.”