Florida’s New Predictive Risk Tool Likely to Drive Down Juvenile Incarceration

The Florida Department of Juvenile Justice is set to become the first state agency to conduct risk assessments using predictive analytics, a process that uses huge collections of data to predict outcomes and patterns.

It’s the sort of news that, on its face, excites at best a handful of stat geeks. But the decision could have major implications for providers of community-based juvenile justice services.

The research that prompted Florida’s decision suggests that significantly fewer juveniles would be slated for residential placements or incarceration under the new process. The new predictive model has superior ability to predict recidivism while also designating far fewer youth in risk categories that often lead to out-of-home placements.

The planned move appears to pave the way for reduced incarceration and increased spending on other alternatives without increased risk to the community.

From the paper co-written by the agency and a company called Algorhythm, analyzing the Florida research:

“Sizable reductions in residential care placements could lead to significant short-and long-term cost savings for the state and/or a freeing up of financial resources to be re-invested in preventative or community-based care.”

“The use of predictive analytics and machine learning will be a game changer in juvenile justice risk assessment,” said Algorhythm consultant Ira Schwartz, who led the federal Office of Juvenile Justice and Delinquency Prevention from 1979 to 1981. “Imagine what implications this has around the country.”

Risk, Scoring and Prediction

Risk assessments have been a part of juvenile justice decision-making since the 1970s. The first iterations involved highly subjective processes that hinged mostly on the view of the assessor or a review of a young person’s criminal history.

It is in the past 20 years that juvenile risk assessments have evolved into actuarial tools, which survey for answers about dozens of factors to establish a risk score. That score is designed to help probation departments and judges handle youth appropriately in the juvenile justice system.

PACT vs PA
Prediction Process: The predictive tool tested on Florida data routed more youth to low-risk categories and predicted recidivism at a better rate

Predictive analytics is a burgeoning corner of the business technology industry that uses current and historical data to predict future behavior. For business purposes, predictive models are built to interpret large sets of data and produce a probability. This is applied to anything from individual credit scores to customer purchasing to vehicle fleet management.

Several child welfare agencies, including the Florida Department of Children and Families, have embraced predictive risk modeling to inform responses to abuse and neglect cases.

The Florida Department of Juvenile Justice (DJJ) Office of Research and Data Integrity started to use predictive analytics in 2010, but only for the purposes of research and evaluation. Office Chief Mark Greenwald had to assure stakeholders at the time that DJJ was not going to replace the state’s risk assessment tool – the Positive Achievement Change Tool (PACT) – with any predictive model.

“I think the concern earlier was the minority report aspect” of predictive analytics, Greenwald said, the idea that “we would give judges something saying, ‘This is gonna happen, so do this.’”

The method still has its skeptics when it comes to dictating process.

“Predictive Analytics, more than anything, is an approach to learning,” said Jesse Russell, director of research for the National Council on Crime and Delinquency, which sells its own juvenile and child welfare risk assessment tool called Structured Decision Making (SDM). “We do predictive analytics in the development of our actuarial model,” Russell added.“But we still use an actuarial model.”

He gives two reasons why NCCD stands by an actuarial model. First is the “black box” aura around predictive analytics, the notion that a computer is taking all kinds of data points and processing an answer. The “freeing up of restrictions on how you work with data means you don’t have to understand whether things are causal,” he said.

Second is the removal of human involvement. “We have stood by the idea that the engagement that an actuarial tool drives is critical, that you have to seek out answers,” Russell said.

Florida First

Greenwald and DJJ have been convinced otherwise. The agency intends to continue using its PACT questions in the future, but will move away from its traditional score-based format, which was developed in partnership with Assessments.com in 2005.

Instead, the information gathered will be routed through a predictive model known as a “decision tree.” The decision comes after research found that a predictive model yielded greater accuracy using the same information as the current PACT scoring process.

Algorhythm, a New York company that builds information technology for social services providers, amassed a control study using five years of DJJ data. Thousands of cases were put through the decision-tree predictive system to identify differences in both classification and accuracy.

From the paper written by DJJ and Algorhythm, shared with The Chronicle before its publication:

“More youth were placed into appropriate risk levels, and recidivism prediction accuracy was increased. And, the placement of more youth into the lower risk levels was not accompanied by significant increases in the overall recidivism rates for the youth in these categories. This suggests that public safety would not be compromised if DJJ deployed the predictive analytics and machine learning algorithms.”

The change will make Florida DJJ the first juvenile justice agency to employ predictive analytics in frontline decision-making. Greenwald says there is no set timeline for the changeover, but that it will happen.

“The difference with predictive is it’s substantially more complex,” Greenwald said. “It’s a more advanced way of doing the scoring.”

He made the analogy of driving on the same highway in a car with a better engine. “There’s nothing wrong with [PACT],” Greenwald said. “There’s still an algorithim behind it.  We’ll still use an actuarial instrument, we’ll just have enhanced information behind it.”

More Demand for Community-Based Providers

The shift toward a predictive risk assessment would not change decision-making for youth receiving those classifications. But the research that prompted this change suggests that under the new assessment, far fewer youth will find their way into the high and moderate-high risk categories.

Florida assigns juvenile offenders to four risk categories: low, moderate, moderate-high, and high. Youth classified as low- or moderate-risk are either diverted or assigned to a low-level intervention like probation.

Youth in the high-risk category are often placed in residential programs or incarcerated, Greenwald said, “provided that there was an attempt at community-based services first.”

Youth classified as “moderate-high” risk “could go either way,” Greenwald said.

The report includes a controlled comparison of the current PACT process and the predictive model. Both classified 72 percent of the youth as “low” risk level.

But the predictive analytics model classified 19 percent of the youth as “moderate,” compared with just 11 percent under the current model. The predictive model set 8 percent of youth to the two higher categories, and the current model set 17 percent.

NCCD’s Russell said that the greater number of youth in low-risk categories is not, in and of itself, persuasive from an accuracy standpoint.

“Any tool can be made to produce any distribution you want,” Russell said. “If that’s something they wanted to achieve and they achieved it, great.”

And, he suggested, higher accuracy can always be achieved by classifying fewer youth in higher-risk categories because, on the aggregate, most of the youth who have contact with the juvenile justice system do not return.

“I could be 80 percent accurate by saying ‘None of them are coming back,’” Russell said. “But how useful would that be?”

That assertion seems to be countered somewhat by the fact that the predictive model was more accurate within each risk category, and significantly more accurate in the higher-risk categories. The predictive model was 52 percent accurate in the moderate-high category (37 percent for the current tool), and 66 percent accurate in the high-risk category (41 percent for the current tool).

John Kelly is senior editor for The Chronicle of Social Change.

Print Friendly, PDF & Email

John Kelly
About John Kelly 1130 Articles
John Kelly is editor-in-chief of The Chronicle of Social Change.

5 Comments

  1. Florida DJJ has stood steadfastly by the PACT while in my county, anywhere from 20% to as high as 44% of the youths arrested in a given month were re-arrests of youths designated as low risk to reoffend by that instrument.

    The number of juvenile residential programs has been sharply reduced, and both DJJ policy and State Statutes have been manipulated to make it more difficult to detain or place offenders, and in return, DJJ had stated that these machinations would result in more funds directed to the counties for prevention, and alternative and community solutions, as they are again promising with this new proposal. We’ve seen not one dime nor one alternative, only the reductions and restrictions. Meanwhile, juvenile offenders laugh at what the delinquency system has become, parents are frustrated and left feeling abandoned by the system, and youths are taught that they are safe to expect no consequences for committing crimes. The courts are left to basically say “Let’s make them get worse; then we’ll intervene.”

    Unfortunately, by the time intervention can be effected in the current climate, the individual youths typically have become much less amenable to treatment, have engaged in more self-destrucctive acts, they have deepened their addictions and amassed more serious criminal records, the community has been subjected to more criminal activity, and families are subject to turmoil from an out-of-control youth.

    Place me in the “Skeptical” category. Crime has, of course, been on the decline, but perhaps despite DJJ’s efforts rather than because of them. I will hope for the best with the new predictive tool, but I have been working in the system too long to let down my guard at this point.

  2. Relationship with the client (Knowing your Cultural and Environment), the overall seriousness of the offense and your local Judicial System has to come first.
    Followed by experience in the job and then using the Tools/Assessment. And assessment is a tool and can hurt more than help when not used or applied properly by the assessor or administration.

  3. Great use case of predictive analytics and machine learning. We over at https://www.lumidatum.com are hoping more governmental groups will come around to the benefits of these types of tools and analytics. Our platform helps simplify this whole process, great post and exciting developments in Florida!

  4. It’s good you’re going to use the SAVRY. However, you might want to test it against a risk assessment tool developed using predictive analytics and machine learning. A “head to head” competition might prove to be very interesting.

  5. Louisiana Office of Juvenile Justice
    P. O. Box 66458, Audubon Station, Baton Rouge, LA 70896
    http://www.ojj.louisiana.gov
    NEWS RELEASE

    FOR IMMEDIATE RELEASE August 22, 2008
    CONTACT: Jerel M. Giarrusso, Tel. 225.287.7898; email Jerel.Giarrusso@la.gov

    Office of Juvenile Justice Adopts Cutting-Edge Method of Assessing Youth

    Baton Rouge, La – Youth placed in the custody of the Office of Juvenile Justice after being adjudicated delinquent will soon benefit from an assessment tool designed to identify individual youths’ treatment needs and predict their risk for violence and re-offending. Entitled SAVRY (Structured Assessment of Violence Risk in Youth), this assessment tool is considered by national experts to be one of the best predictors of this type of risk.

    Deputy Secretary Mary L. Livers, Ph.D, MSW, made the announcement, saying that SAVRY is a well-researched, evidence-based assessment tool that will assist OJJ, the courts and district attorneys to appropriately assess both risk and need. Use of the tool will allow OJJ to more accurately project needs for secure, moderate and non-secure resources on a statewide basis.

    “I am delighted that OJJ has taken this vital step as the agency moves forward with systemic reform of juvenile justice in Louisiana,” said Gov. Jindal. “This is yet another step toward implementation of evidence and community-based services that will be more beneficial to the youth in state custody and their families. More widespread use of community-based programming will allow us to realize significant cost-savings while providing more effective services.”

    SAVRY was adopted by OJJ after careful study, and in partnership with the John D. and Catherine T. MacArthur Foundation, a national leader in juvenile justice research and best practices.

    “Use of a standardized risk/needs assessment tool leads to unbiased decision-making,” Dr. Livers said. “Research clearly shows that putting low-risk youth into high-risk settings can make them worse. Without an assessment tool in place, all youth are assigned to the same programs, which can reduce their chances of success.”

    The SAVRY is evidence-based and uses an approach that examines risk factors associated with violence, applies the factors to the youth being assessed, and classifies severity of the factors. Many items in the SAVRY permit re-assessment that allows identification of changes in risk in response to services provided to the youth.

    MacArthur Foundation experts call the SAVRY one of the top two assessment tools of its kind and are interested in testing the effectiveness of system-wide use. By implementing use of the SAVRY, Louisiana will become a model for other states to emulate, according to the MacArthur Foundation.

Comments are closed.