From Plans to Action: Implementing Evidence-Based Practices Panelists: • • •
Facilitator:
Dan Edwards, PhD, President, • Elizabeth Seigle, Policy Analyst, Evidence-Based Associates Council of State Governments Justice Center Sylvia Rowlands, PhD, Senior Vice President, New York Foundling Patrick J. Kanary, Director, Center for Innovative Practices, The Begun Center for Violence Prevention Research and Education Council of State Governments Justice Center | 1
Presenters Dan Edwards, PhD President, Evidence-Based Associates
Sylvia Rowlands, PhD, Senior Vice President, New York Foundling
Patrick J. Kanary Director, Center for Innovative Practices, The Begun Center for Violence Prevention Research and Education
Council of State Governments Justice Center | 2
Evidence-Based Re-entry Programs: Lessons Learned from Florida’s Redirection Project
Dan Edwards, Evidence-Based Associates (EBA) Second Chance Act Grantee Workshop 22 Cortland St.; New York, NY
June 18, 2015 Council of State Governments Justice Center | 3
Evidence-Based Associates (EBA) Committed to helping communities design and build an evidence-based continuum of care for at-risk children, youth, and families
Council of State Governments Justice Center | 4
Florida in 2001 Decades of Get Tough policies led to massive over-incarceration of youth and adults – nearly 10,000 youth were placed in residential commitment statewide
Florida’s Juvenile commitments Fiscal Year 1999-00 = 9,464
Council of State Governments Justice Center | 5
Redirection Project A partnership between Florida DJJ, the Florida Legislature, Evidence-Based Associates, community-based provider agencies, and selected EBPs
Council of State Governments Justice Center | 6
Florida Redirection Project Utilization trends for 2011-12 Cohort of youth
Council of State Governments Justice Center | 7
Florida Redirection Project 2004-2013 Statewide Diversion and Reentry-Initiative
Council of State Governments Justice Center | 8
Florida Redirection Project Outcomes for 2011-12 Cohort of youth (FL DJJ) at one-year follow-up
Council of State Governments Justice Center | 9
Implementation Challenges In the real world, just because a program is called ‘evidence-based’ – its success is not guaranteed. “The goal of the implementation plan is to have practitioners use these programs as they were designed. … The combination of effective programs and effective implementation methods is required to assure consistent use of programs and to obtain reliable benefits to children and families. “ - Fixsen et al. 2009
Council of State Governments Justice Center | 10
Systemic Risk Factors for EBPs
1. Administrative Disorganization 2. Unengaged Providers/High Turnover 3. Low Referrals/Low Utilization 4. Unhappy Stakeholders 5. Poor Model Fidelity 6. Data Collection – Garbage In, Garbage Out 7. Funding isn’t Sustainable
Council of State Governments Justice Center | 11
Meeting the Challenge Head On A Case Study of FFT in Florida 2006-2011: A Data-Driven Approach to System Improvement
Council of State Governments Justice Center | 12
“Our FFT Teams are not performing” • Form a SWAT team • Establish a Baseline • Define key indicators • Contract expectations, model expectations, agency expectations • List the biggest challenges that you face • Identify your strengths to build upon (i.e. ‘study your best’) • Focus energy on top 1-2 key intervention ideas • Implement Strategies • Monitor Outcomes • (Repeat as necessary) Council of State Governments Justice Center | 13
Forming the SWAT Team • Representatives from: • • • • •
EBA (Technical Assistance and Project Manager) Provider Agency Model Training Team (FFT) Funder (DJJ) Research Team
Council of State Governments Justice Center | 14
FFT Baseline 2006-2007
Council of State Governments Justice Center | 15
Establishing A Baseline • Teams that had higher successful completion rates: • Saw families faster • Fewer Days between Referral and Open Date • Saw families more often • Higher number of average sessions per family each month. • Developed benchmarks for all programs • Linked data to “quality services”
Council of State Governments Justice Center | 16
List the Biggest Challenges 1. Data entry was inconsistent • Team didn’t see its data/got no feedback on trends • Successful teams didn’t receive recognition
2. Agency administrators were not aware of the problem 3. Not confident that data were collected on the right issues 4. As always: garbage in, garbage out (The “GIGO’ Problem) Council of State Governments Justice Center | 17
Define Key Indicators • • • • •
Average sessions per month Time between referral and first session Successful Completion rates Overall recidivism rates Agency Satisfaction • Staff retention • Administrator support • If the measure isn’t there – create it • Short-term recidivism measure Council of State Governments Justice Center | 18
Implement Strategies • Build dashboard to collect, scrub, and report data monthly- highlighting the key problems and concerns • FFT/EBA/Agencies establish quality improvement plans • Agency Administrators received data monthly • Focus on Clinical Supervisor Skills and Accountability • Supervisor clinical and leadership skill-set are key • Supervisors improve relationships with referral sources
Council of State Governments Justice Center | 19
Key Indicator Improvement
Council of State Governments Justice Center | 20
Key Indicator Improvement
Council of State Governments Justice Center | 21
Successful Completion Rates Over Time
Council of State Governments Justice Center | 22
Two years later‌ January 1, 2011-December 31, 2011 N=996 Families
Council of State Governments Justice Center | 23
Tracking Data February 2012
Grade Summary 93% 1.33 74% 25% 121%
Utilization Adherence
State
Discharges
Better Results. Guaranteed.
S-T Recidivism Admissions
Grade Detail Utilization
93%
Adherence
1.3333
State Contract Census
95% BSFT
B
FTE
91% FFT
A
1
A
1
MST
Short-Term Recidivism
25%
121%
25% Admissions as % of Target
Adjudication + Pending %
74%
2 Successful Discharge %
3.39
Admissions
Discharges 74%
Annual Benchmarks
121% Recidivism O.D.S.
Not Met Not Met
Quick Links Grading Explanation Utilization Adherence
Green
Yellow
Red
>84%
>79%
<80%
Regions
North
Central
South
Modalities
FFT
MST
BSFT
Providers
APS
Camelot
CSI
CYS
Eckerd
H.S.A.
ICFH
Lee
TSP
VQ
Average of composite scores of adherence for each modality
Successful Discharges
>69%
Short-Term Recidivism
<25%
>=25%
>39%
Admissions
>79%
>69%
<70%
<70%
WF Circuit
Additional Info
Data Table
Graphing
Council of State Governments Justice Center | 24
Tracking Data Grading 1 quarter Grading 2 quarters Grading 3 quarters Grading 4 quarters
12-18=A 24-36=A 36-54=A 48-72=A
19-25=B 37-49=B 55-73=B 73-97=B
26-32=C 50-62=C 74-92=C 98-122=C
> 33=F > 63=F > 93=F > 122=F
Utilization % of slots filled/# of slots available on the last day of the month
Admissions
% of number of youth admitted in the month/targeted number of monthly admission (designed slot allocation * 3.33/12)
2009-2010 % of new law adjudications, adjudications withheld including Short-term pending for all successful completers who have at least 4 months from Recidivism discharge and not more than 1 year post discharge/ all successful completers. Includes only JJIS data
Discharges % of successful discharges/total discharges successful and unsuccessful
Council of State Governments Justice Center | 25
Tracking Data Quarterly Comparison (Displayed by Provider in Chronological Order) 30
25
20
15
10
5
0 CSI C5 CSI C7 FFT FFT
LEE C20 FFT
CSI C9 ICFH FFT C11 FFT
ICFH VQ C6 CYS Eckerd White CSI C12 CCC White H.S.A. CSI C18 CCC C4 CSI C10 VQ C13 TSP APS C11 FFT C18 C1 C1 MST C15 C8 MST C19 FFT FFT MST FFT C17 C11/16 MST BSFT BSFT MST FFT MST FFT BSFT
Council of State Governments Justice Center | 26
The “My Child” Test: Are we doing our very best?
Dan Edwards President, EBA dedwards@ebanetwork.com
Council of State Governments Justice Center | 27
How State, Local Communities and Universities Work Together To Implement Evidenced Based Practices and Reduce Recidivism of Juvenile Offenders
Eric Shafer â&#x20AC;&#x201C; Assistant Court Administrator â&#x20AC;&#x201C; Montgomery County Juvenile Court Barbara Keen-Marsh, MSW, LISW-S, LICDC - South Community, Inc. Jeff M. Kretschmar, Ph.D. - Begun Center for Violence Prevention Research and Education, Jack, Joseph, and Morton Mandel School of Applied Social Sciences, Case Western Reserve University Council of State Governments Justice Center | 28
Montgomery County Juvenile Court Dayton, Ohio • Ohio’s Behavioral Health Juvenile Justice Initiative (BHJJ) -Began in 2005 with 6 Counties -Required the introduction of Evidenced Based Practices -Funding through: Ohio Dept. of Mental Health Ohio Dept. of Youth Services
29 Council of State Governments Justice Center | 29
LIFE Program: Learning Independence and Family Empowerment
State and Federal Partners
30 Council of State Governments Justice Center | 30
LIFE Program: Learning Independence and Family Empowerment
Local Partners South Community, Inc. Montgomery County Juvenile Court and Reclaiming Futures
ADAMHS Board of Montgomery County
31 Council of State Governments Justice Center | 31
Multiple Pathways to South Community Inc. – Function Family Therapy • Ohio Department of Youth Services • – Parole • Nicholas Residential Treatment Center – 24 bed facility for boys 12-18 – Open Setting • • Juvenile Cognitive Alternative Rehabilitation Effort – 18 bed program for boys housed within the • Detention Center – 90 Day Stay – Aggression Replacement Training (ART)
The Center for Adolescent Services – Community Correctional Facility – 44 bed facility, 34 for boys, 10 for girls MCJC Probation and Drug Court – 850 youth on Probation – 6 Dedicated Probation Officers MCJC Intervention Center (Diversion) – 24/7 Reception and Assessment Center – 2,900 cases diverted annually – Disproportionate Minority Contact Mediation Program 32 Council of State Governments Justice Center | 32
REFERRAL STRUCTURE
Youth is Released from ODYS, JCARE, Nicholas or CAS
Youth Enters Into the Juvenile Court System
South Community Completes an Assessment LIFE Program is Recommended
Parole, Probation Officer or Other Court Personnel Refer Family to Services 33 Council of State Governments Justice Center | 33
LIFE PROGRAM STRUCTURE Therapist Contacts Family Within 48 Hours of Referral
Services Provided: •Home Based Family Therapy •Psychiatric Services •Intensive Probation •Case Management •Co-Occurring Treatment
Other Collaboration: •FAMILY •PROBATION/PAROLE OFFICER •INTERVENTION CENTER •NATURAL HELPER •CHILD WELFARE •OTHER SYSTEMS •CASE WESTERN UNIVERSITY •FUNCTIONAL FAMILY THERAPY, INC. 34 Council of State Governments Justice Center | 34
Why FFT? • Blueprints Model – Evidenced Based Practice • Well Documented • Highly Successful Family Intervention Program for Juvenile Offenders • Consistent with local and state initiatives • Focus on strengths and assists families to recovery • Addition of FFT-Contingency Management
35 Council of State Governments Justice Center | 35
FFT Goals Significant and Long-Term Reduction in Youth Re-Offending and Violent Behavior Low Drop-Out and High Completion Rates Positive Impacts On Family Conflict, Family Communication, Parenting, and Youth Problem Behavior Significant Reduction in Sibling Entry into High-Risk Behaviors, 36 Council of State Governments Justice Center | 36
Service with a Smile • Weekly meetings with Probation officers to maximize collaboration and keep everyone on the same page • Including Probation Officers and Administration in FFT Trainings • Problem Solve any interruption in the flow of referrals– eliminate barriers and SELL SELL SELL to all links in the referral chain---Judges, Magistrates, Traditional Probation Officers, Mental Health Assessors in and out of our agency. • Key Attitude is: how can we make Our Process fit Your Process and Your Needs 37 Council of State Governments Justice Center | 37
Marketing = Matching • A site is Selling Two Things • Service (Micro Level) • Results (Macro Level)
• You have to do both in ways that MATCH a particular customer
38 Council of State Governments Justice Center | 38
Youth and Families: By the Numbers
39 Council of State Governments Justice Center | 39
Demographics • From 2006 through June 2013, 2,545 youth enrolled – average age at intake 15.6 years – 58.4% male – 52.3% Caucasian
• From July 2011 – June 2013 – 67.4% male – 42.9% Caucasian 40 Council of State Governments Justice Center | 40
Demographics 1,040 youth enrolled as of June 2013 ◦ 53% male, 50% Caucasian ◦ Average age: 15.5
75% of caregivers reported annual household income less than $35,000 ◦ 47% reported annual household income less than $20,000 ◦ 20% reported annual household income less than $10,000 41 Council of State Governments Justice Center | 41
Youth and Family History Question
Females
Males
20.2%
15.8%
27.5%***
9.6%
37.2%
44.6%*
Has the child ever talked about committing suicide?
48.4%***
35.9%
Has the child ever attempted suicide?
22.2%**
13.5%
Has the child ever been exposed to domestic violence or spousal abuse, of which the child was not the direct target?
44.6%
43.7%
Has anyone in the childâ&#x20AC;&#x2122;s biological family ever been diagnosed with depression or shown signs of depression?
68.4%
67.3%
Has anyone in the childâ&#x20AC;&#x2122;s biological family had a drinking or drug problem?
65.5%
65.7%
Has the child ever been physically abused? Has the child ever been sexually abused? Has the child ever had a problem with substance abuse, including alcohol and/or drugs?
42 Council of State Governments Justice Center | 42
DSM-IV Diagnoses DSM-IV Axis I Diagnosis
Females
Males
Oppositional Defiant Disorder
51.3%
54.8%
Cannabis-related Disorders
26.2%
28.2%
26.0%***
10.8%
Attention Deficit Hyperactivity Disorder
23.8%
51.6%***
Alcohol-related Disorders
14.2%
11.0%
Bipolar Disorder
12.7%
11.7%
Mood Disorder
11.9%
10.8%
Conduct Disorder
9.2%
18.3%***
Post-traumatic Stress Disorder
6.7%*
3.6%
Depressive Disorders
Average of 2.6 diagnoses per youth, 34% of youth had co-occurring MH and AoD disorders 43 Council of State Governments Justice Center | 43
Ohio Scales â&#x20AC;&#x201C; Problem Severity Problem Severity Scores from Intake to Termination - Montgomery County 30
25
Scores
20
Caregiver
15
Worker Youth
10
5
0
Intake
Termination
*all comparisons from intake to termination are significant at the p < .001 level
44 Council of State Governments Justice Center | 44
Ohio Scales - Functioning
*all comparisons from intake to termination are significant at the p < .001 level
45 Council of State Governments Justice Center | 45
Trauma TSCC Scores from Intake to Termination for Montgomery County 12
10
Subscale Scores
8
Anxiety Depression Anger
6
Posttraumatic Stress Dissociation
4
Sexual Concerns 2
0
Intake
Termination
*all comparisons from intake to termination are significant at the p < .001 level except sexual 46 concerns (p < .05 level) Council of State Governments Justice Center | 46
Substance Use Males
Females
% Ever Used Age of First Use
% Ever Used
Age of First Use
Alcohol
58.7%
13.02
56.7%
13.38
Cigarettes
60.6%
12.27
54.5%
12.47
Marijuana
62.0%*
13.04
53.9%
13.24
Cocaine
4.6%
14.55
6.0%
14.48
Pain Killers (use inconsistent with prescription)
14.4%
14.00
11.3%
13.98
Inhalants
3.2%
13.21
2.4%
13.82
Heroin
1.8%
14.25
1.3%
14.33
Ritalin (use inconsistent with prescription)
5.8%
13.80
5.3%
14.22
Non-prescription Drugs
5.3%
13.40
4.2%
13.74
Hallucinogens
5.9%
14.19
3.7%
14.53
Tranquilizers
13.7%
14.10
11.3%
14.49
47 Council of State Governments Justice Center | 47
Substance Use Self-Report Previous 6 Month Substance Use from Intake to Termination for Males - Montgomery County 100.0% 90.0%
80.0% Percent Used
70.0% 60.0%
Alcohol
50.0%
Cigarettes
40.0%
Marijuana
30.0% 20.0% 10.0% 0.0% Intake
Termination 48 Council of State Governments Justice Center | 48
Substance Use Self-Report Previous 6 Month Substance Use from Intake to Termination for Females - Montgomery County 100.0% 90.0% 80.0%
Percent Used
70.0% 60.0% Alcohol
50.0%
Cigarettes
40.0%
Marijuana
30.0% 20.0% 10.0% 0.0% Intake
Termination 49 Council of State Governments Justice Center | 49
Recidivism Information % of Youth with Misdemeanors
% of Youth with Felonies
% of Youth Known Adjudicated Delinquent
12 months prior to BHJJ
71.0%
17.3%
63.8%
12 months after BHJJ – successful completers
43.3%
12.7%
38.7%
12 months after BHJJ – unsuccessful completers
47.8%
19.9%
42.9%
50 Council of State Governments Justice Center | 50
Recidivism Of all the youth charged with a felony in the year before BHJJ, 77.5% were not charged with a new felony in the year after BHJJ 23 out of 897 BHJJ youth (2.6%) were committed to an ODYS facility any time following their enrollment. In 2005, Montgomery County committed 111 youth to ODYS with 80 revocations.
In 2013, 15 youth committed to ODYS and 4 revocations 51 Council of State Governments Justice Center | 51
Financial Impact • For the entire BHJJ program, using direct State contributions ($12.6 million since 2006)… • Average cost of a youth in BHJJ = $4954 • Average cost of an ODYS placement = $167,000 • $466 per diem, average length of stay = 11.8 months
52 Council of State Governments Justice Center | 52
Successful Foundations Early and consistent connection, communication and collaboration with ALL partners/customers Track and report out to stakeholders meaningful outcomes Ongoing problem solving Ongoing needs assessment Engage with a research/evaluation partner, e.g., Case Western Reserve University Develop sustainability Plans A, B & C 53 Council of State Governments Justice Center | 53
Next Steps • Currently investigating whether BHJJ participation as a youth impacts adult crime • Evaluation of FFT-CM
54 Council of State Governments Justice Center | 54
Contact Information Eric Shafer Montgomery County Juvenile Court 937-225-4164 eshafer@mcjcohio.org Jeff Kretschmar Begun Center for Violence Prevention Research and Education Case Western Reserve University 216-368-2305 jeff.kretschmar@case.edu Barbara Keen-Marsh South Community, Inc. 937-534-1325 bmarsh@southcommunity.com 55 Council of State Governments Justice Center | 55
The New York Foundling Sylvia Rowlands, Senior Vice President Evidence-Based Services
Number of NYC Juvenile Delinquents & Juvenile Offenders in Placement
Council of State Governments Justice Center | 57
ATP Enrollments by Program: 20062014
Council of State Governments Justice Center | 58
Thoughts on Effective Interventions • Basic strategies •Stop implementing programs known to be ineffective or harmful •Select EB programs when available and fit need •When no EB fit, use best evidence available (a data-based decision) •Commit to evaluate all non-EB programs
Council of State Governments Justice Center | 59
Implementation Support â&#x20AC;˘ Implementation Support is intended to achieve the following interrelated goals: 1.
2. 3.
Increase the pace at which EBPs are routinely available through the public mental health or child welfare system. Promote the sustainable, model adherent implementation of EBPs Improve outcomes for child and adult consumers.
â&#x20AC;˘ Outcomes of the Cal-40 Study: Of the 51 sites who implemented TFCO, those in the cohort condition placed two times the youth than the sites in the implementation as usual condition Council of State Governments Justice Center | 60
Looking Forward • • • •
Don’t oversell Intervention are not substitutes for relationships Be aware of the cultural sensitivity issue Avoid, almost at all costs, any modification to a model. When necessary, only to be done in consultation with the developers. • Strategically engaging stakeholders OR • Create welcoming environments • Takes longer than you think
Council of State Governments Justice Center | 61
Questions Dan Edwards, PhD, President, EvidenceBased Associates DEdwards@ebanetwork.com Sylvia Rowlands, PhD, Senior Vice President, New York Foundling Sylvia.rowlands@NYFoundling.org
Patrick J. Kanary, Director, Center for Innovative Practices, The Begun Center for Violence Prevention Research and Education, Jack, Joseph and Morton Mandel School of Applied Social Sciences, Case Western Reserve University patrick.kanary@case.edu
Council of State Governments Justice Center | 62
Thank You Join our distribution list to receive CSG Justice Center project updates! www.csgjusticecenter.org/subscribe www.nationalreentryresourcecenter.org
The presentation was developed by members of the Council of State Governments Justice Center staff. The statements made reflect the views of the authors, and should not be considered the official position of the Justice Center, the members of the Council of State Governments, or the funding agency supporting the work. Citations available for statistics presented in preceding slides available on CSG Justice Center web site. Council of State Governments Justice Center | 63