jmhcp-orientation_2013-02-27_willis by CSG Justice Center

Integrity Accountability Solu2ons

BJA Performance Measures Timothy J. Willis, Ph.D. Senior Research Associate, CSR, Incorporated =mothy.willis @ ojp.usdoj.gov (202) 514-‐8045

At the end of this session, you will be able to answer these ques2ons : Ø

Why do we report performance measurement data?

Ø

Who came up with these measures?

Ø

What are the repor=ng requirements?

Ø

When do I report?

Ø

Where do I report?

Good Performance Measures. . . Link people and dollars to performance Ø  Link programs and resources to results Ø

Ø

Help to jus=fy con=nued funding

Ø

Can be used to manage and improve programs

Performance Measurement

Program Evalua2on

Asks the ques2on: “What happened?”

Asks the ques2on: “Does this work?”

Func2ons like a scoreboard

Func2ons like a postgame analysis

Is conducted con2nually

Has set start and end dates

Costs less to conduct

Costs more to conduct 4

BJA’s 2013–2016 Strategic Plan Performance Measurement Goals Ø

Goal 1: Support eﬀec=ve criminal jus=ce policy, programs, informa=on sharing, and collabora=ons within state, local, and tribal agencies and communi=es

Ø

Goal 2: Promote the use of data, research, and informa=on to increase the eﬀec=veness of criminal jus=ce programs

Ø

Goal 4: Improve criminal jus=ce outcomes through eﬀec=ve grant management and oversight

BJA’s 2013–2016 Strategic Plan Goal 1 Ø

Support eﬀec=ve criminal jus=ce policy, programs, informa=on sharing, and collabora=ons within state, local, and tribal agencies and communi=es

Ø

Relevant Strategies: 1.  Provide high-‐quality relevant training and technical assistance to the

ﬁeld

2.  Ensure BJA program models are eﬀec=ve 3.  Communicate criminal jus=ce solu=ons to the ﬁeld

BJA’s 2013–2016 Strategic Plan Goal 2 Ø

Promote the use of data, research, and informa=on to increase the eﬀec=veness of criminal jus=ce programs

Ø

Relevant Strategies: 1.  Help criminal jus=ce agencies to access, share, and use data in

decisionmaking 2.  Promote and share evidence-‐based promising prac=ces and programs

BJA’s 2013–2016 Strategic Plan Goal 4 Ø

Goal 4: Improve criminal jus=ce outcomes through eﬀec=ve grant management and oversight

Ø

Relevant Strategies: 1.  Maximize the use of informa=on reported in BJA’s Performance

Measurement Tool (PMT) and Training and Technical Assistance Repor=ng System (TTARS) to monitor grantee performance

2.  Maintain a formal process for reviewing grant program performance

(GrantStat)

JMHCP Closeout Report

Available now on BJA’s Web site: www.bja.gov

Example of Performance Data Analysis

JMHCP Performance Measures Survey Available here: hVps://www.BJAperformanceTools.org.

General Award Informa2on Ques2ons 1–2: Grant Ac=vity and Program Type Ø

1. Was there grant ac2vity during the repor=ng period? A.  Yes B.  No C.  If no, please explain: _______________

Ø

2. Which of the following best describes the type of program you are implemen=ng? A.  Law enforcement-‐based program B.  Court-‐based diversion program C.  In-‐facility correc=ons program D.  Community correc=ons program E.  Reentry program

Three Grant Categories Ø

Planning

Ø

Planning and Implementa=on

Ø

Expansion

Three Groups of Measures Planning

Opera=onal

Narra=ve

Five Ac2vity Types

Planning

Opera=onal

Narra=ve

Law Enforcement Court-‐based Diversion In-‐facility Correc2ons Community Correc2ons Reentry Programs

Planning

Opera=onal Law Enforcement

Planning & Implementa2on Grants

Court-‐based Diversion In-‐facility Correc2ons Community Correc2ons Reentry Programs

Narra=ve

Planning

Planning Grants

Narra=ve

Opera=onal Law Enforcement

Expansion Grants

Court-‐based Diversion In-‐facility Correc2ons Community Correc2ons Reentry Programs

Narra=ve

Opera=onal Law Enforcement Court-‐based Diversion In-‐facility Correc2ons Community Correc2ons Reentry Programs

Law Enforcement Categories 1.  Training to respond to people with mental illness 2.  Calls for mental health-‐related events

Opera=onal

3.  Field assessment and screening for mental illness

Law Enforcement Court-‐based Diversion

4.  Disposi=on of people involved in a mental health crisis

In-‐facility Correc2ons Community Correc2ons

5.  Injuries incurred during mental illness– related events

Reentry Programs

Direct Service Categories 1.  Training ac=vity 2.  Program par=cipant demographics

3.  Risk/needs assessments

Opera=onal Law Enforcement Court-‐based Diversion In-‐facility Correc2ons Community Correc2ons Reentry Programs

4.  Treatment for mental illness, substance abuse, and co-‐occurring disorders 5.  Employment, educa=onal, and housing services 6.  Program comple=on rate

7.  Recidivism outcomes

Repor2ng in the PMT

Calendar Year 2013 Repor2ng Schedule FY2013 JAN

←Q1 Reports Due Jan 31 FEB (upload it to GMS)

MAR

FY2013 APR

MAY

←Q2 Reports Due April 30

JUN FY2013 JUL AUG ←Q3 Reports Due July 31 (upload it to GMS) SEP FY2014 OCT ←Q4 Reports Due Oct 31 Q1 NOV Q4

DEC

How to Report on Performance Measures Ø  Create a report in the Performance Measurement

Tool (PMT) quarterly

Ø  Upload PMT report to the Grants Management

System (GMS) semiannually

Ø  Funds are frozen if reports are delinquent

Performance Measurement Tool (PMT)

GMS Progress Report Module

GMS AVachment Upload

Repor2ng Tips Ø

You select the sec=ons that apply to your ac=vi=es. §  “No” can be changed to “Yes” but “Yes” cannot be changed to “No.”

Ø

Alach the PMT report to your semiannual progress report in the GMS.

Ø

Provide detailed responses to the seven narra=ve ques=ons (these are important).

Planning Measures (11 Ques2ons)

This sec=on is skipped for Expansion Grants. JMHCP Planning and Implementation Guide: https://www.bja.gov/Funding/ JMHCPplanImpGuide.pdf

Planning Measures 1–2 Ø

1. Does your program follow a speciﬁc methodology or prac=ce? A.  Yes B.  No C.  If yes, please explain.

Ø

2. Is there a task force or planning commilee in place to help guide program implementa=on? A.  Yes B.  No C.  If yes, provide addi=onal informa=on about its membership and the

cons=tuencies they represent

Planning Measures 3–5 Ø

3. Do you plan to apply for addi=onal JMHCP funding in the future? A.  Yes B.  No C.  Unknown

Ø

4. What is the mission statement of your grant ini=a=ve?

Ø

5. What are the short-‐term and long-‐term goals for the grant ini=a=ve?

Planning Measures 6–7 Ø

6. Do you have a list of mental health, substance abuse, and other service providers and resources for your jurisdic=on? A.  Yes B.  No C.  If no, provide addi=onal informa=on about your plans to iden=fy

available services and resources.

Ø

7. Have you selected a target popula=on for the ini=a=ve? A.  Yes B.  No C.  Describe the clinical and legal parameters of the target popula=on OR

any factors that have prevented you from selec=ng a target popula=on

Planning Measures 8–10 Ø

8. Which of the following interven=on types do you intend to implement? A.  Law enforcement oﬃcer and mental health cross-‐training B.  Jail diversion program C.  Mental health court D.  Specialized proba=on supervision E.  Other (Please explain)

Ø

9. How many people are projected to par=cipate in the program during the grant period?

Ø

10. Do you plan to use a validated risk/needs assessment tool? A.  Yes B.  No C.  If yes, what tool will you use? 34

Planning Measures 11–14 Ø

11. What kind of data will you collect? A.  Process data B.  Outcome data C.  Both D.  None of the above

Ø

12. Describe the data to be collected and who will collect it.

Ø

13. Describe the source of any addi=onal funding that will be used to implement your ini=a=ve.

Ø

14. Describe any outcomes or programma=c changes that were observed since the previous repor=ng period.

Law Enforcement Purpose Area (11 Ques2ons)

Law Enforcement Ques2ons 1–2:

Training in Specialized Responses to People with Mental Illness

Ø

1. During the repor=ng period, how many oﬃcers and staﬀ were trained in specialized police-‐based responses to people with mental illness?

Ø

2. As of the last day of the repor=ng period, how many oﬃcers and staﬀ were available to provide a specialized police-‐based response to people with mental illness?

Law Enforcement Ques2ons 3–4:

Training in Specialized Responses ( con=nued) Ø

3. What types of agencies were represented in training sessions you conducted during the repor=ng period? Check all that apply. A.  Judges B.  Law enforcement C.  Community correc=ons oﬃcers D.  Alorneys E.  Proba=on and parole oﬃcers F.  Program evaluators G.  Other

Ø

4. What training curriculum did you use to train specialized police-‐based responses to people with mental illness? Please enter N/A if not applicable. 38

Law Enforcement Ques2ons 5–6:

Calls Received for Mental Health–Related Events Ø

5. How many total calls for police services were received during the repor=ng period? A.  Of these, how many were for mental illness events? B.  Of these, how many did a specially trained oﬃcer respond to?

Ø

6. Describe the protocol you use to determine if a call is related to a mental health–related event.

Law Enforcement Ques2ons 7–8:

Field Assessment and Screening for Mental Illness Ø

7. During the repor=ng period, how many people received a ﬁeld assessment or screening administered by someone trained to provide a specialized police-‐based response to people with mental illness?

Ø

8. Describe the training they received.

Law Enforcement Ques2on 9:

Disposi=on of People Involved in a Mental Health Crisis Ø

9. What was the disposi=on of people involved in a mental health crisis that involved a law enforcement response? How many people were: A.  Arrested and/or taken into police custody B.  Len on site C.  Taken to an emergency room or other medical facility D.  Taken to a mental health facility E.  Taken to a nonmedical or non-‐mental health–speciﬁc community

agency or organiza=on

Law Enforcement Ques2ons 10–11:

Injuries Incurred during Mental Illness–Related Events Ø

10. How many oﬃcers were injured while responding to events during the repor=ng period involving people with mental illness ?

Ø

11. How many civilians were injured during the repor=ng period in one or more mental health–related events?

Direct Services Purpose Areas (Everything That Isn’t Law Enforcement)

Court-‐based Diversion 37 Ques2ons Reentry Programs

37 Ques2ons

Community Correc2ons 37 Ques2ons In-‐facility Correc2ons

34 Ques2ons 43

Training (4 Ques2ons)

Direct Services Ques2ons 1–2:

Training in Mental Health and Criminal Jus=ce Issues Ø

1. How many staﬀ members and partners received specialized cross-‐training in responding to mental illness and criminal jus=ce incidents during the repor=ng period?

Ø

2. As of the last day of the repor=ng period, how many staﬀ members and/or partners were trained and available to respond to incidents involving people with mental illness?

Direct Services Ques2ons 3–4: Training (con=nued) Ø

3. What types of agencies or individuals were represented in training sessions you conducted during the repor=ng period? A.  Judges B.  Law enforcement C.  Community correc=ons oﬃcers D.  Alorneys E.  Proba=on and parole oﬃcers F.  Program evaluators G.  Other

Ø

4. What curriculum was used to cross-‐train staﬀ members and partners to respond to incidents involving people with mental illness? Please enter N/A if not applicable.

Program Par2cipants (5 Ques2ons)

Direct Services Ques2on 5: Program Par=cipants Ø

5. How many individuals were eligible to par=cipate in the program? A.  Of those eligible, how many were selected to par=cipate? B.  Of those selected, how many NEW par=cipants were admiled? C.  Of those selected, how many par=cipated in the program during the

repor=ng period?

Direct Services Ques2ons 6–9:

Risk/Needs Assessment and Case Plan Ø

6. Does the program use a risk and need assessment? A.  Yes B.  No (Skip next 2 ques=ons)

Ø

7. Describe the risk and need assessment instrument(s) you used: _______________

Ø

8. How many of the people who received a risk assessment during the repor=ng period were assessed as: A.  Low risk? B.  Moderate risk? C.  High risk?

Ø

9. How many par=cipants received a transi=on or case plan for the ﬁrst =me during the repor=ng period? 49

Providing Treatment (9 Ques2ons)

Direct Services Ques2ons 10–12: Treatment for Mental Illness Ø

10. How many people were assessed for mental health services? A.  Of these, how many were recommended for treatment?

Ø

11. How many people did you treat for mental illness during the repor=ng period? A.  Of these, how many par=cipants were NEW during the repor=ng

period?

Ø

12. How many people did you refer to other agencies to receive treatment for mental illness during the repor=ng period?

Direct Services Ques2ons 13–15: Treatment for Substance Abuse Ø

13. How many people were assessed for substance abuse treatment services? A.  Of these, how many were recommended for treatment?

Ø

14. How many people did you treat for substance abuse during the repor=ng period? A.  Of these, how many par=cipants were NEW during the repor=ng

period?

Ø

15. How many people did you refer to other agencies to receive treatment for substance abuse during the repor=ng period?

Direct Services Ques2ons 16–18: Treatment for Co-‐occurring Disorders Ø

16. How many people were assessed for co-‐occurring disorder treatment services? A.  Of these, how many were recommended for treatment?

Ø

17. How many people did you treat for co-‐occurring disorders during the repor=ng period? A.  Of these, how many par=cipants were NEW during the repor=ng

period?

Ø

18. How many people did you refer to other agencies to receive treatment for co-‐occurring disorders during the repor=ng period?

Providing Services (12 Ques2ons)

Direct Services Ques2ons 19–21: Employment Services Ø

19. How many people were assessed for employment services? A.  Of these, how many were recommended for treatment?

Ø

20. How many people did you provide with employment services during the repor=ng period? A.  Of these, how many par=cipants were NEW during the repor=ng

period?

Ø

21. How many people did you refer to other agencies to receive employment services during the repor=ng period?

Direct Services Ques2on 22: Employment Services (Outcome) Ø

22. How many par=cipants who were directly provided with employment services: A.  Obtained employment? B.  Maintained employment for 3 or more months? C.  Maintained employment for 6 or more months?

Direct Services Ques2ons 23–25: Educa=on Services Ø

23. How many people were assessed for educa=on services? A.  Of these, how many were recommended for treatment?

Ø

24. How many people did you provide with educa=on services during the repor=ng period? A.  Of these, how many par=cipants were NEW during the repor=ng

period?

Ø

25. How many people did you refer to other agencies to receive educa=on services during the repor=ng period?

Direct Services Ques2on 26: Educa=on Services (Outcome) Ø

26. How many par=cipants who were directly provided with educa=on services: A.  Earned a GED cer=ﬁcate? B.  Earned a high school diploma? C.  Earned a voca=onal cer=ﬁcate? D.  Earned a higher educa=on degree? E.  Maintained employment for 3 or more months? F.  Maintained employment for 6 or more months?

Direct Services Ques2ons 27–29: Housing Services Ø

27. How many people were assessed for housing services? A.  Of these, how many were recommended for treatment?

Ø

28. How many people did you provide with housing services during the repor=ng period? A.  Of these, how many par=cipants were NEW during the repor=ng

period?

Ø

29. How many people did you refer to other agencies to receive housing services during the repor=ng period?

Direct Services Ques2on 30: Housing Services (Outcome) Ø

30. How many par=cipants who were directly provided with housing services: A.  Obtained housing? B.  Were housed for 3 or more months? C.  Were housed for 6 or more months?

Program Comple2on and Recidivism (7 Ques2ons)

Direct Services Ques2ons 31–32: Program Comple=on Ø

31. During the repor=ng period, how many par=cipants successfully completed the program?

Ø

32. Please enter the number of individuals who did not complete the program due to: A.  Court or criminal involvement (technical viola=on, arrest, convic=on,

revoca=on, reincarcera=on) † B.  Lack of engagement (no-‐shows and nonresponsive par=cipants) C.  Absconding† D.  Reloca=on or case transfer E.  Death or serious illness

† = not shown for in-‐facility correc=ons

Direct Services Ques2ons 33–34: Recidivism of Current Par=cipants† Ø

33. How many current par=cipants were arrested during the repor=ng period for: A.  Administra=ve viola=ons of their condi=on of supervision B.  New oﬀenses

Ø

34. How many current par=cipants were sent to jail or prison during the repor=ng period for: A.  Administra=ve viola=ons of their condi=on of supervision B.  New oﬀenses † = not shown for in-‐facility correc=ons

Direct Services Ques2ons 35–36:

Recidivism of Current Par=cipants( con=nued) † Ø

35. How many days total did current par=cipants spend in jail or prison during the repor=ng period as a result of: A.  Administra=ve viola=ons of their condi=on of supervision B.  New oﬀenses

Ø

36. How many days total did current program par=cipants spend in a hospital or inpa=ent mental health treatment facility because of a mental health crisis?

† = not shown for in-‐facility correc=ons 64

Direct Services Ques2on 37:

Recidivism of Successful Completers Ø

37. How many people successfully completed the program within the past 12 months? A.  Of those, how many were arrested during the repor=ng period for the

first =me since program comple=on? B.  How many of those arrests were for administra=ve viola=ons of their condi=on of supervision (not involving a new offense charge)? C.  How many were for new offenses?

Narra2ve Report (7 Ques2ons)

Narra2ve Ques2ons 1–4 Ø

1. What were your accomplishments this repor=ng period?

Ø

2. What goals were accomplished, as they relate to your grant applica=on?

Ø

3. What problems/barriers did you encounter that prevented you from reaching your goals or milestones?

Ø

4. Is there any assistance that BJA can provide to address any problems/barriers iden=ﬁed in ques=on #3?

Narra2ve Ques2ons 5–7 Ø

5. Are you on track to ﬁscally and programma=cally complete your program as outlined in your grant applica=on?

Ø

6. What major ac=vi=es are planned for the next 6 months?

Ø

7. Based on your knowledge of the criminal jus=ce ﬁeld, are there any innova=ve programs/accomplishments that you would like to share with BJA?

Ques2ons (and Answers)

Planning & Implementa=on Grants

Planning

Opera=onal

Planning Grants

Expansion Grants

Planning

Opera=onal

Law Enforcement

Court-‐based Diversion

In-‐facility Correc2ons

Community Correc2ons

Reentry Programs

Narra=ve

Narra=ve 71

Planning

Opera=onal Law Enforcement Court-‐based Diversion In-‐facility Correc2ons Community Correc2ons Reentry Programs

Narra=ve

Planning Grants

Planning

Opera=onal

Narra=ve

Planning & Implementa=on Grants

Planning

Expansion Grants

Planning

Opera=onal

Law Enforcement

Court-‐based Diversion

In-‐facility Correc2ons

Community Correc2ons

Reentry Programs

Narra=ve

Narra=ve 73

Opera=onal Law Enforcement Court-‐based Diversion In-‐facility Correc2ons Community Correc2ons

Iden2cal

Reentry Programs