Pretrial Recommendation System 05 by Administrator

CJA

NEW YORK CITY CRIMINAL JUSTICE AGENCY NEW YORK CITY CRIMINAL USTICE AGENCY

Jerome E. McElroy Executive Director

AN EVALUATION OF THE NEW PRETRIAL RELEASE RECOMMENDATION SYSTEM IN NEW YORK CITY: PHASE II OF THE POST-IMPLEMENTATION RESEARCH

Qudsia Siddiqi, Ph.D. Project Director

FINAL REPORT June 2005

52 Duane Street, New York, NY 10007

(646) 213-2500

AN EVALUATION OF THE NEW PRETRIAL RELEASE RECOMMENDATION SYSTEM IN NEW YORK CITY: PHASE II OF THE POST-IMPLEMENTATION RESEARCH

Qudsia Siddiqi, Ph.D. Project Director

Data Supervision: Marian Gewirtz Senior Research Analyst Research Assistance: Justin P. Bernstein Research Assistant Elyse J. Revere Junior Research Analyst Systems Programming: Barbara Geller Diaz Associate Director, Information Systems Wayne Nehwadowich Senior Programmer/Analyst Administrative Support: Nyota Muhammad Administrative Associate

June 2005 ÂŠ 2005 NYC Criminal Justice Agency

ACKNOWLEDGEMENTS The author would like to thank Jerome E. McElroy, Executive Director of CJA, and Dr. Richard R. Peterson, Director of Research at CJA, for their timely review and suggestions on the final draft.

The author is grateful to Marian Gewirtz, Senior Research Analyst, for

supervising the extraction and preparation of the data for this report. Special thanks to Barbara Geller Diaz, Associate Director, Information Systems for her expertise. The author appreciates the assistance of Wayne Nehwadowich for extracting the datasets used in this report. The author extends special thanks to Justin P. Bernstein, who worked tirelessly on his contributions to the final draft of the manuscript, and Nyota Muhammad, for her patience and able assistance in constructing and refining innumerable drafts of the tables. Thanks to Elyse J. Revere for preparing the data for analysis. Finally, the author would like to thank the New York State Division of Criminal Justice Services for providing supplemental data for the pre-implementation analysis. The methodology, findings, and conclusions of the study, as well as any errors and omissions are the responsibility of the author.

TABLE OF CONTENTS Introduction..............................................................................................................................1 I.

Old CJA Recommendation System ...............................................................................5

II.

New Recommendation System.......................................................................................7

III. Methodology ..................................................................................................................12 A. Sampling and Data Sources .........................................................................................12 B. Research Design...........................................................................................................14 IV. Results ............................................................................................................................17 A. Defendant Characteristics ............................................................................................17 B. Is the New ROR Recommendation System Consistent Over Time?...........................23 C. Does the New ROR Recommendation System Perform Better than the Old System? ....28 D. What is the Impact of the New Recommendation System on Minority Defendants? ......33 a. Comparing Black and White Defendants ..............................................................39 b. Comparing Hispanic and White Defendants..........................................................41 c. Classification of Defendants into Risk Categories and Their Relative FTA Rates ....41 d. Conclusions on the Impact of the New Recommendation System on Minority Defendants .............................................................................................................44 V.

Summary and Conclusions .........................................................................................45

VI. References .....................................................................................................................48

APPENDIX A: Difference of Proportions Test .................................................................49 APPENDIX B: Mean Cost Rating ......................................................................................50

LIST OF TABLES Table 1

Items in the New Recommendation System .............................................................9

Table 2

New Recommendation System for Adult Defendants ............................................10

Table 3

Arraignment Outcome (Defendant-based) ..............................................................13

Table 4

Release Status for Cases Adjourned at Arraignment (Defendant-based) ....................15

Table 5

Characteristic of Defendants Released Pretrial Regardless of Court of Disposition: A Comparison of the Pre-Implementation At-Risk Sample with the Post-Implementation At-Risk Sample .......................................................18

Table 6

Distribution of Point Scale Score by FTA ...............................................................25

Table 7

New CJA Recommendation System by FTA for Defendants Released Pretrial (Projected and Observed Distributions) ...................................................................27

Table 8

Old CJA Recommendation System by FTA (2001 At-Risk Sample) ......................29

Table 9

Distribution of Old CJA ROR Recommendation System by FTA (PreImplementation At-Risk Sample) ............................................................................30

Table 10

Distribution of New CJA ROR Recommendation System by FTA (Post Implementation At-Risk Sample) ............................................................................31

Table 11

Old Recommendation System: Distribution of Ethnicity by Pretrial FTA for the 2001 At-Risk Sample ............................................................................................35

Table 12

New Recommendation System: Projected Distribution of Ethnicity by Pretrial FTA for the 2001 At-Risk Sample ...........................................................................36

Table 13

Post-Implementation Distributions of Black and White Defendants by FTA ..............40

Table 14

Post-Implementation Distributions of Hispanic and White Defendants by FTA .....................................................................................................................42

Table 15

New Recommendation System: Observed Distribution of Ethnicity by Pretrial FTA for the Post-Implementation Sample ...............................................................43

Table B-1

Mean Cost Rating of FTA by New Point Scale Scores for the 2001 At-Risk Sample ......................................................................................................51

Table B-2

Mean Cost Rating of FTA by New Point Scale Scores for the Post-Implementation At-Risk Sample ......................................................................................................53

AN EVALUATION OF THE NEW PRETRIAL RELEASE RECOMMENDATION SYSTEM IN NEW YORK CITY: PHASE II OF THE POST-IMPLEMENTATION RESEARCH INTRODUCTION The New York City Criminal Justice Agency, Inc. (CJA) makes pretrial releaseon-recognizance (ROR) recommendations for defendants arrested in New York City and held for arraignment in the lower court (Criminal Court). Section 510.30 of the New York State Criminal Procedure Law permits judges to consider many factors including a defendant’s community ties, prior criminal record, the seriousness of the offense, risk of flight, and the weight of the evidence against the defendant, in their pretrial release (and bail) decisions. However, prior to June 30, 2003, CJA’s release recommendation system was based solely upon a defendant's ties to the community. This left it to the arraignment judge to consider, from other sources, all other factors. CJA considered defendants having strong community ties “good risks” to return for scheduled court appearances. The old recommendation system was based on research that was validated in 1974 on a sample of Brooklyn defendants. In the years since that research, CJA’s data continued to show that those defendants recommended for ROR and actually released at arraignment were less likely to fail to appear in Criminal Court than those not recommended. This was true citywide, even though the validation research had been limited to Brooklyn. However, it was not known if the recommendation system had the ability to predict FTA in the upper court (Supreme Court). Furthermore, given the length of time that had elapsed since the last validation, CJA wanted to re-assess the predictive accuracy of the community ties comprising the existing system, and identify other predictors of FTA, if any. The re-evaluation study was originally conducted on a random sample of arrests that were made in the five boroughs comprising New York City in 1989. The study found that although the old system differentiated defendants on risk of flight, there was room

for improvement (Siddiqi, 1999). Therefore, a number of other statistical models were developed. The best model from that analysis was used to develop a point scale by assigning points to all the significant items in the model. The 1989 model was statistically validated on the 1998 and 2001 samples of defendants (Siddiqi, 2000, 2002).

All of the variables that were significant in the

construction sample remained significant in the validation samples. However, for some of the variables, changes were observed in the magnitude of the relationship. The point scale was adjusted to address those changes. Furthermore, changes were also made to address policy concerns. The new point scale was used to develop a new ROR recommendation system. Under that system, point scale scores were used to create low-, medium-, and high-risk categories. The cutoff scores were chosen by considering a number of criteria. The first criterion was to create subgroups of defendants, which would have different average FTA rates, and as such represent different risk levels. The FTA rates under the old system were used as a guide to classify defendants as low risk. It was agreed that the FTA rates for the individual point values comprising the low-risk group should not exceed the average FTA rate for the recommended category of the old system. The cutoff score for the moderate-risk category was guided by the criterion that its FTA rate should be considerably higher than the FTA rate of the low-risk category; and the cutoff score for the high-risk category was selected to include defendants with the highest FTA rate. The second criterion was setting a target proportion of defendants to be recommended for ROR. It was agreed that the proportion of defendants in the low-risk category of the new system should be higher than that under the old system, without increasing the FTA rate for that category The logistic regression model used to develop the new point scale did not control for the effect of ethnicity, and therefore could result in misclassification of minority defendants. To determine whether this was an issue for the new recommendation system, -2-

the final logistic regression model was re-estimated with ethnicity as an additional control variable. The points derived from re-estimated model were identical to those computed for the original model. This finding suggested that the point scale based on the original model would classify defendants without direct bias related to ethnicity. The new ROR recommendation system was presented to criminal justice practitioners for their feedback. In those meetings, concern was expressed over the reliability of the open case variable. CJA interviewers use the rap sheet to record the number of open cases on the interview form. Several judges and defense attorneys believed that the information on the rap sheet is often inaccurate, as many of the open cases are no longer open at the time of the CJA interview. Defendants would then lose points due to inaccurate information. Suggestions were also made to limit the prior FTA information to ten years prior to the sample arrest.1 To address these concerns, the two variables were further examined. Based on the findings from that analysis, the value for the open case variable was reduced from two points to one point in the final scale.2 No changes were made to the prior FTA variable.3

1 The prior FTA variable provides an overall count of failure to appear prior to the sample arrest. It also includes post-disposition and post-sentencing warrants. Out of state warrants and warrants issued on summon appearances (SAP) are excluded from this count. 2 For analysis, it was assumed that half of the defendants with open cases did not actually have an open case at the time of arraignment. A new open case variable was computed by randomly selecting 50 percent of the defendants with open cases and assigning them a value of zero (no open case). This new variable was substituted with the original variable and the final model was re-estimated. The new open-case variable was also a significant predictor of pretrial FTA. Furthermore, the value of the points assigned to the revised variable did not change. Since it was not known which defendants were incorrectly identified as having an open case, the values for the revised open case variable were flipped. Defendants who had at least one open case both in the original and the revised variable were collapsed with defendants having no open cases. The remaining half of the defendants who were assigned a value of zero were recoded as having one open case. When included in the logistic regression model, the variable was still significant. However, the point value decreased to one point. 3The analysis of the prior FTA variable at a 10 year cutoff point revealed that its effect on pretrial FTA was similar to that for the FTA at any time prior to the sample arrest; after rounding, both would contribute five points to a defendantâ&#x20AC;&#x2122;s total score on the new point scale. Therefore, for practical reasons, the original variable was kept in the point scale; recording this information would be easier and faster.

-3-

In January of 2003, the new recommendation was presented to the Criminal Justice Coordinator. At his recommendation, the cutoff scores for the moderate-risk group were slightly altered. The new recommendation system has several advantages over the old recommendation system. It takes into consideration a defendantâ&#x20AC;&#x2122;s community ties and criminal history. The old system, on the other hand, assessed a defendantâ&#x20AC;&#x2122;s risk of flight based solely on his/her ties to the community.

Secondly, the new system is based on

research that was conducted on a citywide sample of defendants. The old system was based on research that was conducted on a sample of Brooklyn defendants only. Thirdly, the new system identifies the at-risk population regardless of the court of disposition. The old CJA recommendation system was limited in its application; it identified the at-risk population exclusively in Criminal Court. Fourth, the new system classifies defendants by their relative risk of flight. The old recommendation system did not differentiate lowrisk defendants from moderate-risk defendants in the 1998 and 2001 at-risk samples. Finally, relative to the old CJA recommendation system, the new system classifies a substantially higher proportion of defendants as low risk, while keeping the FTA rate at the same level. The new system was implemented citywide on June 30, 2003. To measure the performance of the new recommendation system, pre- and postimplementation studies were conducted. The pre-implementation research was based on a cohort of arrests made in the third quarter of 2002 (July 1 through September 30, 2002). It was completed in December of 2003 (Siddiqi, 2003). The post-implementation research was conducted on a cohort of arrests made between November 1, 2003 and January 31, 2004. The data for the post-implementation analysis were extracted in two phases. In Phase I, a defendantâ&#x20AC;&#x2122;s arrest and arraignment information was drawn from the CJA database. This information was used to examine the distribution of defendants under the new recommendation system and judicial release decision-making at arraignment.

In the second phase, data were drawn on post-4-

arraignment appearances, including bench warrants that were issued pretrial. The Phase II data were used to determine whether the new recommendation system performed better than the old recommendation system. The current report describes findings from Phase II of the post-implementation research. The report is organized into five sections. The first two sections describe the old and new recommendation systems.

The third section presents the research

methodology. Section four reports the findings from the analysis of data that were extracted in Phase II. The last section summarizes and concludes the report.

I. OLD CJA RECOMMENDATION SYSTEM The old CJA release recommendation system for adult defendants (15 years of age or older) was comprised of the following six items: 1. whether there was a working telephone in the defendant's residence; 2. whether the defendant had resided at his or her current address for one and one-half years or longer; 3. whether the defendant expected someone (other than the complainant or defense attorney) at Criminal Court arraignment; 4. whether the defendant lived with (a) parent(s), spouse, common-law spouse, grandparent(s), or legal guardian; 5. whether the defendant was employed, in school, or in a job-training program (or a combination of these) full time; 6. whether the defendant's address was in the New York City area (the five boroughs of New York City, Nassau, Suffolk, and Westchester counties). With the exception of expecting someone at arraignment, there were five possible outcomes for each item: "yes," "yes verified," "no," "no verified," and "unresolved conflict." The "yes" and "no" outcomes indicated that the defendant's response had not been verified.

The "yes verified" and "no verified" outcomes were used when the -5-

information provided by the defendant had been verified through a third-party contact. “Unresolved conflict” meant that the information provided by the defendant in the interview did not match the information given by the verifier and attempts to resolve the conflict were unsuccessful. Expecting someone at arraignment had only "yes" and "no" responses. That information was used to determine a defendant’s risk of flight in Criminal Court. Under the old system, a defendant could receive one of the following recommendations: 1. "Recommended": Verified Community Ties (defendant needed a verified New York City area address, and items 2, 4, or 5 verified, and at least two other true items); 2. "Qualified": Unverified Community Ties (defendant had an unverified New York City area address and had three other items assessed in the affirmative); 3. No Recommendation due to: A. Insufficient community ties (less than three items were answered affirmatively) B. Residence outside the New York City area C. Conflicting residence information (defendant and verifier did not agree) D. Incomplete interview; 4. No Recommendation due to: A. Open bench warrant attached to the New York State criminal history sheet B. Criminal history not available C. Bail jumping charge D. For Information Only: murder charge The first three recommendations summarized the strength of the defendant’s community ties. The fourth category excluded from the ROR recommendation those defendants who had an active bench warrant at the time of the interview and those for -6-

whom the absence of a rap sheet precluded ascertaining that information. Defendants were also excluded if arrested on a bail jumping offense, which may be charged in New York State after a defendant does not return to court for thirty days or more after failing to appear while on bail or ROR. Previous failures to appear, for which the defendant returned to court and the warrant was vacated, did not preclude ROR recommendation on a new arrest. Finally, defendants arrested for murder charges did not receive a recommendation.

II. NEW RECOMMENDATION SYSTEM CJA’s new recommendation system is based on a defendant’s ties to the community as well as criminal history. It is comprised of the following items: 1. Does defendant report a NYC area address? 2. Does defendant have a working telephone in his or her residence or a cellular phone? 3. Is defendant employed, or in school, or training program (or a combination of these) full time? 4. Does defendant expect someone at arraignment? 5. Does the prior bench warrant count equal zero? 6. Does the open case count equal zero? Questions 1 through 4 were also part of the old recommendation system. Questions 5 and 6 are new additions to the point scale. Questions 1 through 3 have five response options: "yes," "yes verified," "no," "no verified," and "unresolved conflict." As in the old recommendation system, the "yes" and "no" outcomes indicate that the defendant's response has not been verified, the "yes verified" and "no verified" outcomes are used when the information provided by the defendant has been verified through a third-party contact, and “unresolved conflict” means that the information provided by the defendant in the interview does not match the information given by the verifier and -7-

attempts to resolve the conflict were unsuccessful. Questions 4 through 6 have only "yes" and "no" options. Under the new system, points are assigned to a defendant’s response to the above items. Table 1 presents the points assigned under the new system. As shown by the table, the prior bench warrant item contributes the most to a defendant’s total score; defendants with a prior bench warrant have five points subtracted from their total score, whereas defendants with no history of failure to appear have five points added. A New York City area address is also a strong contributor to a defendant’s total score on the point scale; defendants categorized as “yes verified” have three points added to their point scale scores. Defendants with a negative response (“no” or “no verified”) to that question lose two points.

Expecting someone at Criminal Court

arraignment and having an open case contribute least to a defendant’s total score. Defendants who expect someone at arraignment, or have no open cases, score only one point on each item. Defendants who do not expect anyone at arraignment or have an open case lose one point on each. With respect to the remaining variables in the system, defendants with affirmative responses (“yes” and “yes verified”) to having a telephone in their residence, or a cellular phone, earn one point on the new scale. In contrast, defendants recorded as “no” or “no verified” score negative two points. Because of their statistical insignificance, no points are assigned to defendants who were categorized as “unresolved conflict” on the telephone variable. With regard to being engaged in a full-time activity, defendants with affirmative or verified affirmative responses score one point. Defendants recorded as “no” or “no verified” score negative one point. Defendants categorized as “unresolved conflict” receive negative two points. A defendant’s total score can range from -12 to +12 points and is used to determine CJA’s new release recommendation. The new recommendation system classifies defendants into four categories, which are presented in Table 2. The table also displays the points required to receive one of the recommendations. -8-

Table 1: Items in the New Recommendation System

2 3

4 5 6

Does the defendant have a working telephone in his or her residence? Does the defendant report a NYC area address? Is the defendant employed, or in school, or training program, full time? Does the defendant expect someone at arraignment? Does the prior bench warrant count equal zero? Does the open case count equal zero?

Yes

Yes Verified

No Verified

Unresolved Conflict

-2

-1

-2

-1

-5

-1

Subtotals (Y+YV), (N+NV+UC) Total Score (Y+NV)-(N+NV+UC)

-9-

Table 2: New Recommendation System for Adult Defendants RECOMMENDATION

POINTS

A. RECOMMENDED FOR ROAR

+ 7 Points to + 12 Points

B. MODERATE RISK FOR ROR

+ 3 Points to + 6 Points

C. NOT RECOMMENDED FOR ROR 1. Bench Warrant Attached to NYSID 2. Bail-Jumping Charge 3. Conflicting Residence Information 4. High Risk for FTA

+ 2 Points to - 12 Points

D. NO RECOMMENDATION 1. No NYSID Available 2. For Information Only 3. Interview Incomplete

-10-

The first two categories, A and B, and fourth subcategory of category C are based on a defendant’s total score on the point scale. In order to be categorized as “Recommended for ROR,” a defendant needs seven or more points. This is CJA’s highest positive rating for ROR recommendation.

Defendants with scores ranging

from three to six points receive the second highest recommendation, “Moderate Risk for ROR.” Defendants scoring less than three points are considered high risk and are “Not Recommended for ROR.” The third and fourth categories of the new recommendation system reflect policy exclusions, with the exception of C-4. These exclusions were also applied under the old system. The “Not Recommended for ROR” category includes defendants with a bench warrant attached to the rap sheet (C-1), defendants charged with bail jumping (C-2), defendants with conflicting residence information (C-3) and defendants considered at high risk for FTA due to their point scale scores (C-4). No recommendation is offered to defendants with a missing NYSID (D-1), defendants charged with murder (D-2), or those with an incomplete interview (D-3). The order of priority used to assign a stamp is D (1,2,3), followed by C (1,2,3,4), followed by B and then A. Thus, defendants with any of the policy exclusions listed in C or D are not recommended for ROR even if their point scale score is well above 7.

-11-

III. METHODOLOGY A. Sampling and Data Sources Data for Phase II of the post-implementation analysis were drawn from a cohort of arrests made November 1, 2003 through January 31, 2004. The primary data source was the CJA database.4 The data set contained 82,932 arrests. The arrest volume was slightly lower than that for the pre-implementation dataset (86,997). Eight percent of the arrestees were given Desk Appearance Tickets (DATs). This was two percentage points higher than that reported for the pre-implementation dataset. The remaining 92 percent were held for arraignment in Criminal Court (summary arrests). Since CJA does not make recommendations for arrestees with DATs, they were dropped from the analysis. The Criminal Court data were tracked through March 30, 2004.5 By that time, 98 percent of the cases had reached a disposition in Criminal Court. The Supreme Court data were extracted on December 31, 2004. Approximately 20 percent of the cases had not reached final outcomes by that date. In the post-implementation dataset, 10 percent of the defendants had multiple arrests. Because the objective of the point scale analysis was to examine defendant behavior, the arrest-based file was converted into a defendant-based file, where only the defendantâ&#x20AC;&#x2122;s first arrest during the sampling period was taken. This file contained 47, 914 defendants.6 Their arraignment dispositions are presented in Table 3. For comparison purposes, the distributions for the pre-implementation dataset are also provided. As shown in Table 3, in the post-implementation dataset, 17 percent of the defendants had their cases dismissed at arraignment. Slightly more than one-fourth pled

Information about the arrest is provided by an on-line feed from the New York City Police Department. If a case had multiple dockets, the Criminal Court information, including warrants, was pulled from the docket having the most severe affidavit charge (Penal Law severity). 6 This number excludes juvenile delinquents, and non-juveniles, whose cases were transferred to Family Court prior to arraignment, voided arrests, defendants who were declined prosecution, defendants who had a missing stamp under the new recommendation system and defendants who were under sixteen years of age. 5

-12-

Table 3: Arraignment Outcome (Defendant-based)

ARRAIGNMENT OUTCOME

Pre-Implementation Dataset

Post-Implementation Dataset

(Jul. 1-Sept. 30, 2002)

(Nov. 1, 2003-Jan. 31, 2004)

N=56,413

N=47,914

NON-DISPOSED

31,256

26,483

PLED GUILTY

15,563

13,480

DISMISSED

9,569

7,933

56,413

100

47,914

100

OTHER1

TOTAL 1

Other includes transfer to other borough.

-13-

guilty and more than one-half had cases adjourned pending further appearances. These distributions were similar to those reported for the pre-implementation dataset. The analysis focused on defendants whose cases were not completed at Criminal Court arraignment that were released on recognizance, or released on bail prior to the disposition of all charges in Criminal or Supreme Court. Table 4 presents the release status for defendants whose cases were adjourned at Criminal Court arraignment. As shown by the table, in the post-implementation dataset, 16,881 (65%) defendants were released at arraignment, 15,936 (61%) were RORâ&#x20AC;&#x2122;d and 945 (4%) made bail. The table further shows that the proportion of defendants who were RORâ&#x20AC;&#x2122;d at arraignment was higher than that reported for the pre-implementation sample (61% versus 56%). In the post-implementation sample, of those not released at arraignment, 4,837 defendants were RORâ&#x20AC;&#x2122;d or made bail subsequent to arraignment in Criminal or Supreme Court (data not shown by Table 4). To be specific, the study sample contained 21,718 defendants who were released pretrial in Criminal or Supreme Court. This was 82 percent of the defendants whose cases were adjourned at Criminal Court arraignment.

B. Research Design The performance of the new ROR recommendation system after its implementation was determined by answering the following questions: First, is the new recommendation system consistent over time? Second, does the new system perform better than the old system? The first question was addressed in several ways. First, we compared the pre-and post-implementation distributions of defendants classified and their corresponding FTA rates by point scale scores. Second, we examined the predictive power of the point scale at two time periods (pre- and post-implementation). We applied a difference of proportion test to determine if the differences between the two samples were statistically significant (see Appendix A for computation). We used mean cost rating (MCR) as a measure of the predictive power. MCR, introduced by Duncan, Ohlin, -14-

Table 4: Release Status for Cases Adjourned at Arraignment (Defendant-based) Pre-Implementation

Post-Implementation

Dataset2

Dataset RELEASE STATUS AT ARRAIGNMENT

394

320

BAIL SET, NOT MADE

11,863

9,108

BAIL MADE

1,373

945

ROR

17,375

15,936

31,005

26,309

100

REMAND

TOTAL3 1

The release status at arraignment was missing for 251 defendants.

The release status at arraignment was missing for 174 defendants.

Of those not released at arraignment in the post-implementation sample, 4,837 were later released pretrial

in Criminal or Supreme Court.

-15-

Reiss, and Stanton (1953), measures the predictive efficiency of an instrument over its base rate (see Appendix B for MCR computation). The values for this statistic range from 0 to 1, with 0 indicating no improvement in prediction and 1 suggesting perfect prediction. As a general rule, an MCR of .25 shows utility for classification; a score of .35 or greater indicates significant improvement over existing clinical techniques (Fischer, 1985).

Third, we compared the projected and observed distributions of

defendants in various risk groups and their corresponding pretrial FTA rates. To ease interpretation, the third and fourth categories of both old and new recommendation systems were collapsed into one category, which was labeled as â&#x20AC;&#x153;Not Recommended for ROR.â&#x20AC;? More specifically, this category included defendants who were either considered high risk for an ROR recommendation or who, for policy reasons, did not receive a CJA recommendation. The pretrial FTA measured the issuance of a bench warrant at any appearance prior to the disposition of a defendant's case in Criminal or Supreme Court. To determine whether the new ROR recommendation system performs better than the old system, we compared the two systems with respect to the distribution of defendants recommended for ROR and their corresponding FTA rate.

Second, we

compared the two systems with respect to their ability to classify defendants on the basis of risk of flight. We also examined the impact of the new recommendation system on minority defendants. To determine whether the new recommendation system fairly classifies minority defendants, we compared the observed failure rates for Black and Hispanic defendants with those for White defendants.

-16-

IV. RESULTS A. Defendant Characteristics Table 5 displays characteristics of defendants for the Phase II postimplementation sample. The sample includes defendants whose cases were continued at arraignment, and who were released pretrial in Criminal or Supreme Court. To facilitate comparisons, distributions from the pre-implementation at-risk sample are also provided. As shown by the table, in both samples, the majority of the defendants were male. Slightly less than half were Black, one-third were Hispanic and slightly more than onetenth were White. The median age in the post-implementation dataset was 29 years. In the post-implementation sample, Manhattan had the highest proportion of defendants whose cases were adjourned at arraignment and were released pretrial (30%). This was followed by Brooklyn (26%), the Bronx (22%), Queens (19%), and Staten Island (3%).

The pre-implementation sample had similar distributions, with the

exception of the Bronx, where slightly fewer defendants were released pretrial (19% in the pre-implementation sample). The average FTA rates for the pre-and postimplementation samples were virtually the same (15% for the former, 14% for the latter). Examining the community-ties items, an overwhelming majority of the defendants in both samples reported living at a New York City area address. Three-fifths reported living with someone at the time of their arrest. Slightly less than three-fourths reported living at their current address for 18 months or longer. Relative to the preimplementation at-risk sample, a slightly higher proportion of the defendants in the postimplementation sample reported having a telephone or cellular phone (74% for the preimplementation sample, 78% for the post-implementation sample). In both samples, half of the defendants reported being employed, in school, or in a training program full time. The proportion of defendants who expected someone at Criminal Court arraignment

-17-

Table 5: Characteristic of Defendants Released Pretrial Regardless of Court of Disposition: A Comparison of the Pre-Implementation At-Risk Sample with the Post-Implementation At-Risk Sample Pre-Implementation At-Risk Sample N=25,024 Defendant Characteristics

Post-Implementation At-Risk Sample N=21,718

20,927 4,094 25,021

84 16 100

18,246 3,468 21,714

84 16 100

Black Hispanic White Other1 Total

11,410 8,551 3,241 1,485 24,687

46 35 13 6 100

9,689 7,435 2,768 1,522 21,414

45 35 13 7 100

Age at Arrest 18 and under 19-20 years 21-24 years 25-29 years 30-34 years 35- 39 years 40 and older Total Median Age (Years)

2,616 2,272 4,059 3,915 3,299 3,218 5,645 25,024 29

10 9 16 16 13 13 23 100

2,456 1,813 3,412 3,417 2,756 2,652 5,212 21,718 29

11 8 16 16 13 12 24 100

6,705 7,747 4,721 978 4,873 25,024

27 31 19 4 19 100

5,631 6,439 4,117 684 4,847 21,718

26 30 19 3 22 100

SOCIO-DEMOGRAPHIC ATTRIBUTES Sex Male Female Total Ethnicity

CASE-PROCESSING CHARACTERISTICS Borough of Arrest Brooklyn Manhattan Queens Staten Island Bronx Total

Page 1 of 4

-18-

Table 5 (contd.) Pre-Implementation At-Risk Sample N=25,024 Defendant Characteristics CASE-PROCESSING CHARACTERISTICS

Post-Implementation At-Risk Sample N=21,718

3,845 21,179 25,024

15 85 100

3,001 18,717 21,718

14 86 100

Yes Unverified Yes Verified No, Unverified No Verified Unresolved Conflict Total

15,314 7,174 1,337 177 290 24,292

63 29 6 1 1 100

14,957 4,459 1,262 95 157 20,930

72 21 6 1 1 100

Verified Length of residence of at least 18 months Yes Unverified Yes Verified No, Unverified No Verified Unresolved Conflict Total

11,576 5,699 5,144 1,398 489 24,306

48 23 21 6 2 100

11,555 3,653 4,753 805 255 21,021

55 17 23 4 1 100

Verified Family Ties Within Residence Yes Unverified Yes Verified No, Unverified No Verified Unresolved Conflict Total

9,075 5,595 7,592 1,667 343 24,272

37 23 31 7 2 100

9,238 3,658 7,079 912 162 21,049

44 17 34 4 1 100

FTA Regardless of Court of Disposition Yes No Total COMMUNITY-TIES ITEMS Verified NYC Area Address

Page 2 of 4

-19-

Table 5 (contd.) Pre-Implementation At-Risk Sample N=25,024 Defendant Characteristics

Post-Implementation At-Risk Sample N=21,718

8,162 16,021 24,183

34 66 100

6,571 14,370 20,941

31 69 100

11,141 6,901 5,086 402 714 24,244

46 28 21 2 3 100

11,891 4,414 4,161 144 379 20,989

57 21 20 1 2 100

8,680 4,055 8,225 2,870 435 24,265

36 16 34 12 2 100

8,672 2,618 7,777 1,715 253 21,035

41 12 37 8 1 100

11,175 13,599 24,774

45 55 100

9,966 11,638 21,604

46 54 100

5,414 19,376 24,790

22 78 100

4,883 16,755 21,638

23 77 100

6,868 17,922 24,790

28 72 100

6,024 15,614 21,638

28 72 100

COMMUNITY-TIES ITEMS Expects Someone at Arraignment Yes No Total Verified Telephone Yes Unverified Yes Verified No, Unverified No Verified Unresolved Conflict Total Verified Full Time Employment/ School/ Training Yes Unverified Yes Verified No, Unverified No Verified Unresolved Conflict Total

CRIMINAL HISTORY First Arrest Yes No Total Prior Felony Convictions Yes No Total Prior Misdemeanor Convictions Yes No Total

Page 3 of 4

-20-

Table 5 (contd.) Pre-Implementation At-Risk Sample N=25,024 Defendant Characteristics

Post-Implementation At-Risk Sample N=21,718

Open Cases Yes No Total

4,891 19,899 24,790

20 80 100

3,766 17,872 21,638

17 83 100

Type of Warrant Attached to Rap Sheet Bench Warrant No Bench Warrant Total

965 23,818 24,783

4 96 100

800 20,688 21,488

4 96 100

6,222 18,804 25,026

25 75 100

5,508 15,760 21,268

26 74 100

308 3,435 1,500 5,379 2,510 9,101 1,606 1,047

1 14 6 22 10 37 6 4

195 2,502 1,241 4,040 2,112 8,597 1,532 1,334

1 12 6 19 10 40 7 6

24,886

100

21,553

100

8,958 3,404 5,224 1,134 820 428 814 4,072 24,854

36 14 21 5 3 2 3 16 100

7,005 3,047 4,340 1,013 733 573 977 3,842 21,530

33 14 20 5 3 3 5 18 100

Prior FTA Yes No Total TOP ARREST CHARGE SEVERITY A Felony B Felony C Felony D Felony E Felony A Misdemeanor B Misdemeanor Other Total

TOP ARREST CHARGE TYPE Violent Property Drug Weapon Criminal Mischief VTL DWI Other Total 1

Other includes Asian, American Indian, and others.

Other includes Unclassified Misdemeanors, Violations, Infractions, and charges outside the N.Y. State Penal Law and Vehicle and Traffic Law (e.g., Administrative and Public Health Codes).

-21-

decreased from 34 percent in the pre-implementation sample to 31 percent in the postimplementation sample. Table 5 further shows a decline from the pre-implementation sample to the postimplementation sample in the proportion of defendants whose affirmative responses for the community-ties items were verified. The differences ranged from eight-percentage points for living at a New York City area address (29% for the preimplementation sample, 21% for the post-implementation sample) to four percentage points for being engaged in a full-time activity (16% for the preimplementation sample, 12% for the post-implementation sample).

The differences

between verified, affirmative responses to having a telephone or a cellular phone, living at current address for 18 months or longer and living with someone were seven percentage points, six percentage points and six percentage points, respectively. Table 5 also shows a slight decrease in the proportion of defendants with verified negative responses for some of the items. To be specific, the proportion of defendants with verified negative responses for the full-time activity item decreased from 12 percent in the pre-implementation sample to eight percent in the post-implementation sample. Similarly, the proportion of defendants with verified negative responses to living with someone dropped from seven percent to four percent. The differences for length of time at current address and having a telephone or cellular phone were three percentage points and one percentage point, respectively. The two samples did not differ with respect to verified negative responses for the New York City area address variable. It should be noted that in both samples, only one percent of the defendants had a verified negative response to that item. With the exception of open cases, the two samples were very similar with respect to the various measures of a defendantâ&#x20AC;&#x2122;s criminal history. To be specific, in both pre-and post-implementation samples, slightly over one-half of the defendants had been arrested previously. Slightly more than one-quarter had been convicted previously on -22-

misdemeanor charges. Approximately one-fourth had (a) prior felony conviction(s). Onefourth had a history of FTA. Four percent had an open bench warrant attached to their rap sheet. The proportion of defendants with an open case at the time of sample arrest decreased from 20 percent in the pre-implementation sample to 17 percent in the post-implementation sample. As shown by Table 5, the proportion of defendants arrested for a felony charge decreased from 53 percent in the pre-implementation at-risk sample to 48 percent in the post-implementation at-risk sample. This was mainly due to a decrease in defendants arrested for B and D felonies. In contrast, the proportion of defendants arrested for misdemeanor or lesser charges increased from 47 percent in the pre-implementation sample to 53 percent in the post-implementation sample. The two samples were very similar with respect to the type of the top arrest charge, with the exception of violent offenses. The proportion of defendants arrested for violent offenses dropped from 36 percent in the pre-implementation sample to 33 percent in the post-implementation sample. To summarize the findings from this section, the two samples were quite similar with respect to most of their characteristics. The exceptions included type and severity of the top arrest charge, open cases and expecting someone at arraignment.

B. Is the New ROR Recommendation System Consistent Over Time? To assess the consistency of the new recommendation system over time, pre- and post-implementation distributions of defendants, and their corresponding FTA rates were examined. For pre-implementation projections we used both 2001 and 2002 samples. Table 6 compares the distributions of point scale score by FTA for both the 2001 and post-implementation at-risk samples.7 Because of the fewer defendants scoring -9,

The table shows overlaps in FTA rates for several scores. It should be noted, however, that for some of the points, this is mainly due to rounding to whole numbers.

-23-

-11, and -12 points, we collapsed -9 or fewer points into one category. For the same reason, we collapsed defendants scoring -4 points with defendants scoring -3 points. In addition, defendants with scores of 1 and 2 points were collapsed. Finally, defendants with a score of 11 points were combined with defendants who scored 12 points. Most of these categories contained less than one percent of the sample each. The table shows that the pre- and post- distributions of defendants classified at each score were quite similar. The same was true for their corresponding FTA rates, with the exception of defendants scoring -5 and -2 points. Defendants scoring those points were observed to have lower FTA rates than projected (21% versus 29%, and 17% versus 22%, respectively). We used a difference of proportions test to see if the differences between projected and observed FTA rates for these two scores were statistically significant (see Appendix A). We found that only at a score of -5 points, the projected and observed FTA rates differed significantly. Since this is the only category where the difference is significant, we expect that it is likely due to random fluctuation. We also examined the predictive power of the point scale by computing its mean cost rating at two time periods (pre- and post-implementation). According to Fischer, an MCR of .35 or greater indicates significant improvement over base rate (Fischer, 1985). The value of MCR of the point scale for the pre-implementation sample was .31, suggesting that the new point scale would improve prediction by about 31 percent of the total possible improvement over the base rate (Table B-1). The MCR of the point scale after its implementation was 33 percent (B-2). This finding suggested that the predictive power of the new point scale did not decrease over time. Finally, the consistency of the new recommendation system was examined by comparing the projected and observed classification of defendants into various risk

-24-

Table 6: Distribution of Point Scale Score by FTA 2001 Sample Total Defendants Defendants with FTA

Post-Implementation Sample Total Defendants Defendants with FTA

N 626 727 499 497 1157

% 2 3 2 2 5

N 225 245 143 129 336

% 36 34 29 26 29

N 408 604 416 357 1135

% 2 3 2 2 6

N 157 196 106 88 240

% 38 32 26 25 21

-3 -2 -1 0

1408 534 400 1017

5 2 2 4

290 120 76 215

21 22 19 21

1229 330 311 709

6 2 2 3

219 56 53 155

18 17 17 22

2 3 4 5 6 7 8 9 10

1665 546 1299 3013 594 4562 1213 1873 2269

7 2 5 12 2 18 5 7 9

360 107 221 381 84 473 124 176 189

22 20 17 13 14 10 10 9 8

1251 452 1012 2909 354 4299 748 1394 1550

6 2 5 14 2 21 4 7 8

242 77 175 338 51 325 59 96 103

19 17 17 12 14 8 8 7 7

12 Total

1379 25278

5 100

77 3971

6 16

829 20297

4 100

47 2783

6 14

Points 1 -9 or < -8 -7 -6 -5

Due to fewer cases, defendants with -10 or fewer points were collapsed with defendants with -9 points, defendants scoring -4 points were collapsed with defendants scoring -3 points, defendants scoring 1 point were collapsed with defendants scoring 2 points and defendants scoring 11 points were collapsed with defendants scoring 12 points. MCR for the 2001 Sample = .311 MCR for the Post-Implementation Sample = .331

-25-

categories and their corresponding FTA rates (Table 7). The projected numbers were obtained by statistically applying the new recommendation system to the 2002 at-risk sample (pre-implementation sample). The observed numbers represent the actual distributions of defendants from the 2003-4 post-implementation sample. As shown by the table, the projected and observed distributions for the categories comprising the new recommendation system were virtually the same, with the exception of a slight shift in the average FTA rate for the “not recommended” category. To be precise, using the pre-implementation sample, we projected that the new system would classify 43 percent of the defendants as “recommended for ROR”, 22 percent as “moderate risk for ROR” and 35 percent as “not recommended for ROR.” Their FTA rates would be eight percent, 14 percent and 24 percent, respectively. The distributions from the post-implementation sample showed that under the new system, 42 percent of the defendants were recommended for ROR. Their FTA rate was 7 percent. Twenty-two percent were categorized as “moderate risk” with an FTA rate of 14 percent. Thirty-six percent were not recommended for ROR. Their FTA rate dropped from the projected 24 percent to 21 percent. The difference in FTA rate was not statistically significant. As projected, the new recommendation system classified defendants on the basis of risk of failure. The above findings suggested that the new point scale was stable across different samples. Its predictive ability did not decrease over time. Furthermore, the classification of defendants into low, moderate and high risks produced the same results as projected.

-26-

-2735

100

8,355

23,811

NOT RECOMMENDED FOR ROR

TOTAL

5,146

MODERATE RISK FOR ROR

10,310

RECOMMENDED FOR ROR

New CJA ROR Recommendation System

3,639

2,051

737

851

21,718

7,904

4,781

9,033

100

Defendants Classified

3,001

1,695

647

659

Defendants with FTA

Observed (Post-Implementation At-Risk Sample)

Defendants Classified

Defendants with FTA

Projected (Pre-Implementation At-Risk Sample)

Table 7: New CJA Recommendation System by FTA for Defendants Released Pretrial (Projected and Observed Distributions)

C. Does the New ROR Recommendation System Perform Better than the Old System? The performance of the new recommendation system was measured in two ways: Did the new system perform better with respect to distinguishing defendants on the basis of risk of flight? Did the new system improve upon the previous system by classifying more defendants as low risk without increasing their FTA rate? To address the first question, we compared the two systems with respect to their FTA rates. Beginning with the old system, the observed FTA rates for the recommended, qualified, and not recommended categories in the 2001 at-risk sample were 10 percent, 12 percent and 22 percent, respectively (Table 8). These figures suggested that the old CJA recommendation system, when applied to the 2001 at-risk sample did not distinguish between low- and moderate-risk defendants. The new system, in comparison, was better able to differentiate low- and moderate-risk defendants from the high-risk defendants (Table 7). Defendants classified as low risk had the lowest FTA rate (7%), the FTA rate for moderate-risk defendants was midway between the FTA rates for low-risk and highrisk defendants (14%), and high-risk defendants had the highest FTA rate (21%). These findings suggested that relative to the previous system, the new recommendation system was better able to categorize defendants with respect to their risk of flight. To address whether the new ROR recommendation system improved over the old system, we examined the proportion of defendants in various risk categories, and their corresponding FTA rates. Both citywide and borough distributions were examined. Table 9 provides the distributions for the old system. These distributions were taken from the pre-implementation at-risk sample. The distributions for the new system were taken from the post-implementation at-risk sample (Table 10). As shown by Table 9, under the old recommendation system, citywide 27 percent of the defendants were recommended for ROR. One-tenth of them failed to appear at least once in Criminal or Supreme Court. In comparison, the new system classified 42 percent of the defendants as â&#x20AC;&#x153;recommended for -28-

-299,355

NOT RECOMMENDED FOR ROR

25,024

9,021

QUALIFIED FOR ROR

TOTAL

6,648

RECOMMENDED FOR ROR

Old CJA ROR Recommendation System

100

Defendants Classified

3,845

2,094

1,106

645

Defendants with FTA

Table 8: Old CJA Recommendation System by FTA (2001 At-Risk Sample)

-30894 13%

6,705 100%

TOTAL

451 20%

2,283 34%

NOT RECOMMENDED (High Risk)

232 11%

211 9%

2,148 32%

2,274 34%

Brooklyn Defendants Defendants Classified with FTA

QUALIFIED (Moderate Risk)

RECOMMENDED (Low Risk)

Old CJA ROR Recommendation System

7,747 100%

3,574 46%

2,794 36%

1,379 18%

1,547 20%

962 27%

405 15%

180 13%

Manhattan Defendants Defendants Classified with FTA

4,721 100%

1,324 28%

1,692 36%

1,705 36%

496 11%

214 16%

164 10%

118 7%

978 100%

383 39%

390 40%

205 21%

153 16%

97 25%

39 10%

17 8%

Borough of Arrest Queens Staten Island Defendants Defendants Defendants Defendants Classified with FTA Classified with FTA

Table 9: Distribution of Old CJA ROR Recommendation System by FTA (Pre-Implementation At-Risk Sample)

4,873 100%

1,791 37%

1,997 41%

1,085 22%

755 16%

370 21%

266 13%

119 11%

Bronx Defendants Defendants Classified with FTA

25,024 100%

9,355 37%

9,021 36%

6,648 27%

3,845 15%

2,094 22%

1,106 12%

645 10%

Total (Citywide) Defendants Defendants Classified with FTA

-31-

668 12%

5,631 1 100%

Percentages do not sum to 100% due to rounding.

TOTAL

405 20%

2,007 36%

NOT RECOMMENDED

108 10%

155 6%

1,060 19%

2,564 46%

Brooklyn Defendants Defendants Classified with FTA

MODERATE RISK FOR ROR

RECOMMENDED FOR ROR

New CJA ROR Recommendation System

6,439 1 100%

2,667 41%

1,638 25%

2,134 33%

1,064 17%

618 23%

256 16%

190 9%

Manhattan Defendants Defendants Classified with FTA

4,117 100%

1,114 27%

891 22%

2,112 51%

447 11%

210 19%

102 11%

135 6%

684 100%

275 40%

107 16%

302 44%

74 11%

43 16%

11 10%

20 7%

Borough of Arrest Queens Staten Island Defendants Defendants Defendants Defendants Classified with FTA Classified with FTA

Table 10: Distribution of New CJA ROR Recommendation System by FTA (Post-Implementation At-Risk Sample)

4,847 100%

1,841 38%

1,085 22%

1,921 40%

748 15%

419 23%

170 16%

159 8%

Bronx Defendants Defendants Classified with FTA

21,718 100%

7,904 36%

4,781 22%

9,033 42%

3,001 14%

1,695 21%

647 14%

659 7%

Total (Citywide) Defendants Defendants Classified with FTA

RORâ&#x20AC;? (Table 10). Their FTA rate declined from 10 percent under the old system to seven percent under the new system. Similar findings were observed for each borough. The largest increase in the proportion of defendants receiving an ROR recommendation was observed for Staten Island where under the new system 44 percent of the defendants were recommended for ROR versus 21 percent under the old system. The FTA rates under the old and new system were eight and seven percent, respectively. The lowest increase was found for Brooklyn where under the new system 46 percent received an ROR recommendation versus 34 percent under the old system. The FTA rate dropped from nine percent under the old system to six percent under the new system. The increases for the Bronx, Manhattan and Queens were 18 percentage points (40% versus 22%), 15 percentage points (33% versus 18%), and 15 percentage points (51% versus 36%), respectively.

The FTA rates for the Bronx, Manhattan and Queens declined by three

percentage points (from 11% to 8%), four percentage points (from 13% to 9%) and one percentage point (from 7% to 6%), respectively. Tables 9 and 10 further show that relative to the old system, the proportion of moderate-risk defendants under the new system decreased for the city as well as for each borough. The citywide decrease was 14 percentage points (22% versus 36%). The borough differences ranged from 24 percentage points for Staten Island (16% versus 40%) to 11 percentage points for Manhattan (25% versus 36%). The decreases for Brooklyn, Queens and the Bronx were 13 percentage points (19% versus 32%), 14 percentage points (22% versus 36%), and 19 percentage points (22% versus 41%), respectively. Under the new system, citywide, the FTA rate for the moderate-risk group increased by 2 percentage points. Changes in each borough were of a similar magnitude, in Brooklyn the FTA rate for the moderate risk group decreased by 1 percentage point (to 10%), Manhattan and Queens it increased by 1 percentage point each (to 16% and 11% respectively), and Staten Island remained the same (10%). The only exception occurred -32-

in the Bronx, where the average FTA rate for the moderate-risk group increased from 13 percent from the old system, to 16 percent under the new system. Finally, citywide the old and new recommendation systems were virtually identical with respect to the proportion of the defendants considered high risk and their corresponding FTA rate. Citywide, under the old system, 37 percent of the at-risk defendants were not recommended for ROR. Their FTA rate was 22 percent. When the new recommendation system was implemented, 36 percent of the defendants were not recommended for release with an FTA rate of 21 percent. Similar differences were observed for each borough, with the exception of the Bronx. In the Bronx, the proportion of high-risk defendants increased from 37 percent under the old system to 38 percent under the new system. The average FTA rate for those defendants increased from 21 percent under the old system to 23 percent under the new system. To summarize, when compared with the previous CJA recommendation system, the new system after its implementation 1) recommended more defendants for ROR, while decreasing their FTA rate slightly; 2) deceased the proportion of defendants in the moderate-risk category with a slight increase in their FTA; and categorized the same proportion as high risk, while slightly decreasing their FTA rate.

D. What is the Impact of the New Recommendation System on Minority Defendants? After developing the new recommendation system we examined its impact on the classification of minority defendants. First, we examined the distributions of different ethnic groups under the old system. Comparisons were then made with the new recommendation system, when applied to the 2001 at-risk sample. We compared Black, White and Hispanic defendants. Due to the small number of cases, we did not include defendants who were categorized as â&#x20AC;&#x153;other.â&#x20AC;?

-33-

Table 11 presents the distribution of ethnicity by FTA under the old recommendation system. Under the old system, 22 percent of White defendants were recommended for ROR, 38 percent received a qualified rating, and 40 percent were not recommended for ROR. The average FTA rates of White defendants in these categories were 8 percent, 9 percent and 18 percent, respectively. In comparison, 26 percent of Black defendants were recommended for ROR, 34 percent were categorized as qualified for ROR, and 40 percent were not recommended for ROR. The average FTA rates of Black defendants in these categories were 12 percent, 14 percent, and 24 percent, respectively. Distributions observed for Hispanic defendants were similar to those for Black defendants. This suggests that under the old recommendation system, Black and Hispanic defendants did not differ from White defendants with respect to the proportion classified as recommended, qualified, or not recommended.

However, their

corresponding FTA rates suggest that the old recommendation system did not effectively classify defendants by their relative risk of flight. For example, the average FTA rate for Black defendants in the recommended category was higher than the average FTA rate for all defendants in that category (12% versus 10%). Despite their low FTA rate, close to two-fifths of the White defendants were considered moderate risk (qualified rating). Table 12 presents the projected distributions from the application of the new recommendation system to the 2001 at-risk sample. The proportion of White defendants classified as low risk increased from 22 percent under the old system, to 47 percent under the new system. Their average FTA rate dropped from 8 percent to 7 percent. The new system also classified a considerably higher proportion of Black defendants as low risk (39% versus 26%), while decreasing the FTA rate of low-risk Black defendants by 2 percentage points. Thus, under the new system, there was an increase in the number of defendants classified low risk, for both Blacks and Whites, but the increase in the percentage of defendants that were classified as low risk was far less for Black defendants than for White defendants. -34-

-351374 3476

TOTAL

1338

764

100

425

245

122

White Defendants Defendants Classified Defendants with FTA N % N %

NOT RECOMMENDED FOR ROR

QUALIFIED FOR ROR

RECOMMENDED FOR ROR

RECOMMENDATION

12343

4934

4228

3181

100

2132

1185

578

369

Black Defendants Defendants Classified Defendants with FTA N % N %

9114

3557

3299

2258

100

1451

796

424

231

Hispanic Defendants Defendants Classified Defendants with FTA N % N %

Table 11: Old Recommendation System: Distribution of Ethnicity by Pretrial FTA for the 2001 At-Risk Sample

-36941 3212

TOTAL

756

1515

100

383

195

109

White Defendants Defendants Classified Defendants with FTA N % N %

NOT RECOMMENDED FOR ROR

MODERATE RISK FOR ROR

RECOMMENDED FOR ROR

RECOMMENDATION

11719

5071

2028

4620

2025

1286

299

440

Black Defendants Defendants Classified Defendants with FTA N % N %

8737

3339

1741

3657

100

1378

781

249

348

Hispanic Defendants Defendants Classified Defendants with FTA N % N %

Table 12: New Recommendation System: Projected Distribution of Ethnicity by Pretrial FTA for the 2001 At-Risk Sample

The new system classified fewer defendants as moderate risk. When applied to the 2001 at-risk sample, this had more of an impact on Blacks than Whites. To be specific, the proportion of Black defendants considered moderate risk decreased from 34 percent under the old system, to 17 percent under the new system. In comparison, the decrease for White defendants was from 38 percent to 24 percent. The table further shows that under the new system, the proportion of White defendants who were not recommended for ROR decreased from 40 percent to 29 percent. In contrast, the proportion of Black defendants not recommended for ROR increased slightly under the new system. Under the new system, White and Hispanic White defendants exhibited similar patterns of distributions of recommendations and FTA rates. The exception to this was defendants categorized as high risk. As mentioned above, under the new system, the proportion of defendants considered high risk decreased considerably. However, the new system had only a negligible effect on the proportion of Hispanic defendants considered high risk. To summarize, the new system classified more defendants as low risk. However, the increase in the proportion of Black defendants considered low risk was lower than the increase for White defendants.

Furthermore, the proportion of high-risk defendants

increased more among Blacks than among Whites. The new system had similar effects with regards to classification of Hispanic defendants, relative to White defendants. The logistic regression model used to develop the new point scale did not control for the effect of ethnicity. Therefore, it could result in misclassification of minority defendants. To illustrate, defendants having a telephone in their residence/cellular phone are less likely to FTA, and so in the new point-scale recommendation system, those with a telephone earn points. Let us assume that the telephone variable was significantly correlated with ethnicity, with Blacks and Hispanics less likely to have a telephone in their residence/cellular phone than White defendants. If the effect of ethnicity is not -37-

directly controlled for in the logistic regression model, Blacks and Hispanics will lose points, due to their relationship with the telephone variable. The same could be said about other community ties variables used in the scale. Additional analysis was conducted to determine whether this is an issue for the new recommendation system (data not shown). The analysis showed that in the 2001 atrisk sample, the proportion of defendants having a telephone in their residence/cellular phone was slightly higher for White defendants (84%), than for Black (76%) and Hispanic defendants (70%). A disparity is also observed for being engaged in a full-time activity. White defendants had slightly higher rates (59%) than Black (52%) and Hispanic (54%) defendants. In contrast, White defendants had the lowest proportion of expecting someone at arraignment (33% for Whites, 38% for Blacks and 42% for Hispanics). There was also slight variation with respect to living in the New York City area, with 90 percent of White defendants, 93 percent of Black defendants, and 95 percent of Hispanic defendants reporting an address in the New York City area. Although these differences were not substantial, the final logistic regression model was re-estimated with ethnicity as an additional control variable (data not shown). All of the variables that were statistically significant in the original model remained significant in the re-estimated model. Ethnicity was also found to be a significant predictor of pretrial FTA. When controlling for the other variables in the model, Black and Hispanic defendants were more likely to FTA than those with the mean effect of the variable. In contrast, White defendants were less likely to FTA than defendants with the mean effect of that variable. Points were assigned to significant variables in the re-estimated model8. These points were identical to those computed for the original model. This finding suggests that the point scale based on the original model would classify defendants without direct bias related to ethnicity. 8

For policy reasons, no points were assigned to the ethnicity variable.

-38-

After the implementation of the new recommendation system, we compared the observed FTA rates of Black and Hispanic defendants with that of White defendants to determine if they were fairly classified on the basis of risk of failure. a. Comparing Black and White Defendants Table 13 presents the distributions of Black and White defendants and their corresponding FTA rates on the new point scale. Due to fewer cases, we have collapsed several categories9. We begin with the low-risk group, which includes defendants scoring 7 through 12 points on the new point scale. The table shows that for all those points, the FTA rates for Black and White defendants were quite similar. The moderate-risk category includes defendants who score 3 through 6 points on the scale. Table 13 shows that with the exception of those who scored 6 points on the scale, Black and White defendants had similar FTA rates. Of those who scored 6 points, Blacks had a higher FTA rate than Whites (14% versus 9%). Under the new recommendation system, defendants who score 2 through -12 points are considered high risk for FTA, and are not recommended for ROR. In the postimplementation at-risk sample, Black and White defendants were observed to have similar FTA rates for the high-risk scores, with the exception of those scoring -7 points. Of defendants who scored -7 points, Black defendants had a higher FTA rate than White defendants (28% versus 23%). The above analysis suggests that Black and White defendants had similar FTA rates at most of the point scale scores, and were fairly categorized into low-, moderateand high-risk categories on the basis of risk of failure.

Defendants scoring -12, -11, -10, or -9 points were collapsed with defendants scoring -8 points, defendants scoring -6 points were collapsed with defendants scoring -5 points, defendants scoring -4, -3, -2, -1 or 0 points were collapsed with defendants scoring 1 point, defendants scoring 5 points were collapsed with defendants scoring 6 points, and defendants scoring 11 points were collapsed with defendants scoring 12 points.

-39-

Table 13: Post-Implementation Distributions of Black and White Defendants by FTA

Point Scale 2 Score -8 or < -7 -5 1 2 3 4 6 7 8 9 10 12 Total 1

Black Defendants Defendants Classified Defendants with FTA N % N % 579 6 189 33 243 3 69 28 829 9 199 24 1399 15 249 18 576 6 129 22 219 2 34 16 387 4 55 14 1333 14 184 14 1645 18 123 8 359 4 32 9 618 7 45 7 674 7 44 6 379 4 16 4 9240

1368

White Defendants Defendants Classified Defendants with FTA N % N % 60 2 20 33 61 2 14 23 125 5 26 21 240 10 47 20 80 3 15 19 67 3 12 18 53 2 9 17 513 21 46 9 676 28 52 8 82 3 5 6 178 7 9 5 197 8 9 5 84 4 3 4 1 100% 2416 267 11

Percentages do not sum to 100% due to rounding.

Due to fewer cases, defendants scoring -9 or fewer points were collapsed with defendants scoring -8 points, defendants scoring -6 points were collapsed with defendants scoring -5 points, defendants scoring -4, -3, -2, -1 or 0 points were collapsed with defendants scoring 1 point, defendants scoring 5 points were collapsed with defendants scoring 6 points, and defendants scoring 11 points were collapsed with defendants scoring 12 points.

-40-

b. Comparing Hispanic and White Defendants Table 14 compares Hispanic and White defendants. Due to insufficient number of defendants, we have collapsed several categories10. As shown by the table, Hispanic defendants did not differ significantly from White defendants with respect to observed FTA rates. This finding applies to almost all point scale scores, suggesting that the new point scale is not biased with respect to Hispanic defendants. c: Classification of Defendants into Risk Categories and Their Relative FTA Rates In addition to examining FTA rates at individual points, we examined the classification of defendants into various risk categories, and their corresponding FTA rates, in the post-implementation sample. Table 15 shows that in the post implementation sample, 48 percent of Whites were considered low risk. In comparison, 38 percent of Blacks and 42 percent of Hispanics were categorized as low risk. Although differing by the proportion classified, the three groups of defendants were observed to have similar FTA rates, 6 percent for Whites, 7 percent for Blacks and 8 percent for Hispanics. In the post-implementation sample, the proportion of defendants classified as moderate risk was slightly lower for Black defendants (20%) than for White or Hispanic (24% each) defendants. Their FTA rates were higher than the FTA rates for the low-risk groups. Finally, Blacks had the highest proportion of defendants not recommended for ROR (42%), followed by Hispanics (34%) and Whites (28%). However, Black, White, and Hispanic defendants not recommended for ROR had similar FTA rates (23% for Blacks, and 21% each for Whites and Hispanics).

Defendants scoring -12, -11, -10, -9, or -8 points were collapsed with defendants scoring -7 points, defendants scoring -6 points were collapsed with defendants scoring -5 points, defendants scoring -4, -3, -2, -1 or 0 points were collapsed with defendants scoring 1 point, defendants scoring 5 points were collapsed with defendants scoring 6 points, and defendants scoring 11 points were collapsed with defendants scoring 12 points.

-41-

Table 14: Post-Implementation Distributions of Hispanic and White Defendants by FTA

Point Scale Score 2 -7 or < -5 1 2 3 4 6 7 8 9 10 12 Total 1

Hispanic Defendants Defendants Classified Defendants with FTA N % N % 450 7 152 34 481 7 96 20 845 12 169 20 512 7 88 17 130 2 25 19 500 7 100 20 1126 16 134 12 1497 21 131 9 264 4 19 7 497 7 36 7 543 8 45 8 296 4 24 8 7141 1019 14 100% 1

White Defendants Defendants Classified Defendants with FTA N % N % 121 5 34 29 125 5 26 21 240 10 47 20 80 3 15 19 67 3 12 18 53 2 9 17 513 21 46 9 676 28 52 8 82 3 5 6 178 7 9 5 197 8 9 5 84 4 3 4 2416 267 11 100% 1

Percentages do not sum to 100% due to rounding.

Due to fewer cases, defendants scoring -8 or fewer points were collapsed with defendants scoring -7 points, defendants scoring -6 points were collapsed with defendants scoring -5 points, defendants scoring -4, -3, -2, -1 or 0 points were collapsed with defendants scoring 1 point, defendants scoring 5 points were collapsed with defendants scoring 6 points, and defendants scoring 11 points were collapsed with defendants scoring 12 points.

-42-

-43-

NOT RECOMMENDED FOR ROR TOTAL

MODERATE RISK FOR ROR

RECOMMENDED FOR ROR

New Recommendation System

28 100

2768

775

676

1317

311

161

White Defendants Defendants Classified Defendants with FTA N % N %

9689

4033

1924

3732

100

1463

913

275

Black Defendants Defendants Classified Defendants with FTA N % N %

7435

2522

1773

3140

100

1061

537

259

265

Hispanic Defendants Defendants Classified Defendants with FTA N % N %

Table 15: New Recommendation System: Observed Distribution of Ethnicity by pretrial FTA for the Post-Implementation Sample

d: Conclusions on the Impact of the New Recommendation System on Minority Defendants To conclude, including ethnicity in the model had no effect on points derived from the original model. This suggests that the new recommendation system based on the original model is free of direct bias related to ethnicity. It classifies defendants on their relative risk of flight. The differences in the proportion of low- and high-risk categories between Black and White defendants, or between Hispanic and White defendants, are mainly due to their risk of flight. The FTA rates observed in the post-implementation sample support this claim.

-44-

V. Summary and Conclusions In June of 2003, based on several years of research at CJA, a new release-onrecognizance recommendation system for adult defendants was implemented in New York City Criminal Courts. Pre- and post-implementation studies were conducted to measure its performance. Those studies showed that the pre- and post-implementation samples were similar with respect to their characteristics, except for slight variations in expecting someone at arraignment, open cases and type and severity of top arrest charge. The two samples were quite similar with respect to their average FTA rates. The preimplementation sample had 15 percent failures while 14 percent failed to appear at least once prior to the disposition of their case in Criminal or Supreme Court in the postimplementation sample. A comparison of the failure rates by point scale scores between the two samples showed no significant differences, with the exception of a score of -5 points. Defendants scoring -5 points were observed to have a significantly lower FTA rate than projected. In the absence of any other significant differences, we attribute this to random fluctuations. Furthermore, when categorized into low, medium and high risks, the pre- and postimplementation samples behaved similarly. These findings suggested that the new recommendation system was valid across different samples. The post-implementation evaluation further revealed that relative to the old system, the new system recommended a considerably higher proportion of defendants for ROR without increasing their FTA rate. The new system also improved prediction, by distinguishing defendants on the basis of risk of failure. Defendants categorized as low risk were observed to have the lowest FTA rate, whereas defendants classified as high risk had the highest FTA rate. The FTA rate for the moderate-risk defendants fell midway between the low and high risk groups.

In comparison, the old CJA

recommendation system, from 1998 and onward, did not accurately distinguish low-risk defendants from moderate-risk defendants. -45-

An analysis of the impact of the new recommendation system on minority defendants revealed that Black and Hispanic defendants had similar FTA rates at most of the points. In the post-implementation sample, defendants categorized as moderate or high risk were observed to have higher FTA rates than defendants classified as low risk This was true regardless of a defendantâ&#x20AC;&#x2122;s ethnicity. This finding suggested that the new recommendation system did not misclassify minority defendants. In light of the above findings, we expect the new system to continue to perform reliably over time. Changes in release/detention policies may lead to changes in the defendant population. Although we expect that minor variations would not affect the performance of the new recommendation system as the range of scores used to aggregate defendants into various risk groups is quite broad (6 through 12 points for the low risk group, 7 through 3 points for the moderate risk group and 2 through -12 points for the high risk group), we suggest that it should be re-evaluated periodically. We expect that under the new system, the classification of defendants on their likelihood of pretrial FTA would assist judges in determining the risk involved with the release of a defendant. To determine whether there has been any shift in judgesâ&#x20AC;&#x2122; release decisions at arraignment, we examined the new recommendation system with respect to judges release decisions in phase I of the post-implementation evaluation (Siddiqi, 2004). The findings from that research indicated that the overall ROR rate under the new system was slightly higher than that under the old system, which may be attributed to random fluctuations in judicial behavior. However, more differences emerged when release decisions were examined across recommendation categories. The new recommendation system was found to be somewhat more consistent with release decisions than the old system. Under the old system, the low- and moderate-risk groups did not differ with respect to release decisions at arraignment. Under the new system, defendants recommended for ROR had the highest ROR rate, which was followed by the moderate-

-46-

risk group. Defendants who were not recommended by CJA had the lowest ROR rate. It is however too early to draw any conclusions from this analysis. Finally, the identification of defendants as moderate and high risk for FTA allows the consideration of other release options, such as supervised- or conditional release, aimed at reducing their risk of FTA. It should be noted that supervised release is not currently an option in New York City.

-47-

REFERENCES Blalock, Hubert M. Jr., (1972). Social Statistics. McGraw-Hill Book Company, Inc.: USA. Curvelier, Steven Jay and Dennis W. Potts (1993). Bail Classification Project, Harris County, Texas. Final Report. State Justice Institute, Virginia. Duncan, O., L. Ohlin, A. Jr. Reiss, and H. Stanton (1953). â&#x20AC;&#x153;Formal Devices for Making Selection Decisions,â&#x20AC;? American Journal of Sociology. 58(6): 574-584. Fischer, D. (1985) Prediction and Incapacitation: Issues and Answers. Statistical Analysis Center, Iowa Office for Planning and Programming: Iowa. Siddiqi, Qudisa (1999). Assessing Risk of Pretrial Failure to Appear in New York City, A Research Summary and Implications for Developing Release Recommendation Schemes: New York. New York City Criminal Justice Agency. Siddiqi, Qudisa (2000). Prediction of Pretrial Failure to Appear and Alternative RiskClassification Schemes in New York City: A Validation Study: New York. New York City Criminal Justice Agency. Siddiqi, Qudisa (2002). Prediction of Pretrial Failure to Appear and an Alternative Pretrial Release Risk-Classification Scheme in New York City: A reassessment Study: New York. New York City Criminal Justice Agency. Siddiqi, Qudisa (2003). An Examination of the Existing and New Pretrial Release Recommendation Schemes in New York City: A pre-Implementation Analysis: New York. New York City Criminal Justice Agency. Siddiqi, Qudisa (2004). An Evaluation of the New Pretrial Release Recommendation System in New York City: Phase I of the Post-Implementation Research: New York. New York City Criminal Justice Agency.

-48-

APPENDIX A Difference of Proportions Test A comparison of the observed FTA rates with projected FTA rates at individual points comprising the new point scale showed that the pre- and post-distributions were quite similar, with the exception of defendants scoring -5 and -2 points. Defendants scoring those points were observed to have lower FTA rates than projected (21% versus 29% at -5 points, and 7% versus 22% at -2 points). We applied a difference of proportions test to determine if those differences were statistically significant.

performed the test of proportions as follows (Cuvelier and Potts, 1993):

f1 â&#x2C6;&#x2019; f 2 â&#x2C6;&#x2019; d

f (1 â&#x2C6;&#x2019; f )(

Where... f=

1 1 + ) n1 n 2

x1 + x2 n1 + n2

f1 and f2 are the proportions of failures (cases with FTA) for samples 1 and 2 d is the difference in the base rates between samples 1 and 2 x1 and x2 are the number of failures for samples 1 and 2 n1 and n2 are the total number of observations for samples 1 and 2 Using the above formula (see Blalock, 1972 for a detailed discussion), we calculated the t values for scores of 6 and -2 as 3.253 and 1.24, respectively. The t value for the score of 6 was significant at .05 level, indicating a less than 5 percent likelihood that the difference between the projected and observed FTA occurred by chance. The value of t for the score of -2 was not significant at .05 level, suggesting that the difference between the projected and observed FTA at that point was due merely to chance.

-49-

APPENDIX B Mean Cost Rating

Mean Cost Rating (MCR) is an instrument-assessment method--a measure of classification efficiency (Cuvelier and Potts, 1993). This statistic indicates the proportion by which an instrument improves prediction over the base rate. The MCR can range from 0 (or 0%) to 1 (or 100 %). A value of 0 indicates that an instrument is totally non-predictive; predictive ability is equal to the base rate. If an instrument perfectly predicts failure/success, it will achieve a value of 1. According to Cuvelier and Potts (1993) and Fischer (1985), an MCR of .25 shows utility for classification, while an MCR of .35 or greater indicates significant improvement over the base rate. The formula used to calculate the MCR is presented below: MCR = ∑ C i k U i – 1 - ∑ U i k C i – 1 i=1

i=1

Where: i = each of the risk levels taken in succession from high to low risk k = the number of risk levels (the total number of point scores) C = the cumulative relative frequency of successes at the ith level U = the cumulative relative frequency of failures at the ith level Table B-1 shows the computation of MCR for the new point scale when applied to the 2001 at-risk sample. In the first column are the individual point scores, followed by the number of cases, the frequency of success (cases with no FTA) and failure (cases with FTA), the proportions of cases with a particular score that failed and did not fail, the cumulative proportion of success and failure, and the calculated MCR success and failure figures. To obtain the values reported in each cell of the MCR success column, each cell in the cumulative success column is multiplied by the cell diagonally above it in the -50-

-51-

Point Score -12 -11 -10 - 9 - 8 - 7 - 6 - 5 - 4 - 3 - 2 - 1 0 1 2 3 4 5 6 7 8 9 10 11 12

Total N 86 2 516 22 727 499 497 1,157 293 1,115 534 400 999 18 1,665 546 1,299 3,013 594 4,562 1,213 1,873 2,269 8 1,371 25,278

Freq Success 53 1 335 12 482 356 368 821 230 888 414 324 787 15 1,305 439 1,078 2,632 510 4,089 1,089 1,697 2,080 8 1,294 21,307

Freq Failure 33 1 181 10 245 143 129 336 63 227 120 76 212 3 360 107 221 381 84 473 124 176 189 0 77 3,971

Prop Success Prop Failure Cum Prop Success Cum Prop Failure MCR Success MCR Failure .002 .008 .002 .008 .000 .000 .000 .000 .003 .009 .000 .000 .016 .046 .018 .054 .000 .000 .001 .003 .019 .057 .001 .001 .023 .062 .041 .118 .002 .002 .017 .036 .058 .154 .007 .006 .017 .032 .075 .187 .012 .011 .039 .085 .114 .271 .021 .020 .011 .016 .125 .287 .034 .033 .042 .057 .166 .344 .048 .043 .019 .030 .186 .375 .064 .062 .015 .019 .201 .394 .075 .073 .037 .053 .238 .447 .094 .090 .001 .001 .239 .448 .107 .107 .061 .091 .300 .539 .134 .129 .021 .027 .321 .566 .173 .170 .051 .056 .371 .621 .210 .199 .124 .096 .495 .717 .307 .266 .024 .021 .519 .738 .372 .365 .192 .119 .711 .857 .525 .445 .051 .031 .762 .889 .653 .631 .080 .044 .841 .933 .748 .711 .098 .048 .939 .981 .876 .825 .000 .000 .939 .933 .921 .876 .061 .019 1.000 1.000 .933 .939 1.000 1.000 8.682 11.928 6.316 6.005 MCR: .311

Table B-1: Mean Cost Rating of FTA by New Point Scale Scores for the 2001 At-Risk Sample

cumulative failure column. In other words, .002 is multiplied by 0, .003 by .008, .018 by .009, and so on. Likewise, the values reported in the MCR failure column are derived by multiplying each cell in the cumulative failure column by the cell diagonally above it in the cumulative success column. In other words, .008 is multiplied by 0, .009 by.002, .054 by .003, and so on. The sum of the values in the MCR failure column is then subtracted from the sum of the values in the MCR success column to obtain the overall MCR statistic (6.316 â&#x20AC;&#x201C; 6.005 = .311). Thus, the MCR is calculated as being .311 (or 31%), demonstrating significant improvement over the base rate, and an ability to successfully differentiate between groups of defendants on the basis of their likelihood of FTA. The MCR for the new point scale after its implementation is calculated to be .334 (Table B-2).

-52-

-53-

Point Score -12 -10 - 9 - 8 - 7 - 6 - 5 - 4 - 3 - 2 - 1 0 1 2 3 4 5 6 7 8 9 10 11 12

Total N 55 334 19 604 416 357 1,135 166 1,063 330 311 690 19 1,251 452 1,012 2,909 354 4,299 748 1,394 1,550 2 827 20,297

Freq Success Freq Failure Prop Success Prop Failure Cum Prop Success Cum Prop Failure MCR Success MCR Failure 26 29 .001 .010 .001 .010 .000 .000 209 125 .012 .045 .013 .055 .000 .000 16 3 .001 .001 .014 .056 .001 .001 408 196 .023 .070 .038 .127 .002 .002 310 106 .018 .038 .055 .165 .007 .006 269 88 .015 .032 .071 .197 .012 .011 895 240 .051 .086 .122 .283 .024 .020 136 30 .008 .011 .130 .294 .037 .036 874 189 .050 .068 .179 .361 .053 .047 274 56 .016 .020 .195 .382 .071 .068 258 53 .015 .019 .210 .401 .080 .078 539 151 .031 .054 .241 .455 .096 .095 15 4 .001 .001 .241 .456 .110 .110 1,009 242 .058 .087 .299 .543 .136 .131 375 77 .021 .028 .320 .571 .174 .171 837 175 .048 .063 .368 .634 .210 .203 2,571 338 .147 .121 .515 .755 .326 .278 303 51 .017 .018 .532 .774 .402 .398 3,974 325 .227 .117 .759 .890 .587 .474 689 59 .039 .021 .799 .912 .711 .692 1,298 96 .074 .034 .873 .946 .796 .756 1,447 103 .083 .037 .955 .983 .904 .858 2 0 .000 .000 .955 .983 .939 .939 780 47 .045 .017 1.000 1.000 .983 .955 17,514 2,783 1.000 1.000 8.887 12.233 6.662 6.330 MCR: .331

Table B-2: Mean Cost Rating of FTA by New Point Scale Scores for the Post-Implementation At-Risk Sample