Juveniles ROR 99 by Administrator

CJA

NEW YORK CITY CRIMINAL JUSTICE AGENCY NEW YORK CITY CRIMINAL USTICE AGENCY

Jerome E. McElroy Executive Director Mary A. Eckert, Ph.D. Associate Director for Research

RELEASE-ON-RECOGNIZANCE RECOMMENDATION SYSTEM FOR JUVENILE OFFENDERS ARRAIGNED IN NEW YORK CITY ADULT COURTS PART I. VALIDATION STUDY

Mary T. Phillips, Ph.D. Project Director

FINAL REPORT

August 1999

52 Duane Street, New York, NY 10007

(646) 213-2500

RELEASE-ON-RECOGNIZANCE RECOMMENDATION SYSTEM FOR JUVENILE OFFENDERS ARRAIGNED IN NEW YORK CITY ADULT COURTS PART I. VALIDATION STUDY

Written by: Mary T. Phillips, Senior Research Analyst Marian Gewirtz, Senior Research Analyst

Systems Programmer: Dale Sealy Programmer/Analyst

Research Project Staff: Raymond Caligiure Senior Research Assistant Alisha Fenton Research Assistant Aaron Kupchick Research Assistant Vanessa Martin Research Assistant Bernice Linen-Reed Administrative Assistant

August 1999

ÂŠ 1999 NYC Criminal Justice Agency, Inc.

ACKNOWLEDGEMENTS This work was partially funded by the Annie E. Casey Foundation as part of New York Cityâ&#x20AC;&#x2122;s participation in the Juvenile Detention Alternatives Initiative (JDAI), which had as a goal the reduction of reliance on secure detention for juveniles.

We are also grateful to the

Department of Juvenile Justice (DJJ) for supplying supplemental data used in this research and to the Office of Court Administration (OCA), whose records we used to check data in the computerized database maintained by the New York City Criminal Justice Agency, Inc. (CJA). The assistance provided by the Annie E. Casey Foundation, DJJ, and OCA does not imply these organizationsâ&#x20AC;&#x2122; agreement with the the methods, analyses, or conclusions of this research. CJA is solely responsible for the contents of this report.

ROR-Recommendation System for Juvenile Offenders, Part I: Validation Study

Table of Contents List of Figures and Tables.................................................................................................................ii Preface............................................................................................................................................... 1 I. Introduction.................................................................................................................................... 3 The New York City Criminal Justice Agency........................................................................... 3 The Need for a Release-On-Recognizance Recommendation System for Juvenile Offenders . 3 How the JO ROR-Recommendation System Works................................................................. 5 II. Description of the Data File ....................................................................................................... 10 Selection of Cases ................................................................................................................... 10 Court-Processing Status of Cases in the Data File .................................................................. 11 Defendants with Multiple Cases.............................................................................................. 12 Creation of the At-Risk Sample .............................................................................................. 14 III. Description of the Research Population .................................................................................... 16 Comparisons Between FY 1992 and 1996-1997 Cohorts ....................................................... 16 Distribution of ROR-Recommendation Categories for 1996-1997 Arrests ............................ 22 IV. How Did the ROR-Recommendation System for Juveniles Affect Release? ......................... 25 Release Status at Arraignment................................................................................................. 25 Effect of the Recommendation System on Release at Arraignment........................................ 27 Effect of the Separate Components of the Recommendation System on ROR at Arraignment ...................................................................................................... 33 Effect of the Recommendation System on Release Prior to Sentence .................................... 35 V. How Well Did the ROR-Recommendation System for Juveniles Predict Failure to Appear? . 38 Interrelationships With Other Factors ..................................................................................... 40 Recommendation, FTA, and Release Type......................................................................... 41 Recommendation, FTA, and Bail Amount ......................................................................... 43 Recommendation, FTA, and Charge Severity .................................................................... 47 Replication of FY 1992 Configuration Analysis..................................................................... 51 Classification Tables ............................................................................................................... 57 Logistic Regression Model...................................................................................................... 61 VI. Discussion................................................................................................................................ 66 The Significance of Release in Criminal Court....................................................................... 66 The Problem of Bias Introduced by Using a Sample of Released Defendants........................ 69 Could the Courts Do Just As Well Without the JO Recommendation System? ..................... 71 Plans for Further Analysis ....................................................................................................... 73 Appendix A: Juvenile Offenses ..................................................................................................... 74 Appendix B: Procedural Manual for Juvenile Offenders............................................................... 75 Appendix C: Use of Tests of Statistical Significance in Cohort Studies....................................... 86 Appendix D: Procedures for Checking Data Errors....................................................................... 89 Appendix E: Calculation and Interpretation of Relative Improvement Over Chance (RIOC) Scores............................................................... 92 Bibliography.................................................................................................................................... 99 i

ROR-Recommendation System for Juvenile Offenders, Part I: Validation Study

List of Figures and Tables Figure 1 Figure 2 Figure 3 Figure 4 Figure 5

Table 1 Table 2 Table 3 Table 4 Table 5 Table 5A Table 6 Table 7 Table 8 Table 9 Table 10 Table 11 Table 12 Table 13 Table 14 Table 15

Juvenile Offender Recommendations: Criteria for Assigning Categories..................... 8 Court-Processing Status of Cases in the Data File as of June 30, 1998........................ 13 Configuration Analysis Results for Juvenile Offenders Arrested in FY 1992.............. 53 Configuration Analysis Results for Juvenile Offenders Arrested in 1996-1997 (14-Month Cohort)........................................................................................................ 55 Likelihood of Release Prior to Sentencing Shown Separately for Criminal Court and Supreme Court Cases ................................ 67

Descriptive Data for Juvenile Offenders: Defendant and Arrest Information ............ 17 Descriptive Data for Juvenile Offenders: Court-Processing Information.................... 18 CJA Recommendation for Juvenile Offenders by Borough ......................................... 23 Release Status at Criminal Court Arraignment by Borough......................................... 26 Release Status at Criminal Court Arraignment by CJA Recommendation .................. 28 Release Status at Criminal Court Arraignment by CJA Recommendation and Borough of Arrest ........................................................ 30 ROR Rates at Criminal Court Arraignment by Criterion Items for the Recommendation System .................................................... 34 Release Outcome Prior to Sentence by CJA Recommendation.................................... 36 Failure to Appear by Recommendation Category and Borough of Arrest.................... 39 Failure to Appear by Release Type and Recommendation Category............................ 42 Bail Amount Posted at First Release by Recommendation Category........................... 44 Range and Average Bail Amount Posted at First Release by Borough of Arrest and Occurrence of FTA.............................................................. 46 Failure to Appear by Bail Amount and Recommendation Category ............................ 48 Failure to Appear by Arraignment Affidavit Charge Severity and Recommendation Category .................................................................................... 49 Classification Tables for FY 1992 and 1996-1997 JO Cohorts .................................... 58 Logistic Regression Analysis Predicting Failure to Appear: Application of the JO Recommendation System to 1996-1997 JO Arrests.................. 62

ROR-Recommendation System for Juvenile Offenders, Part I: Validation Study

List of Figures and Tables Figure 1 Figure 2 Figure 3 Figure 4 Figure 5

Table 1 Table 2 Table 3 Table 4 Table 5 Table 5A Table 6 Table 7 Table 8 Table 9 Table 10 Table 11 Table 12 Table 13 Table 14 Table 15

ROR-Recommendation System for Juvenile Offenders, Part I: Validation Study

PREFACE On April 28, 1996, the New York City Criminal Justice Agency, Inc. (CJA) implemented a release-on-recognizance recommendation system for children under the age of 16 who are brought for prosecution in the adult courts in New York City under the Stateâ&#x20AC;&#x2122;s Juvenile Offender Law.

The recommendation system provides the Criminal Courts1 at arraignment with a

recommendation for release on recognizance (ROR) based on the juvenileâ&#x20AC;&#x2122;s community ties. An earlier report (referred to hereafter as the 1997 Implementation Report) described the development of the recommendation system and presented data on the distribution of recommendation categories and release status at arraignment for all juvenile offenders (JOs) arrested during the first eight months of the systemâ&#x20AC;&#x2122;s operation (Phillips, 1997). The ROR-recommendation system for JOs, like the recommendation system for adults that has been in use in New York City for over 20 years, is based on the premise that it is possible to identify defendants who are likely to fail to appear for scheduled court appearances, and that this can be done using objective criteria developed with rigorous statistical techniques.

recommendation system incorporating these criteria can then be a useful aid to the courts in deciding whether a given defendant can be released on recognizance with a low risk of failure to appear (FTA). The hope was that the JO recommendation system would result in the release on recognizance of more defendants. Even if the release rate did not rise, the recommendation system could help effect better, empirically grounded decisions.

New York has a two-tiered court system. The lower (Criminal) court has jurisdiction over cases in which the most severe charge is a misdemeanor or a charge of lesser severity. The superior (Supreme) court has jurisdiction over cases in which the most severe charge is a felony. All cases are arraigned in Criminal Court, and cases sustained at the felony level must be brought for prosecution into Supreme Court. Only juveniles charged with certain felony offenses are brought into adult court under the Juvenile Offender Law, so if prosecution on the felony charge proceeds, JO cases must be transferred to Supreme Court for disposition.

-1-

ROR-Recommendation System for Juvenile Offenders, Part I: Validation Study

For this research, we augmented the data file used for the 1997 Implementation Study with an additional six months of arrests, through June 30, 1997. We also added to the research file release status, court-processing data, and outcome data for Criminal Court and Supreme Court appearances through final disposition (and sentencing, if there was a conviction), or through June 30, 1998, whichever was earlier. The purpose of the current study is twofold. The first objective is to continue the outcome evaluation of the effect of the JO recommendation system on release rates that was begun in the research described in the 1997 Implementation Report. That report showed that release rates did not rise after implementation of the recommendation system, and in fact they declined. The current study confirms and extends that analysis with additional data. The second objective is to validate the ROR-recommendation system by assessing its success in predicting failure to appear. It was not possible to address this issue in the 1997 Implementation Report because not enough time had passed to allow for the completion of court processing â&#x20AC;&#x201D; and hence, opportunity for FTA â&#x20AC;&#x201D; for a large enough group of cases. The research results for this project are being released in two separate reports. Part I presents statistics describing the cohort of juvenile offenders arrested during 14 months in 1996 and 1997 and validates the ROR recommendation system, addressing both of the research objectives described in the previous paragraph.

Part II will be issued next year. The second

report will explore some related issues, such as the timing of release and failure to appear; length of time it took defendants to return to court after failing to appear; and factors that may be associated with the detention of JOs who were recommended for release.

-2-

ROR-Recommendation System for Juvenile Offenders, Part I: Validation Study

I. INTRODUCTION The New York City Criminal Justice Agency The New York City Criminal Justice Agency, Inc. (CJA) is a not-for-profit corporation that provides a variety of criminal justice services to the City of New York through a contract with the Mayor’s Office of the Criminal Justice Coordinator. These services consist of three principal activities. The first activity, which has a direct relevance to this report, is to interview arrestees held for Criminal Court arraignment in order to provide prosecutors, defense counsel, and judges with background information regarding arrestees’ community ties.

The community-ties

information is used for the release-recommendation system, which is based on the likelihood that individual defendants, if released on their own recognizance, would appear for subsequent court appearances. The second activity is to notify released defendants of future court appearances. The third is to perform research and evaluation regarding issues related to the effective operation of criminal justice processes in New York City. The Need for a Release-On-Recognizance Recommendation System for Juvenile Offenders CJA has been fulfilling its first function — providing a release recommendation based on community ties information — for adults since 1977. Juveniles were not included in this system because it was not until 1978 that New York passed its Juvenile Offender (JO) Law, which created a category of juveniles who are brought for prosecution directly in adult court. Once the JO law was passed and juveniles began to be arraigned in adult court, they were interviewed along with the adults but the applicability of the adult recommendation system to juveniles seemed questionable. Consequently, information on juveniles’ community ties was presented to

-3-

ROR-Recommendation System for Juvenile Offenders, Part I: Validation Study

the court, but with no recommendation about release risk. The JO Law mandated that 14- and 15-year-old juveniles arrested in New York for certain serious felony offenses, and 13-year-olds arrested for second-degree murder, be prosecuted directly in the adult court as juvenile offenders. (The specific juvenile offender felony charges are listed in Appendix A.) Cases may subsequently be “waived down” to juvenile court (Family Court in New York), but the court of original jurisdiction for juvenile offenders in New York is the adult court. This is in contrast to the system used in many states whereby original jurisdiction for juveniles is in the juvenile court, and they may be “waived up” to adult court. In the 1980s CJA first conducted research on the factors that predict failure to appear for juveniles (Metchik, 1987), but it was not until the Annie E. Casey Foundation helped fund new research, completed in 1996, that an ROR-recommendation system for juveniles could be developed (Winterfield, 1996). These studies were summarized in the 1997 Implementation Report (pp. 4-9). Concern about overcrowding in the secure detention facilities for juveniles operated by the Department of Juvenile Justice (DJJ) first prompted the development of the JO recommendation system, and those concerns have not abated. The DJJ secure detention facilities house juvenile delinquents (JDs) — children under 16 whose cases are processed in the Family Courts — as well as JOs. Spofford Juvenile Center, until 1998 the City’s only secure detention facility for youth under age 16 (JOs and JDs), was closed in August 1998 after two smaller replacement facilities in the Bronx and Brooklyn were opened earlier in the year. However, plans have been announced to reopen Spofford before the end of 1999 because of a 31 percent rise in average daily population over the previous year. The average daily population in secure detention reported in December 1998 was 343 juveniles — 38 percent higher than the combined capacity

-4-

ROR-Recommendation System for Juvenile Offenders, Part I: Validation Study

of the new facilities, which have beds for only 248. The number of JOs arraigned each year has decreased sharply during the 1990s,2 and JOs comprise a declining proportion of the juveniles admitted to secure detention: 14 percent in 1997, down from 30 percent in 1994 (NCCD, 1998, Fig. 4-3). Consequently, overcrowding is now attributable more to JDs than to JOs. However, two circumstances inflate the contribution of JOs to the problem of overcrowding beyond what their small numbers would seem to indicate. First, data presented in this report show that JOs are being detained at a much higher rate than a few years ago. Second, JOs tend to stay in detention longer than JDs, and so they take up a disproportionate number of beds: 43 percent on average in 1997 (NCCD, 1998, Fig. 14-4). Consequently, a reduction in the number of detained JOs could help substantially to alleviate overcrowding. How the JO ROR-Recommendation System Works CJA interviewers who conduct all other pre-arraignment interviews also interview juvenile offenders. Interviewers are instructed to follow the standard procedure for conducting any ROR interview, with the differences outlined in the Procedural Manual for Juvenile Offenders, which is appended to this report (Appendix B). The manual explains how to identify JOs and assign an ROR recommendation category to them, using an attachment to the standard interview form. The empirical research completed in 1996 found that the two best predictors of a juvenileâ&#x20AC;&#x2122;s failure to appear are school attendance and whether the juvenile expects someone to come to the Criminal Court arraignment. The two predictor items appear as questions on the JO attachment:

In Fiscal Year 1992 (July 1, 1991 through June 30, 1992) 1,552 JOs were arraigned; in Fiscal Year 1997 slightly more than half that number (792) were arraigned.

-5-

ROR-Recommendation System for Juvenile Offenders, Part I: Validation Study

1. Is defendant in school full or part time? 2. Does defendant expect someone at arraignment? The CJA interviewer assigns a JO recommendation category based on the defendant’s response to these questions and the results of attempts to verify3 school attendance. (Verification is not attempted for the defendant’s claim to expect someone at arraignment.) On the front of the CJA Interview Report is a space for the ROR recommendation. Prior to April 28, 1996, “Juvenile: For Information Only [4E]” was entered in this space on JO reports. Beginning April 28, 1996, interviewers enter on JO reports one of the ROR-recommendation categories shown below:

JUVENILE OFFENDER RECOMMENDATION CATEGORIES [J7] Juvenile Offender: No Recommendation Due To: [J7A] A. Bench Warrant Attached to NYSID [J7B] B. No NYSID Available [J7C] C. Murder Charge (125.25 or 110/125.25) [J7D] D. Interview Incomplete [J6] Juvenile Offender: Not Recommended [J5] Juvenile Offender: Recommended

The categories are listed in descending numerical order (J7 first, J5 last) because this is the order of precedence. A “J5” or “J6” category may be overridden by a policy consideration that excludes arrestees from receiving a recommendation.

The four policy exclusions are:

During the interview the arrestee is asked for the name and telephone number of someone who could verify certain items of information. This contact person is called by CJA staff. If the contact person confirms the information, either “Yes verified” or “No verified” is recorded. If the contact person cannot be reached, the arrestee’s answer is recorded as “Yes” or “No” without verification. If the contact person gives information that conflicts with the arrestee’s statement, and this conflict cannot be resolved by further questions, the response is recorded as an “Unresolved conflict.”

-6-

ROR-Recommendation System for Juvenile Offenders, Part I: Validation Study

outstanding bench warrant (J7A); no NYSID sheet4 (J7B); arrest on a murder charge (J7C); or an incomplete interview (J7D). If there is no policy exclusion, the recommendation is assigned as follows: [J5] RECOMMENDED. Juveniles who claim that they are enrolled in school either full or part time, and for whom this response is verified, are “recommended” for release on recognizance. If school status could not be verified, the juvenile is still recommended if he or she is expecting someone at arraignment. [J6] NOT RECOMMENDED. Juveniles who claim that they are not enrolled in school, and for whom this response is verified, are “not recommended” for release on recognizance. If school status could not be verified, the juvenile is not recommended unless he or she is expecting someone at arraignment. The process by which the recommendation category is assigned can be visualized as branches depending on responses to the two criterion questions. This is illustrated in Figure 1.

New York State Identification (NYSID) sheets are initiated by the defendant’s fingerprints and describe his or her criminal history (also known as “RAP” sheets). After the defendant is fingerprinted by the police, the prints are transmitted electronically to the New York State Division of Criminal Justice Services (DCJS) in Albany. A few hours later DCJS returns a NYSID sheet to Central Booking, and CJA staff use it to transcribe the defendant’s NYSID number and criminal history information onto the CJA Interview Report.

-7-

ROR-Recommendation System for Juvenile Offenders, Part I: Validation Study

New York City Criminal Justice Agency Figure 1 Juvenile Offender Recommendations: Criteria for Assigning Categories

1. Does defendant attend school part- or full-time?

Yes verified J5 Recommended

Yes or No (unverified) or Unresolved conflict [go to next question]

No verified J6 Not Recommended

2. Does defendant expect someone at arraignment? Yes J5 Recommended

No J6 Not Recommended

-8-

ROR-Recommendation System for Juvenile Offenders, Part I: Validation Study

The research done in 1996 showed that juvenile offenders arrested in Fiscal Year 1992 who fit the criteria for “not recommended” were more than twice as likely to fail to appear for a court appearance than those who fit the criteria for “recommended.” The ROR-recommendation scheme was then implemented on the assumption that if it worked well to predict risk of FTA for the FY 1992 cohort, the scheme would also work to predict FTA for more recent cohorts. One task of the current research was to find out if this assumption could be validated for the group of JOs arrested in 1996 and 1997. The results of the validation analysis are presented in section V, following a description of the data file (section II), some descriptive statistics for the juvenile offenders in the study (section III), and the findings concerning the effects of the ROR-recommendation system on release rates (section IV).

-9-

ROR-Recommendation System for Juvenile Offenders, Part I: Validation Study

II. DESCRIPTION OF THE DATA FILE Selection of Cases In order to provide criminal justice information and research services, CJA maintains an arrest-based, automated database containing detailed information on all New York City arrests, court appearances, and case dispositions in Criminal Court and Supreme Court. From this database we selected all juvenile offenders (JOs) arrested on April 28, 1996, through June 30, 1997. The starting date was selected to be the first day of the implementation of the new JO recommendation system. The closing date was selected to provide a data set that included all of Fiscal Year 1997, so that the results could be more easily compared to other Fiscal Year data (particularly the FY 1992 data set used to develop the JO recommendation system).

estimated that the number of arrests occurring within this time period would be sufficient for the analysis.5 An arrest cutoff date of June 30, 1997, would also allow at least a full year for all cases to be completed by June 30, 1998. Arrests within the specified dates were identified as JOs using the following criteria: ·

age is 14 or 15 at the time of arrest and the top arrest charge is a JO charge (see Appendix A for JO charges); or

age is 13 and the top arrest charge is second-degree murder; or

the CJA recommendation category is J5, J6, or J7 (or 4E, the category used for juveniles prior to implementation of the recommendation system).

The estimate was done by means of a statistical technique known as “power analysis.” The power analysis showed that 485 cases would be the minimum needed to detect a small effect (R2 =.05) at a significance level of .05 and a power of .99 (meaning that there is a 99% chance that a statistically significant effect this small would be detected by the analysis), assuming that up to four independent variables were entered in the multivariate analysis. The number of cases in the final analytic sample was 705, which is well over the minimum requirement as estimated by the power analysis. (The analytic sample included only cases in which the defendant had been released; the number of cases in the complete data set was larger.) For a discussion of the use of statistical tests of significance for data sets that are not random samples, see Appendix C.

- 10 -

ROR-Recommendation System for Juvenile Offenders, Part I: Validation Study

To the cases selected using these criteria were added three April 27, 1996, arrests because the juveniles were interviewed by CJA staff on April 28 and were assigned a JO recommendation category according to the new procedures. We dropped from the file all cases that were declined prosecution or were transferred to Family Court prior to arraignment, keeping only cases with a Criminal Court arraignment. The resulting file contained 1,065 cases. Of these, 77 were deleted because, after checking all cases with conflicting data regarding eligibility for juvenile offender status, we decided that the defendants in these cases were probably not juvenile offenders. In addition, extensive efforts were made to check the data for errors in court-processing information, especially release status and outcomes of court appearances. (See Appendix D for details concerning data-checking procedures and decisions regarding cases with conflicting data.) Since the data file contained every arraigned JO arrested within the time frame of the validation study, it is not a statistical sample, which would consist of some representational portion of all the cases. Instead, the cases comprise an entire 14-month cohort. In this respect the validation cohort is comparable to the Fiscal Year 1992 cohort used to develop the JO recommendation system. Court-Processing Status of Cases in the Data File The final file contained 988 docketed cases from arrests that had occurred during the 14 months from the end of April 1996 through June 30, 1997.6 We collected court-processing data

A few cases were missed because both date of birth and CJA recommendation were blank in the database. We found 3 such cases while checking rearrests and other data for several hundred juvenile offenders in the data file; for those cases, we knew that the defendant was a JO because of the sample case. There is no way of knowing how many other JOs we might have missed because date of birth and CJA recommendation were blank, but this is a rare occurrence in the database. In any event, a JO without a CJA recommendation could not be used in the validation analysis (although such cases might alter the overall FTA rate).

- 11 -

ROR-Recommendation System for Juvenile Offenders, Part I: Validation Study

through June 30, 1998, allowing at least a full year for all arrests to be processed by the courts and for the collection of FTA data. By that time 94 percent of the cases in the data file had been completed, either by transfer to Family Court (28%), by a favorable outcome in Criminal Court (29%) or in Supreme Court (2%), or by the defendantâ&#x20AC;&#x2122;s conviction and sentencing in Supreme Court (35%). A handful of cases were still working their way through the courts (3%), a few were still open after the defendant failed to appear and did not return prior to disposition or sentencing (1%), and a few were missing data on case outcome (1%). Figure 2 illustrates in more detail the court-processing status of the cases in the data file. Defendants With Multiple Cases Since this is an arrest-based file, defendants with more than one arrest were counted as many times as they were arrested within the time range of the study. The number of defendants with more than one arrest during the study period was small: there were 950 unique defendants in the sample, of whom fewer than four percent had another case in the data file.7 Counting a defendant multiple times could bias the results if it occurred frequently, but this number was too small to have much of an effect. Treating multiple arrests as though they represented different defendants can also be justified on the grounds that the same defendantâ&#x20AC;&#x2122;s circumstances (such as school attendance) could change from one arrest to the next, resulting in a different CJA recommendation and thus, a different prediction regarding the defendantâ&#x20AC;&#x2122;s likelihood of FTA.

914 (96 %) had only one case; 34 had two cases, and two had three cases. The number of defendants in the data file who were rearrested during the study period was actually somewhat larger than this because some were rearrested as adults or as juvenile delinquents. This rearrest count includes only JO cases that were arraigned. Not counted are rearrests as an adult or JD, rearrests that were not prosecuted, and rearrests that were not collected in the data file because date of birth and CJA recommendation were both missing in the database (see footnote 6). When we checked rearrests for detention status in order to verify failure to appear, however, we checked all rearrests, not just those included in the data file.

- 12 -

ROR-Recommendation System for Juvenile Offenders, Part I: Validation Study

New York City Criminal Justice Agency Figure 2 Court-Processing Status of Cases in the Data File as of June 30, 1998 N=988 JO arrests 4/28/96 through 6/30/97 that resulted in an arraignment in Criminal Court

N= 267 (27%) Transferred to Family Court from Criminal Court

N=289 (29%) Case ended in Criminal Court

N=286 (29%) Disposed in Criminal Court Dismissed = 227 Dismissed (not sealed) = 57 Adjourned in contemplation of dismissal = 2

N= 12(1%) Transferred to Family Court from Supreme Court

Convicted and sentenced Dismissed Dismissed (not sealed) Acquitted Committed to Commissioner of Mental Health

= 343 = 17 = 3 = 2 =

N=3 (<1%) Defendant failed to appear prior to disposition and did not return

N=16 (2%) Did not reach final disposition in Supreme Court

N=398 (40%) Reached final disposition in Supreme Court

N=366 (37%) Case concluded

N=21 (2%) Defendant convicted, awaiting sentencing

N=432 (44%) Transferred to Supreme Court

N=8 (<1%) Defendant convicted, failed to appear for sentencing and did not return

N=6 (<1%) Supreme Court data missing

N=3 (<1%) Defendant convicted, sentencing data missing

Summary: Status of cases as of 6/30/98 Transferred to Family Court prior to disposition 279 (28%) Court processing completed in CC or SC 652 (66%) Favorable outcome in CC 286 (29%) Favorable outcome in SC 22 (2%) Conviction & sentence in SC 343 (35%) Case still open 34 (3%) Convicted, awaiting sentence 21 (2%) Not yet disposed 13 (1%) Defendant failed to appear prior to completion of case and did not return 14 (1%) Missing information on case outcome 9 (1%) Total: 988

-13-

N=13 (1%) Disposition pending (adjourned to a date later than 6/30/98)

N=3 (<1%) Defendant failed to appear prior to disposition and did not return

ROR-Recommendation System for Juvenile Offenders, Part I: Validation Study

Validating these differing CJA recommendations, based on different circumstances, requires treating each case uniquely.8 Consequently, in the remainder of this report we may occasionally refer to “juveniles” or “defendants” as though the person is the unit of analysis, but the reader should keep in mind that the unit of analysis is the arrest (case). Creation of the At-Risk Sample As previously noted, one objective of this study is to validate the premise that JOs who were “not recommended” for release on recognizance would fail to appear more often than those who were “recommended.” In order for the comparison of FTA rates to be meaningful, both groups (“not recommended” and “recommended”) must have had an opportunity to fail to appear. If the defendant was in custody during the entire processing of the case, he or she would have had no opportunity to fail to appear. Including such defendants in a category of “never failed to appear” would be misleading. If given a chance, they might have failed. The fact that they were not released may indicate that the judge considered the risk of FTA to be high enough to justify detention. Therefore, we needed to create an “at-risk sample” to use for the FTA analysis. This sample consisted of only those cases in which the defendant was at risk for failure to appear for at least one appearance during the processing of the case at any time from Criminal Court arraignment through sentencing in Supreme Court.9 We did not include any court dates after sentencing.

If there was no conviction, the final court date used in the analysis was the

Of the 36 defendants who were represented by more than one case in the file, almost half (16) did not receive the same CJA recommendation in all their cases. 9 We counted time at risk to sentencing rather than to final disposition in order to replicate the analysis of the FY 1992 arrests used to develop the JO recommendation system. Analyses of FTA often do not include court appearances following final disposition because factors affecting FTA may change for convicted defendants. Moreover, the CJA ROR-recommendation is meant to apply only to pre-trial release. Part II of this report will present an analysis of FTA to final disposition.

- 14 -

ROR-Recommendation System for Juvenile Offenders, Part I: Validation Study

appearance on which there was a favorable disposition or a transfer to Family Court; in the absence of either of those outcomes, the latest court appearance through June 30, 1998, was the final date used. We defined “at risk” in the following way: Defendants who were released (either on their own recognizance or on bail) at any given court appearance were considered at risk for the next appearance. A defendant was considered at risk even in the absence of a release at any court appearance if supplemental data from DJJ indicated that he or she posted bail prior to a court date included in the study. A defendant need be at risk for only one court appearance through sentencing in order to be included in the at-risk analytic sample. If the earliest release occurred at the latest court appearance in the study, we did not consider the defendant at risk for purposes of the research because the next scheduled appearance — the one for which he or she was at risk — was not included in the study. Also, a defendant who was released at a court appearance but was rearrested and was still in custody at the next court date was not at risk for that appearance. (The case would still be included in the at-risk sample if the defendant were at risk for another court date.) The at-risk sample includes 705 cases, or 72 percent of all cases with more than one appearance date. There were 13 cases that were dismissed or transferred to Family Court at Criminal Court arraignment; the defendants in those cases had no opportunity to be at risk for a following adult court appearance.

No Criminal Court information was available for one

additional case. Defendants in the remaining 269 cases were never at risk as defined above. Whether or not a defendant was released is of interest, of course, not only in defining the atrisk sample, but as an important analytic focus in its own right. Release rates and the effect of the ROR recommendation on release are the topic of section IV.

- 15 -

ROR-Recommendation System for Juvenile Offenders, Part I: Validation Study

III. DESCRIPTION OF THE RESEARCH POPULATION Comparisons Between FY 1992 and 1996-1997 Cohorts Tables 1 and 2 present descriptive data for the JOs in this research along with comparable information describing the juveniles in the FY 1992 data set that was used in the research to develop the recommendation system. The FY 1992 data include 12 months of arrests. The 1996-1997 data include 14 months of arrests. The middle column in both tables contains data for the 12 months of arrests from July 1, 1996, through June 30, 1997 (FY 1997), which comprise a subset of the 1996-1997 data set. The FY 1997 data are presented to provide annual numbers that are more directly comparable to FY 1992. Although the FY 1992 and the FY 1997 data both cover 12 months of arrests, there were some differences between them in data collection procedures. These differences arose from the fact that the earlier research used a pre-existing data set that had been compiled for a broader purpose, whereas the 1996-1997 data set was compiled specifically for this research. First, the FY 1992 arrests were followed through nine months of court appearances (or until the end of the case, whichever came first), whereas the FY 1997 arrests were followed for at least 12 months (or until the end of the case, whichever came first). Second, at-risk appearances were not calculated in precisely the same way. For the FY 1992 analyses, release status was calculated using only the four milestone appearances available in the data set, whereas for the FY 1997 analyses release status at every court appearance was used in the calculation. Finally, much more extensive data checking was done for the FY 1997 arrests in order to verify the JO status of every case in the file and to check release statuses and outcomes. These differences in data-collection and checking procedures may account for a small portion of the variations in defendant and case characteristics found between the two cohorts.

- 16 -

ROR-Recommendation System for Juvenile Offenders, Part I: Validation Study

New York City Criminal Justice Agency Table 1 Descriptive Data for Juvenile Offenders: Defendant and Arrest Information Defendant and Arrest Information Age · 13 years old · 14 years old · 15 years old Gender · Male · Female Ethnicity · Black · Hispanic · White · Other Borough of arrest · Brooklyn · Manhattan · Queens · Staten Island · Bronx Criminal record (categories are not mutually exclusive) · First arrest ·

No prior felony conviction

No open cases

FY 19921 N = 1552

FY 19972 N = 792

1996-1997 (14 mo.)3 N = 988

11 <1% 556 37% 954 63% n = 1521

1 <1% 276 35% 514 65% n = 791

1 349 637

<1% 35% 65% n = 987

1375 89% 165 11% n = 1540

697 88% 91 12% n = 788

866 118

88% 12% n = 984

1009 451 55 37

65% 29% 4% 2%

510 65% 210 27% 34 4% 30 4% n = 784

628 274 39 39

64% 28% 4% 4% n = 980

591 364 293 18 286

38% 23% 19% 1% 18%

306 142 139 13 192

385 192 161 16 234

39% 19% 16% 2% 24%

Not available 1437 99% n = 1451 1173 81% n = 1451

555

39% 18% 18% 2% 24%

73% n = 762 741 >99% n = 742 634 85% n = 742

681

73% n = 935 908 >99% n = 909 779 86% n = 909

The base number used in some percentages is less than the total number in the sample because cases with missing information were not included. The base number for each cell is given if it is less than the sample N. Percentages may not sum to exactly 100% because of rounding.

- 17 -

ROR-Recommendation System for Juvenile Offenders, Part I: Validation Study

New York City Criminal Justice Agency Table 2 Descriptive Data for Juvenile Offenders: Court-Processing Information Court Processing Information Affidavit Charge · 1st degree robbery · 2nd degree robbery · 2nd degree murder (including attempted murder) · All other JO charges Release status at Criminal Court arraignment · ROR · Bail set and made · Bail set and not made · Remand Bail amount set at Criminal Court arraignment · Mean effective bail · Median effective bail · Minimum effective bail · Maximum effective bail · Standard deviation Criminal Court disposition · Transfer to Supreme Court · Transfer to Family Court · Dismissal

FY 19921 N = 1552

FY 19972 N = 792

847 56% 383 26% 115 8% 155 10% n = 1500

444 58% 204 27% 36 5% 85 11% n = 769

548 261 45 108

57% 27% 5% 11% n = 962

795 52% 25 2% 665 43% 57 4% n = 1542

300 38% 20 3% 441 56% 20 3% n = 781

370 29 554 23

38% 3% 57% 2% n = 976

$63244 $1500 $100 $250,000 18,857 n = 681

$8598 $3500 $35 $250,000 20,646 n = 461

1996-1997 (14 mo.)3 N = 988

$7799 $3500 $35 $250,000 18,545 n = 583

714 47% 415 27% 389 26% n = 1518

343 43% 218 28% 229 29% n = 790

432 267 284

44% 27% 29% n = 983

Supreme Court disposition · Conviction · Transfer to Family Court · Dismissal/acquittal

536 24 38

90% 4% 6%

299 92% 10 3% 17 5% n = 326

375 12 22

92% 3% 5% n = 409

Supreme Court sentence · Imprisonment only · Split sentence (imprisonment and probation) · Probation · Conditional or unconditional discharge

192 41 239 --

41% 9% 51% --

126 52 160 4

n = 472

105 39% 37 14% 126 46% 4 1% n = 272

37% 15% 47% 1% n = 342

1196 77% n = 1552

559 71% n = 792

705

71% n = 988

101 8% n = 1196 (at-risk only)

56 10% n = 559 (at-risk only)

Ever released in Criminal or Supreme Court5 · Yes, released prior to at least one appearance before sentencing Ever failed to appear in Criminal or Supreme Court · Yes, warrant ordered at least once before sentencing

n = 598

75 11% n = 705 (at-risk only)

Winterfield, Laura A., Developing a Release-On-Recognizance System for Juveniles Arraigned in New York City Adult Courts. CJA, May 1996. Some unpublished data from the study are included here. 2 Includes juvenile offenders arrested from the first day of Fiscal Year 1997, July 1,1996, through June 30, 1997. 3 Includes juvenile offenders arrested from April 28, 1996 (the date that JO recommendations were implemented by CJA) through June 30, 1997. Three juveniles arrested on April 27 were included because they were interviewed by CJA on April 28 and were processed using the new procedures. Only docketed cases were included in the 1996-1997 and the FY 1992 research files. 4 Mean bail amount is slightly higher than reported in Winterfield (1996) because we reanalyzed her data to exclude 4 cases with a bail amount of $1 (which may mean that there was a higher bail set on another docket). There were no $1 bail amounts in the 1996-1997 data. 5 At-risk appearances were not calculated in precisely the same way for the FY 1992 analysis as for the 1996-1997 analyses. For the FY 1992 analysis, release status was calculated using only four milestone appearances, whereas for the current research release status at every court appearance was used in the calculation. The results should be roughly comparable, nonetheless.

- 18 -

ROR-Recommendation System for Juvenile Offenders, Part I: Validation Study

Table 1 presents information on the defendant and the arrest. A JO arrested in FY 1997 (middle column) was most likely to be 15 years old (65%), male (88%), black (65%), with no prior felony convictions (almost 100%) or open cases (85%). This profile is very similar to the profile of the full 14-month 1996-1997 research population and to the profile of FY 1992 JOs. In each group, the JOs were more likely to have been arrested in Brooklyn than in any other borough (38% or 39%). Table 2 compares the cohorts in terms of court-processing variables.

In most respects,

strong similarities persist throughout the court-related data between the two Fiscal Years, and between FY 1997 and the full 14-month 1996-1997 research population. Most defendants in all three groups were charged with robbery (82% to 85%), in about the same proportions of firstdegree and second-degree robbery. Skipping over release status at arraignment and bail amount for the moment, we find in the next row that cases in both cohorts showed very similar rates of transfer to the Supreme Court from the Criminal Court (43% in FY 1997, compared to 47% in FY 1992) and of transfer to the Family Court (28% in FY 1997, compared to 27% in FY 1992). In both cohorts, cases prosecuted in the Supreme Court were most likely to result in conviction (92% in FY 1997 and 90% in FY 1992). The percentages for the full 14-month 1996-1997 research population differed by only one or two percentage points from the FY 1997 figures for each of these variables. The types of sentences the convicted juveniles received in Supreme Court were also very similar across cohorts. Probation was the most frequent sentence in both research populations, accounting for 46 percent of sentences in the FY 1997 group and 51 percent in FY 1992. In both groups about half of the convicted JOs were sentenced to incarceration, although the FY 1997 group received more sentences of prison and probation (â&#x20AC;&#x153;splitâ&#x20AC;? sentences: 14% in FY 1997

- 19 -

ROR-Recommendation System for Juvenile Offenders, Part I: Validation Study

compared to 9% in FY 1992) and fewer sentences of imprisonment alone (39% in FY 1997 compared to 41% in FY 1992). Again, data for the full 14-month 1996-1997 group were very similar to data for FY 1997. Finally, FTA rates were compared. The issuance of a non-stayed bench warrant was used as the indicator of a failure to appear, and only the first warrant per case was counted (see Appendix D). FTA rates were similar for the two Fiscal Year cohorts: 8 percent of at-risk JOs in FY 1992 and 10 percent in FY 1997 failed to appear at least once prior to sentencing.10

The rate for the

full 14-month at-risk cohort was 11 percent.11 The similarities between the two cohorts end when we move to a consideration of release rates and bail amounts. A hope for the JO recommendation system was that its implementation would result in higher rates of release on recognizance, but Table 2 shows that instead of rising after 1992, ROR rates actually declined. Over half (52%) of the JOs in FY 1992 were released on recognizance at Criminal Court arraignment, compared to only 38 percent in FY 1997. This decline in ROR rates was accompanied by a rise in bail-setting, while the small proportion of JOs remanded without bail remained under five percent (4% in FY 1992 and 3% in FY 1997). Bail amounts rose, too: the mean (average) effective bail amount set in FY 1997 was over $2,000 higher than in FY 1992. (The â&#x20AC;&#x153;effective bailâ&#x20AC;? is the amount the defendant would need to post to obtain release, which would be the lower of the bond amount or the cash alternative, if there is one.) The median bail amount more than doubled during this time period (the median is the

This is a case-based FTA rate obtained by dividing the number of cases with a defendant who failed to appear one or more times (up to the sentencing appearance) by the total number of cases. Thus more than one failure to appear during the processing of the same case was ignored. The results are nearly the same as a defendant-based FTA rate because very few defendants had more than one case included in the study. 11 The rates given in the text include post-disposition failures. When these are excluded, the 1996-1997 FTA rate to final disposition was 8% of the at-risk defendants.

- 20 -

ROR-Recommendation System for Juvenile Offenders, Part I: Validation Study

midpoint: half were greater and half less than this amount). About the same proportion made bail at arraignment in FY 1992 and in FY 1997 (2% and 3% respectively), but with a larger percent having bail set, the result was a large increase in the proportion of the total cohort held on bail leaving arraignment. Table 2 also compares “ever released” rates for the two cohorts (second row from the bottom). This measure combines release on recognizance and on bail, and was computed using the procedures described under “Creation of the At-Risk Sample.” There was a drop of six percentage points from FY 1992 (77% ever released prior to sentencing, or prior to final disposition for nonconvictions) to FY 1997 (71%). This decline is much smaller than the decline in the percent released at arraignment, which implies that for the later cohort, there was a somewhat higher rate of post-arraignment releases. This also means that a larger proportion of the FY 1997 at-risk sample had spent some time in detention after arraignment (43%, compared to 31% for FY 1992). Data on both the 12-month FY 1997 arrests and the 14-month 1996-1997 arrests have been presented thus far because of concerns that the inclusion of the two additional months — May and June, 1996 — might affect the characteristics of juvenile arrests in the research (perhaps in ways related to seasonality) and thereby bias the findings. Instead, it appears that the data from FY 1997 are quite similar to the data for the 14-month period of 1996-1997. In light of the markedly reduced volume of juvenile arrests and the far lower rates of release for the recent research period when compared to the FY 1992 cohort, it is important to use the larger pool of arrests in an effort to maximize the number of juveniles screened by the new recommendation scheme. The larger volume is also important to the assessment of failure to appear, the outcome criterion for validation of the ROR-recommendation system. For these reasons, results reported

- 21 -

ROR-Recommendation System for Juvenile Offenders, Part I: Validation Study

from this point on will be limited to the entire 14-month cohort of 1996-1997 arrests. Distribution of ROR-Recommendation Categories for 1996-1997 Arrests Based on the FY 1992 cohort, an estimate was made that 70 percent of JOs who were arraigned citywide would be recommended for release, with borough variations ranging from 62 percent in Manhattan to 77 percent in Queens. Table 3 shows that these estimates were low. Citywide, about 79 percent of 1996-1997 JOs were recommended for release, ranging from a low of 71 percent in Manhattan to 86 percent in Queens. (In Staten Island, 94 percent were recommended, but there were too few cases to make a meaningful comparison.) 12 However, this finding was not mirrored by a decrease of the same magnitude in “not recommended” JOs. In FY 1992, 11 percent would have been not recommended — very close to the 10 percent found in 1996-1997. Instead, the policy exclusion categories (warrant attached, no NYSID, murder charge, incomplete interview) decreased from 19 percent to 4 percent, to yield the increase in the recommended category. In a small number of cases (6) the CJA interviewer treated the defendant as a JO, but did not use the new procedure to assign a recommendation. Instead, the old 4E category (“Juvenile: For Information Only ”) was assigned. This happened occasionally while interviewers were making the transition from the old to the new system; there are no 4E categories in the research file for 1997 arrests.13

Data presented later in this report (Table 5) show that when cases with missing recommendation or release status were omitted from the base, 85% of JOs were recommended, and 11% were not recommended. 13 A larger number of cases (22) were assigned a 4E category according to the 1997 Implementation Report, even though that research was based on a subset of the cases used for this report. The explanation is that more thorough data cleaning for the current research revealed that most of those 4E categories were not recorded on the Interview Form, but had been entered erroneously into the database even though a valid JO category was on the Interview Form. The current research file contains corrected data.

- 22 -

ROR-Recommendation System for Juvenile Offenders, Part I: Validation Study

New York City Criminal Justice Agency Table 3 CJA Recommendation For Juvenile Offenders By Borough 1996-1997 (14 Months)1 Recommendation Category Recommended (J5) Not recommended (J6) No recommendation: Bench Warrant (J7A) No recommendation: No NYSID (J7B) No recommendation: Murder Charge (J7C) No recommendation: Incomplete Interview (J7D) Juvenile: for information only (4E: category used prior to implementation; no recommendation) Adult recommendation category assigned Missing recommendation Total

Brooklyn

Manhattan

Queens

320 (83.1%) 27 (7.0%) 2 (0.5%)

136 (70.8%) 18 (9.4%) 1 (0.5%)

139 (86.3%) 14 (8.7%) —

1 (0.3%)

2 (1.0%)

—

3 (0.3%)

17 (4.4)

4 (2.1%)

4 (2.5%)

—

2 (0.9%)

27 (2.7%)

—

1 (0.5%)

—

1 (0.1%)

2 (0.9%)

6 (0.6%)

15 (6.4%) 3 (1.3%) 234 (100%)

40 (4.0%) 26 (2.6%) 988 (100%)

2 (0.5%)

Staten Island 15 (93.8%) 1 (6.3%) —

Bronx

Citywide

172 (73.5%) 39 (16.7%) 1 (0.4%)

782 (79.1%) 99 (10.0%) 4 (0.4%)

(1.0%)

5 (1.3%) 11 (2.9%) 385 (100%)

19 (9.9%) 9 (4.7%) 192 (100%)

1 (0.6%) 3 (1.9%) 161 (100%)

0 16 (100%)

Percentages may not sum to exactly 100% because of rounding.

1 Includes juvenile offenders arrested from April 28, 1996 (the date that JO recommendations were implemented by CJA) through June 30,1997. Three juveniles arrested on April 27 were included because they were interviewed by CJA on April 28 and were processed using the new procedures. Only docketed cases were included.

- 23 -

ROR-Recommendation System for Juvenile Offenders, Part I: Validation Study

A few juveniles (40, or 4%) were treated by CJA interviewers as adults. There are several ways in which this could have happened: the defendant may have not have been identified by the NYPD as a JO; the defendant may have claimed during the interview to be 16 or older (although if identified by the NYPD as a JO, he or she should still have been interviewed as a JO); or the CJA interviewer may simply have erred. Finally, some juveniles (26, or 3%) were missing a recommendation. A few were probably data entry errors, but in general most missing recommendations occur because a defendant is not interviewed. This can happen because JOs are held in separate detention cells and may not come to the attention of the CJA interviewer, or because the defendant is rushed directly to the arraignment court without time for an interview. The next two sections present our findings regarding the central questions of this study. Section IV analyzes the effect of the ROR recommendation on release rates, and section V addresses the relationship between the recommendation and FTA rates.

- 24 -

ROR-Recommendation System for Juvenile Offenders, Part I: Validation Study

IV. HOW DID THE ROR-RECOMMENDATION SYSTEM FOR JUVENILES AFFECT RELEASE? Release Status at Arraignment Table 4 presents data on release status at arraignment in Criminal Court. Citywide release rates have already been reported in Table 2; Table 4 presents the results separately for each borough.

The top part of the table categorizes the data by release status (released on

recognizance, bail set, or remanded), and the lower part reorganizes the same data in order to distinguish released (released on recognizance and on bail) from detained (held on bail and remanded) defendants. Citywide, about 38 percent were released on recognizance at arraignment. Most of the remainder had bail set, and 2 percent were remanded. The Bronx differed markedly from the other boroughs in that it was the only borough in which nearly half (49%) were released on recognizance. (In Staten Island 50% were released on recognizance, but again the number of cases was too small for the percentage to be stable.) This is not what one would expect from the distribution of positive recommendations: the Bronx was near the bottom in the percentage of juveniles recommended for release (74%, as shown in Table 3). Likewise, Queens had the lowest release-on-recognizance (ROR) rate at arraignment (33%), yet this borough had the highest proportion recommended for release (86%). Five cases, three of which were transferred to Family Court at arraignment, were missing the release status. An additional seven defendants did not have a release status because their cases were disposed at arraignment; all seven cases were dismissed. Very few of those with bail set were able to make bail, so the total proportion of JOs who were released at arraignment (41% citywide) is not much larger than the proportion released on

- 25 -

ROR-Recommendation System for Juvenile Offenders, Part I: Validation Study

New York City Criminal Justice Agency Table 4 Release Status at Criminal Court Arraignment By Borough 1996-1997 (14 months)1 Release Status At Arraignment Released on recognizance Bail set Remanded Total Release Outcome At Arraignment RELEASED Released on recognizance Bail made DETAINED Bail not made Remanded Total Missing release status Disposed at arraignment Total cases

Brooklyn

Manhattan

Queens

130 34.2% 243 63.9% 7 1.8% 380 100%

67 35.6% 115 61.2% 6 3.2% 188 100%

52 32.5% 106 66.3% 2 1.3% 160 100%

142 37.4% 130 34.2% 12 3.2% 238 62.6% 231 60.8% 7 1.8% 380 100%

71 37.8% 67 35.6% 4 2.1% 117 62.2% 111 59.0% 6 3.2% 188 100%

62 38.8% 52 32.5% 10 6.3% 98 61.3% 96 60.0% 2 1.3% 160 100%

3 2

1 3

385

192

Staten Island

Bronx

Citywide

113 48.7% 111 47.8% 8 3.4% 232 100%

370 37.9% 583 59.7% 23 2.4% 976 100%

16 100%

116 50.0% 113 48.7% 3 1.3% 116 50.0% 108 46.6% 8 3.4% 232 100%

399 40.9% 370 37.9% 29 3.0% 577 59.1% 554 56.8% 23 2.4% 976 100%

0 1

0 0

1 1

5 7

161

234

988

8 50.0% 8 50.0% 0 16 100%

8 50.0% 8 50.0% 0 8 50.0% 8 50.0% 0

Percentages may not sum to exactly 100% because of rounding.

1 Includes juvenile offenders arrested from April 28, 1996 (the date that JO recommendations were implemented by CJA) through June 30, 1997. Three juveniles arrested on April 27 were included because they were interviewed by CJA on April 28 and were processed using the new procedures. Only docketed cases were included.

- 26 -

ROR-Recommendation System for Juvenile Offenders, Part I: Validation Study

recognizance (38%). It is clear that CJA recommended many more JOs for release than were actually released at arraignment. Effect of the Recommendation System on Release at Arraignment Table 5 shows that the court did not follow CJA's recommendation for release in the majority of cases. Of course, the CJA recommendation is based entirely on community-ties information, whereas judges are required by statute to take other factors into account.14 Risk of flight is the only statutory reason for detaining a defendant in New York, but in weighing that risk, judges must take other things into consideration, such as the likelihood of conviction and the possible sentence. The prosecutor and defense attorney also play a role in the release decision. Consequently, we would not expect the CJA recommendation to be followed in every case. Regardless of recommendation, judges were most likely to set bail at arraignment (59% of those with a recommendation), which was usually not made at that time. Of 771 who were recommended for release, only 321 (42%) were released on recognizance. Even though less than half of the recommended JOs were released, the ROR rate for the “not recommended” category was only 31 percent, which is even lower. This means that a juvenile who was recommended was more likely to be released on recognizance than one who was not recommended, but only slightly: the chance of release increased from about three out of 10 to four out of 10. The most probable outcome for a JO receiving either recommendation was to be held on bail.

CPL 510.30 states that the release decision must be based on “the kind and degree of control or restriction that is necessary to secure [the defendant’s] court attendance when required. In determining that matter, the court must, on the basis of available information, consider and take into account . . . (i) [the defendant’s] character, reputation, habits and mental condition; (ii) His employment and financial resources; and (iii) His family ties and the length of his residence if any in the community; and (iv) His criminal record if any; and (v) His record of previous adjudication as a juvenile delinquent. . .; and (vi) [Prior failures to appear]; and (vii) . . . the weight of the evidence against him . . .; and (viii) . . . the sentence which may be or has been imposed upon conviction.”

- 27 -

ROR-Recommendation System for Juvenile Offenders, Part I: Validation Study

New York City Criminal Justice Agency Table 5 Release Status at Criminal Court Arraignment By CJA Recommendation 1996-1997 (14 months)1 Release Status at Arraignment

Released on recognizance Bail made

Bail not made

Remanded

Column Total

Recommended (J5) 321 41.6% (90.9%) 28 3.6% (100%) 420 54.5% (82.8%) 2 0.3% (12.5%) 771 100% (85.3%)

Not Recommended (J6) 30 30.6% (8.5%) 0

65 66.3% (12.8%) 3 3.1% (18.8%) 98 100% (10.8%)

Recommendation Category No No No RecomRecomRecommendation: mendation: mendation: Bench No NYSID Murder Warrant Charge (J7A) (J7B) (J7C) 1 0 1 3.7% 33.3% (0.3%) (0.3%) 0 0 0

4 100% (0.8%) 0

2 66.7% (0.4%) 0

4 100% (0.4%)

3 100% (0.3%)

15 55.6% (3.0%) 11 40.7% (68.8%) 27 100% (3.0%)

No Recommendation: Incomplete Interview (J7D) 0

1 100% (0.2%) 0

1 100% (0.1%)

Row Total

353 39.0% (100%) 28 3.1% (100%) 507 56.1% (100%) 16 1.8% (100%) 904 100% (100%)

Percentages may not sum to exactly 100% because of rounding. 84 cases without a JO recommendation stamp or without a release status were not included in this table.

Within each cell, percentages in the middle row are column percentages; percentages at the bottom (in parentheses and in italics) are row percentages. Note that 85.3% of the 904 cases with valid recommendation and release data were recommended for release. This is a larger percentage than shown in Table 3 (79.1% recommended for release) because in Table 3 the percentage is based on the entire sample of 988 cases.

Includes juvenile offenders arrested from April 28, 1996 (the date that JO recommendations were implemented by CJA) through June 30, 1997. Three juveniles arrested on April 27 were included because they were interviewed by CJA on April 28 and were processed using the new procedures. Only docketed cases were included in the 1996-1997 and the FY 1992 research files.

- 28 -

ROR-Recommendation System for Juvenile Offenders, Part I: Validation Study

Being recommended also increased the likelihood of making bail at arraignment, although that likelihood remained very small. Of 65 not-recommended JOs who had bail set, not one was able to post bail at arraignment; even for recommended JOs only 6 percent of those with bail set were able to post bail at arraignment (this percentage is not shown in the table: 28 of 448 with bail set). Combining those who made bail with those who were released on recognizance produces a slightly stronger relationship between recommendation and release than if only ROR were considered â&#x20AC;&#x201D; although the recommendation is intended only for ROR. The proportion of recommended JOs who were released either on recognizance or on bail was 45 percent, which is 14 percentage points higher than the proportion of not-recommended JOs who were released at arraignment. Conversely, almost all of the defendants who were released had been recommended for release: 91 percent of those who were released on recognizance had been recommended (see percentages in italics at the bottom of each cell, Table 5). However, this is not a particularly revealing finding given that 85 percent of the entire JO cohort were recommended for release. Table 5A presents the same data separately for each borough. In the Bronx, relatively high release rates combined with low recommended (J5) rates resulted in the release of a majority of defendants who were recommended for release (55% ROR) â&#x20AC;&#x201D; the only borough in which over half of the recommended defendants were released on recognizance. (In Staten Island, 53% of recommended JOs were released on recognizance, but the number is too small for a meaningful comparison.) As noted previously, Queens was the borough with the highest recommended rate (excluding Staten Island) and the lowest ROR rate, which combined to make this the borough with the lowest proportion of recommended juveniles who were released on recognizance (36%).

- 29 -

ROR-Recommendation System for Juvenile Offenders, Part I: Validation Study

New York City Criminal Justice Agency Table 5A Release Status at Criminal Court Arraignment By CJA Recommendation And Borough of Arrest 1996-1997 (14 months) BROOKLYN Release Status at Arraignment

Released on recognizance Bail made

Bail not made

Remanded

Column Total

Recommended

Not Recommended (J6) 7 25.9% (5.5%) 0

Recommendation Category No No No RecomRecomRecommendation: mendation: mendation: Bench No NYSID Murder Warrant Charge (J7A) (J7B) (J7C) 0 0 1 100% (0.8%) 0 0 0

(J5) 119 37.8% (93.7%) 12 3.8% (100%) 184 58.4% (85.2%) 0

20 74.1% (9.3%) 0

2 100% (0.9%) 0

315 100% (87.0%)

27 100% (7.5%)

2 100% (0.6%)

1 100% (0.3%)

10 58.8% (4.6% 7 41.2% (100%) 17 100% (4.7%)

No Recommendation: Incomplete Interview (J7D) 0

Row Total

127 35.1% (100%) 12 3.3% (100%) 216 59.7% (100%) 7 1.9% (100%) 362 100% (100%)

23 cases without a JO recommendation category or without a release status were not included in this table. Percentages may not sum to exactly 100% because of rounding. Within each cell, column percentages are in the middle row; row percentages are at the bottom (in parentheses and in italics).

- 30 -

ROR-Recommendation System for Juvenile Offenders, Part I: Validation Study

Table 5A (continued)

MANHATTAN Release Status at Arraignment

Released on recognizance Bail made

Bail not made

Remanded

Column Total

Recommended (J5) 51 38.6% (91.1%) 4 3.0% (100%) 77 58.3% (81.9%) 0

132 100% (83.5%)

Not Recommended (J6) 5 27.8% (8.9%) 0

12 67.8% (12.8%) 1 5.6% (25%) 18 100% (11.4%)

Recommendation Category No No No RecomRecomRecommendation: mendation: mendation: Bench No NYSID Murder Warrant Charge (J7A) (J7B) (J7C) 0 0 0

1 100% (1.1%) 0

2 100% (2.1%) 0

1 100% (0.6%)

2 100% (1.3%)

No Recommendation: Incomplete Interview (J7D) 0

1 25.0% (1.1%) 3 75.0% (75%) 4 100% (2.5%)

1 100% (1.1%) 0

1 100% (0.6%)

Row Total

56 35.4% (100%) 4 2.5% (100%) 94 59.5% (100%) 4 2.5% (100%) 158 100% (100%)

34 cases without a JO recommendation category or without a release status were not included in this table. Percentages may not sum to exactly 100% because of rounding.

QUEENS Release Status at Arraignment

Released on recognizance Bail made

Bail not made

Remanded

Column Total

Recommended (J5) 50 36.0% (96.2%) 10 7.2% (100%) 79 56.8% (84.9%) 0

139 100% (89.1%)

Not Recommended (J6) 2 15.4% (3.8%) 0

Recommendation Category No No No RecomRecomRecommendation: mendation: mendation: Bench No NYSID Murder Warrant Charge (J7A) (J7B) (J7C) 0 0 0

No Recommendation: Incomplete Interview (J7D) 0

11 84.6% (11.8%) 0

13 100% (8.3%)

3 75.0% (3.2%) 1 25.0% (100%) 4 100% (2.6%)

5 cases without a JO recommendation category or without a release status were not included in this table. Percentages may not sum to exactly 100% because of rounding.

- 31 -

Row Total

52 33.3% (100%) 10 6.4% (100%) 93 59.6% (100%) 1 0.6% (100%) 156 100% (100%)

ROR-Recommendation System for Juvenile Offenders, Part I: Validation Study

Table 5A (continued)

STATEN ISLAND Release Status at Arraignment

Released on recognizance Bail made Bail not made

Remanded Column Total

Recommended (J5) 8 53.3% (100%) 0 7 46.7% (87.5%) 0 15 100% (93.8%)

Not Recommended (J6) 0

0 1 100% (12.5%) 0 1 100% (6.3%)

Recommendation Category No No No RecomRecomRecommendation: mendation: mendation: Bench No NYSID Murder Warrant Charge (J7A) (J7B) (J7C) 0 0 0

No Recommendation: Incomplete Interview (J7D) 0

0 0

Row Total

8 50.0% (100%) 0 8 50.0% (100%) 0 16 100% (100%)

Percentages may not sum to exactly 100% because of rounding.

BRONX Release Status at Arraignment

Released on recognizance Bail made

Bail not made

Remanded

Column Total

Recommended (J5) 93 54.7% (84.5%) 2 1.2% (100%) 73 42.9% (76.0%) 2 1.2% (50%) 170 100% (80.2%)

Not Recommended (J6) 16 41.0% (14.5%) 0

21 53.8% (21.9%) 2 5.1% (50%) 39 100% (18.4%)

Recommendation Category No No No RecomRecomRecommendation: mendation: mendation: Bench No NYSID Murder Warrant Charge (J7A) (J7B) (J7C) 0 0 1 50.0% (0.9%) 0 0 0

1 100% (1.0%) 0

1 100% (0.5%)

No Recommendation: Incomplete Interview (J7D) 0

1 50.0% (1.0%) 0

2 100% (0.9%)

Row Total

110 51.9% (100%) 2 0.9% (100%) 96 45.3% (100%) 4 1.9% (100%) 212 100% (100%)

22 cases without a JO recommendation category or without a release status were not included in this table. Percentages may not sum to exactly 100% because of rounding. Within each cell, column percentages are in the middle row; row percentages are at the bottom (in parentheses and in italics).

- 32 -

ROR-Recommendation System for Juvenile Offenders, Part I: Validation Study

Ironically, Queens was also the borough in which the CJA recommendation had the largest association with the release decision. The difference in ROR rates between recommended defendants (36%) and not-recommended defendants (15%) was 21 percentage points in Queens, which is a larger difference than in the other boroughs (again excluding Staten Island). This could be an artifact of the small number of not-recommended defendants in Queens (n=13), but the same pattern has historically been found with adult cases in Queens where the numbers are far larger (see CJA’s Semi-Annual Report: Second Half of 1994, and earlier years). Effect of the Separate Components of the Recommendation System on ROR at Arraignment To investigate the possibility that arraignment judges may have given more weight to either of the elements of the recommendation system singly than to the assigned recommendation category, we analyzed the effect on the ROR rate of the two components separately. It could be the case that judges had long considered school attendance in making the release decision, for instance, and continued to rely primarily on this criterion after the recommendation system was implemented. The row and column totals in Table 6 present ROR rates separately for each value of the school attendance variable, and for each value of whether the juvenile is expecting someone at arraignment. The inner cells of the table present ROR rates for all the possible combinations of the two criterion items. The ROR rates for the “yes” responses on both items are in the range of 40 percent (for “Yes - expects someone at arraignment” and “Yes verified - school attendance”) to 42 percent (for “Yes unverified - school attendance”). These rates are nearly identical to the 42 percent ROR rate for recommended JOs presented in Table 5, so it does not appear that either factor was given more weight individually than the recommendation category.

- 33 -

ROR-Recommendation System for Juvenile Offenders, Part I: Validation Study

New York City Criminal Justice Agency Table 6 ROR Rates at Criminal Court Arraignment By Criterion Items for the Recommendation System 1996-1997 (14 months) Expects Someone at Arraignment School attendance

Yes verified

Yes 187 (42%) base=448

No 18 (28%) base=64

Row Total1 205 (40%) base=512

Yes (unverified)

116 (47%) base=248

15 (23%) base=65

131 (42%) base=313

7 (30%) base=23

2 (33%) base=6

9 (31%) base=29

No (unverified)

6 (22%) base=27

1 (7%) base=15

7 (17%) base=42

No verified

9 (43%) base=21

2 (18%) base=11

11 (34%) base=32

Column Total

330 (41%) base=810

40 (24%) base=166

U (unresolved conflict)

Row and column totals were calculated on the total number for that value, including cases with missing values on the other variable in the table. For this reason the totals are based on a larger number than the sum of the cells. - 34 -

ROR-Recommendation System for Juvenile Offenders, Part I: Validation Study

In a departure from the recommendation system, however, judges apparently did not consider juveniles to be good risks if they were not expecting someone at arraignment, even if it was verified that they were attending school. A JO with verified school attendance would be recommended for ROR regardless of expectations of someone at arraignment (in the absence of any policy exclusions), because of a low risk of failure to appear. Nevertheless, only 28 percent of the 64 juveniles in this group (verified school attendance and not expecting anyone at arraignment) were released on recognizance at arraignment. (On the other hand, juveniles with “No verified” school attendance who were expecting someone at arraignment had an unexpectedly high ROR rate — 43 percent — but the 21 defendants in this group were too few for any conclusions to be drawn.) Effect of the Recommendation System on Release Prior to Sentence Thus far the focus has been on release at arraignment in Criminal Court, since the CJA recommendation for both juveniles and adults is designed to affect only that arraignment decision. Any effect the recommendation might have on post-arraignment release would be unexpected, and in fact Table 7 shows that the modest effect that recommendation had on release at arraignment was all but erased when post-arraignment releases were included. Sixty-two percent of recommended JOs, compared to 61 percent of not-recommended JOs, were released on recognizance at their first release prior to sentencing. When release at Criminal Court arraignment is excluded from the analysis, JOs who were recommended actually had a postarraignment ROR rate that was lower than the ROR rate for not-recommended defendants (not shown). Procedural considerations in post-arraignment release decisions sometimes override concerns about risk of flight, so it is not surprising that the association between recommendation and release at arraignment did not persist beyond the arraignment appearance. For example, it is

- 35 -

ROR-Recommendation System for Juvenile Offenders, Part I: Validation Study

New York City Criminal Justice Agency Table 7 Release Outcome Prior to Sentence By CJA Recommendation 1996-1997 (14 months) Recommendation Category

Ever Released Status

Recommended

(J5)

Released on recognizance at first release Posted bail at first release Total released (prior to sentence or final disposition if no conviction)

Held on bail at arraignment, never released Remanded at arraignment, never released Total detained (through sentence or final disposition if no conviction)

Column Total

Not recommended

(J6)

No No No No recommendation recommendation recommendation recommendation because of because no because of because of Bench Warrant NYSID murder charge incomplete interview (J7A) (J7B) (J7C) (J7D)

Row Total

475 62% (88%)

60 61% (11%)

2 50% (<1%)

1 33% (<1%)

2 7% (<1%)

—

540 60% (100%)

112 15% (88%)

10 10% (8%)

—

1 33% (<1%)

4 15% (3%)

1 100% (<1%)

128 14% (100%)

587 76% (88%)

70 71% (10%)

2 50% (<1%)

2 67% (<1%)

6 22% (<1%)

1 100% (<1%)

668 (74%) (100%)

183 24% (82%)

27 28% (12%)

2 50% (1%)

1 33% (<1%)

10 37% (4%)

—

223 25% (100%)

1 <1% (8%)

1 1% (8%)

—

11 41% (85%)

—

13 1% (100%)

184 24% (78%)

28 29% (12%)

2 50% (<1%)

1 33% (<1%)

21 78% (9%)

—

236 26% (100%)

771 100% (85%)

98 100% (11%)

4 100% (<1%)

3 100% (<1%)

27 100% (3%)

1 100% (<1%)

904 100% (100%)

Percentages may not sum to exactly 100% because of rounding. 84 cases without a JO recommendation stamp or without a release status were not included in this table. Within each cell, percentages in the top row are column percentages; percentages at the bottom (in parentheses and in italics) are row percentages.

- 36 -

ROR-Recommendation System for Juvenile Offenders, Part I: Validation Study

likely that many juveniles in this study who had been detained at arraignment were later released because an indictment was not obtained within the required time.15 A positive recommendation was associated with a slightly higher likelihood of making bail, with the result that 15 percent of recommended JOs versus 10 percent of not-recommended JOs posted bail at first release prior to sentence. (About the same relationship was found for postarraignment releases when arraignment bail-making was excluded; not shown.) Information on defendantsâ&#x20AC;&#x2122; financial resources was not available for this research, but it is reasonable to suppose that ability to make bail is related to community ties and would thus also be related to a positive recommendation.

Section 180.80 of the New York Criminal Procedure Law mandates that a defendant held in custody upon a felony complaint must be released on recognizance if an indictment has not been voted or a hearing commenced within 120 hours after arrest (144 hours if a Saturday, Sunday, or legal holiday occurs during such custody). With the data available for this research, we could not conclusively identify the cases in which the defendant was released to comply with Section 180.80, but an examination of arrest, release, and indictment dates indicates that at least 40% of the post-arraignment RORs probably fit in this category.

- 37 -

ROR-Recommendation System for Juvenile Offenders, Part I: Validation Study

V. HOW WELL DID THE ROR-RECOMMENDATION SYSTEM FOR JUVENILES PREDICT FAILURE TO APPEAR? The prior research on FY 1992 arrests showed that JOs who met the criteria for a positive recommendation for ROR were at a lower risk of failure to appear than JOs who did not meet the criteria. The first step in validating this relationship for the 1996-1997 cohort is to compare FTA rates for recommended versus not-recommended defendants. As mentioned earlier, the issuance of a non-stayed bench warrant was used as the indicator of failure to appear, and only one failure per case was counted. Case-based FTA rates were obtained by dividing the number of cases with a failure to appear by the total number of at-risk cases. Table 8 presents FTA rates for each recommendation category, including only JOs who were released at some time prior to sentencing (or disposition, if not convicted). The FTA rates are given as percentages in parentheses in each cell, citywide and separately for each borough. Each cell of the table also contains the number of JOs with a failure to appear, and the base for calculating the FTA rate. The columns to compare in this table are “recommended” (J5) versus “not recommended” (J6), because there are too few cases in the “J7” categories (policy exclusions) for any meaningful comparisons. Moreover, the recommendation system does not attempt to assess the risk of failure for JOs in these categories (with the possible exception of J7A, “Bench warrant attached to NYSID sheet”). They are included in the table for comprehensiveness. Citywide (bottom row), there is a large difference in the FTA rate between recommended JOs (9%) and not-recommended JOs (29%). Since the not-recommended group was over three times more likely to fail to appear than the recommended group, we can conclude that the recommendation predicted FTA fairly well. This conclusion is supported by the results of

- 38 -

ROR-Recommendation System for Juvenile Offenders, Part I: Validation Study

New York City Criminal Justice Agency Table 8 Failure to Appear1 By Recommendation Category and Borough of Arrest 1996-1997 (14 months) AT-RISK ONLY Recommendation Category No Recommendation: Bench Warrant (J7A)

No Recommendation: No NYSID

5 (31%) (base=16)

0 (base=1)

14 (13%) (base=108)

4 (27%) (base=15)

1 (100%) (base=1)

—

Queens

6 (6%) (base=98)

1 (11%) (base=9)

—

7 (7%) (base=107)

Staten Island

1 (8%) (base=13)

0 (base=1)

—

1 (7%) (base=14)

Bronx

8 (6%) (base=138)

10 (36%) (base=28)

—

0 (base=2)

—

18 (11%) (base=168)

Column Total

50 (9%) (base=580)

20 (29%) (base=69)

1 (50%) (base=2)

0 (base=6)

Borough of Arrest

Recommended

Not Recommended

(J5)

(J6)

Brooklyn

21 (9%) (base=223)

Manhattan

(J7B)

No Recommendation: Murder Charge (J7C) 0 (base=4)

No Recommendation: Incomplete Interview (J7D) —

0 (base=1)

Row Total

26 (11%) (base=245) 20 (16%) (base=126)

72 (11%) (base=660)

45 cases without a JO recommendation stamp were not included in this table. The number (and percentage) within each cell represents the number (and percentage) who failed to appear. The base within each cell is the number who failed to appear plus the number who did not fail to appear.

FTA was counted up to the sentencing appearance (or to final disposition, if not convicted). Case-based FTA rates were obtained by dividing the number of cases with a failure to appear by the total number of at-risk cases. - 39 -

ROR-Recommendation System for Juvenile Offenders, Part I: Validation Study

configuration analysis and a logistic regression model, which are presented later in this section. There were borough differences in FTA rates, and in the strength of the relationship between recommendation and FTA. The FTA rate was highest in Manhattan (16%) and lowest in Queens and Staten Island (7% in each). Recall that the recommended rate was lowest in Manhattan (Table 3), and highest in Queens and Staten Island. These borough differences in recommended rates account for some of the difference in FTA rates, but not much. Among recommended JOs only, Manhattan had an FTA rate (13%) that was still considerably larger than the rate for Queens (6%). (As noted throughout, Staten Island had too few cases for the results there to be interpreted.) The predictive ability of the recommendation system was strongest in the Bronx, where 6 percent of recommended JOs failed to appear, compared to 36 percent of those who were not recommended — a difference of 30 percentage points (or, the not-recommended group was six times more likely to FTA than the recommended group). The relationship of recommendation to FTA seemed relatively weak in Queens (6% FTA for recommended defendants; 11% for notrecommended defendants), but the number of “not recommended” in Queens was so small (9) that caution must be used in interpreting the results there as well as in Staten Island. Interrelationships With Other Factors Several possible interrelationships among ROR recommendation, FTA, and a third variable were analyzed. In brief, the results showed that: ·

Release type had no overall effect on FTA, but it did have an effect on FTA for some juveniles, depending on their recommendation for ROR. Also, the effect of recommendation on FTA varied by type of release.

For juveniles who were released on bail, the recommendation affected the amount of bail set,

- 40 -

ROR-Recommendation System for Juvenile Offenders, Part I: Validation Study

and the amount of bail affected FTA. ·

The severity of the arraignment charge was associated with FTA and with recommendation category. Controlling for charge severity, the effect of the ROR recommendation on FTA remained strong. Controlling for recommendation, the effect of charge severity on FTA remained moderate.

Each of these interrelationships is explored below. Recommendation, FTA, and Release Type Data have already been presented (in Table 5) showing that the ROR recommendation was related to release status at arraignment, although not very strongly. A defendant who was not recommended was more likely to have bail set — and to be detained at arraignment. Very few JOs were released on bail at arraignment (28), but enough posted bail post-arraignment (100 additional) to make possible an analysis of the three-way relationship between recommendation, FTA rates, and release type (ROR or bail). In spite of the presumption that bail discourages failure to appear, Table 9 shows that the FTA rate was identical — 11 percent — regardless of whether release was on recognizance or on bail. Without controlling for any other factors, then, the type of release had no effect at all on FTA; bail apparently provided no additional incentive for appearing in court. When recommendation category was taken into account, a possible relationship between bail and FTA appeared. For JOs who were recommended for ROR, the FTA rate was still about the same regardless of release type; the trivial difference found in the FTA rate between JOs released on recognizance (8%) and those released on bail (10%) was in the opposite direction from what would be expected if bail discourages failure to appear. But for JOs who were not recommended, being released on bail apparently lowered the FTA rate from 31 percent for

- 41 -

ROR-Recommendation System for Juvenile Offenders, Part I: Validation Study

New York City Criminal Justice Agency Table 9 Failure to Appear1 By Release Type and Recommendation Category 1996-1997 (14 months) AT-RISK ONLY Recommendation Category Release Type at First Release

(J5)

(J6)

No Recommendation: Bench Warrant (J7A)

Released on recognizance

39 (8%) (base=468)

18 (31%) (base=59)

1 (50%) (base=2)

0 (base=1)

0 (base=2)

â&#x20AC;&#x201D; (base=0)

58 (11%) (base=532)

Released on bail

11 (10%) (base=112)

2 (20%) (base=10)

â&#x20AC;&#x201D; (base=0)

1 (100%) (base=1)

0 (base=4)

0 (base=1)

14 (11%) (base=128)

Column Total

50 (9%) (base=580)

20 (29%) (base=69)

1 (50% (base=2)

1 (50%) (base=2)

0 (base=6)

0 (base=1)

72 (11%) (base=660)

Recommended

Not Recommended

No Recommendation: No NYSID (J7B)

No Recommendation: Murder Charge (J7C)

No Recommendation: Incomplete Interview (J7D)

Row Total

- 42 -

ROR-Recommendation System for Juvenile Offenders, Part I: Validation Study

defendants released on recognizance to 20 percent for those released on bail. This is difficult to interpret, though, because only ten juveniles who were not recommended were released on bail. The FTA rate for such a small base number is inherently unstable. With the same reservation in mind, one might also conclude from Table 9 that for juveniles who were released on recognizance, the CJA recommendation was more strongly associated with FTA than for those released on bail. For those who were released on recognizance, having been recommended for ROR decreased the FTA rate by 23 percentage points, from 31 percent for those who were not recommended to 8 percent for those who were recommended. The relationship between recommendation and FTA did not appear as strong for those released on bail. (After all, the recommendation is for ROR, not release on bail.) For the juveniles who posted bail, having been recommended for ROR decreased the FTA rate by 10 percentage points. Recommendation, FTA, and Bail Amount If release on bail affected FTA only for high-risk (not-recommended) juveniles, perhaps they were required to post higher amounts than low-risk (recommended) juveniles. If that were true, it might suggest that the amount is what made the difference. Bail amounts for the recommended JOs might have been too low to provide any incentive to appear. In spite of the very small number of not-recommended JOs who were released on bail, we were still interested in finding out whether this explanation might have any basis in fact. This was done by comparing bail amounts posted by recommended and not-recommended defendants. Table 10 shows that both mean and median bail amounts were actually higher for JOs who were recommended for release, thus disproving the hypothesis that bail was ineffective for recommended JOs because amounts set for them were lower. Table 10 includes only defendants

- 43 -

ROR-Recommendation System for Juvenile Offenders, Part I: Validation Study

New York City Criminal Justice Agency Table 10 Bail Amount Posted at First Release By Recommendation Category (Juveniles released on bail at earliest release) Bail Statistics

Recommendation Category Recommended Not Recommended (J5)

Mean Median Minimum Maximum Standard deviation Number

$2951 $1500 $250 $50,000 5455 112

(J6) $2190 $1000 $150 $10,000 2870 10

Mean: the arithmetic average (sum of bail amounts divided by number of cases). Median: the midpoint; half of the cases have a value that is greater and half have a value that is less. Standard deviation: a measure of dispersion, indicating how closely the data are clustered around the mean. The larger the standard deviation, the less homogeneous are the data. The large standard deviations in this table indicate that many cases have bail amounts that differ considerably from the mean. The standard deviation is the square root of the variance. The variance is calculated by subtracting each bail amount from the mean, squaring the difference, and taking the average of the squared differences.

- 44 -

ROR-Recommendation System for Juvenile Offenders, Part I: Validation Study

who were released on bail; we found a similar result for everyone with bail set at arraignment, including those who were later released on recognizance, and those who were never released. For all juveniles who had bail set at arraignment, the median bail amount (although not the mean) was higher for JOs who were recommended than for those who were not recommended (not shown). This makes us more confident in concluding that higher bail amounts cannot be the reason why having bail set apparently lowered the FTA rate for not-recommended JOs without having any effect on FTA among those who were recommended. This implies the further conclusion that high bail did not discourage failure to appear, and that hypothesis is apparently confirmed by the data in Table 11, which compares bail amounts for those who failed to appear versus those who did not fail. Again, the analysis is hampered by the small number of defendants in the comparison groups. Almost every cell in the borough comparisons has fewer than 20 cases, and the citywide total for the FTA group is only 15. The results, therefore, cannot be considered conclusive. With that caveat in mind, however, we observe that those who failed to appear had posted higher median bail amounts than those who did not fail. Citywide, the median bail amount was $500 higher for JOs who failed to appear ($2000) than for those who did not fail ($1500). The mean was higher (by $597) for the â&#x20AC;&#x153;no FTAâ&#x20AC;? group, but it was skewed upward by two cases with extremely high bail amounts of $50,000. The high standard deviation (a measure of how widely dispersed are the data around the mean) for the no-FTA group is another indication that the mean is skewed, suggesting that the more meaningful comparison is the median amount. We do not offer much comment on borough differences in these findings because, with fewer than 10 failures to appear in every borough, no reliable conclusions can be drawn. It does appear that in Manhattan, alone of all the boroughs, high bail may have lowered the FTA rate.

- 45 -

ROR-Recommendation System for Juvenile Offenders, Part I: Validation Study

New York City Criminal Justice Agency Table 11 Range and Average Bail Amount Posted at First Release By Borough of Arrest and Occurrence of FTA 1996-1997 (14 months) (Juveniles released on bail at earliest release) Borough

Total Released on Bail at First Release

No FTA

FTA

52 (90%) $3323 $1000 $250 $50,000 7631

6 (10%) $3833 $2250 $500 $10,000 3804

58 (100%) $3376 $1000 $250 $50,000 7307

17 (81%) $3985 $2500 $250 $15,000 4037

4 (19%) $2250 $1750 $500 $5000 1936

21 (100%) $3655 $2500 $250 $15,000 3754

30 (91%) $2417 $1500 $500 $25,000 4432

3 (9%) $3833 $5000 $1500 $5000 2021

33 (100%) $2545 $1500 $500 $25,000 4269

Brooklyn · · · · · ·

Number (%) Mean Median Minimum Maximum Standard deviation

Manhattan · · · · · ·

Number (%) Mean Median Minimum Maximum Standard deviation

Queens · · · · · ·

Number (%) Mean Median Minimum Maximum Standard deviation

Staten Island · · · · · ·

Number (%) Mean Median Minimum Maximum Standard deviation

3 (100%) $2167 $1000 $500 $5000 2466

0 — — — — —

3 (100%) $2167 $1000 $500 $5000 2466

Bronx · · · · · ·

Number (%) Mean Median Minimum Maximum Standard deviation

19 (90%) $7574 $2500 $150 $50,000 12,215

2 (10%) $2500 $2500 $2000 $3000 707

21 (100%) $7090 $2500 $150 $50,000 11,689

121 (89%) $3830 $1500 $150 $50,000 7553

15 (11%) $3233 $2000 $500 $10,000 2678

136 (100%) $3764 $1500 $150 $50,000 7176

Citywide · · · · · ·

Number (%) Mean Median Minimum Maximum Standard deviation

1 case missing bail amount was excluded from the analysis (the case had no FTA).

- 46 -

ROR-Recommendation System for Juvenile Offenders, Part I: Validation Study

Total Released on Bail at First Release

No FTA

FTA

52 (90%) $3323 $1000 $250 $50,000 7631

6 (10%) $3833 $2250 $500 $10,000 3804

58 (100%) $3376 $1000 $250 $50,000 7307

17 (81%) $3985 $2500 $250 $15,000 4037

4 (19%) $2250 $1750 $500 $5000 1936

21 (100%) $3655 $2500 $250 $15,000 3754

30 (91%) $2417 $1500 $500 $25,000 4432

3 (9%) $3833 $5000 $1500 $5000 2021

33 (100%) $2545 $1500 $500 $25,000 4269

Brooklyn · · · · · ·

Number (%) Mean Median Minimum Maximum Standard deviation

Manhattan · · · · · ·

Number (%) Mean Median Minimum Maximum Standard deviation

Queens · · · · · ·

Number (%) Mean Median Minimum Maximum Standard deviation

Staten Island · · · · · ·

Number (%) Mean Median Minimum Maximum Standard deviation

3 (100%) $2167 $1000 $500 $5000 2466

0 — — — — —

3 (100%) $2167 $1000 $500 $5000 2466

Bronx · · · · · ·

Number (%) Mean Median Minimum Maximum Standard deviation

19 (90%) $7574 $2500 $150 $50,000 12,215

2 (10%) $2500 $2500 $2000 $3000 707

21 (100%) $7090 $2500 $150 $50,000 11,689

121 (89%) $3830 $1500 $150 $50,000 7553

15 (11%) $3233 $2000 $500 $10,000 2678

136 (100%) $3764 $1500 $150 $50,000 7176

Citywide · · · · · ·

Number (%) Mean Median Minimum Maximum Standard deviation

1 case missing bail amount was excluded from the analysis (the case had no FTA).

- 46 -

ROR-Recommendation System for Juvenile Offenders, Part I: Validation Study

However, this finding would have to be replicated in a larger sample for any importance to be attached to it. Table 12 shows in a complementary way that low bail was associated with higher FTA rates. Bail amounts posted at the first release were dichotomized into “low bail” (lowest amount through the median, which was $1500) and “high bail” (over $1500). The FTA rate was higher for defendants with high bail (16% of them failed to appear) than for defendants with low bail (7% failed). This did not change when recommendation was held constant, but this further division into subgroups produced cells with such small numbers that once again the results must be viewed with considerable caution. Recommendation, FTA, and Charge Severity Charge severity is a possible confounding factor in the relationship between ROR recommendation and FTA. If recommended JOs were charged with less severe offenses than not-recommended JOs, then perhaps it was the severity of the offense rather than the recommendation category that was associated with risk of flight. Table 13 presents FTA rates separately for cases in which the most severe arraignment affidavit charge was a Class-B felony and for those in which the charge was a Class-C felony, controlling for recommendation. (There were too few JOs with Class-A felonies to analyze.) Charge severity did affect FTA: those charged with more serious (Class-B) felonies had an FTA rate (13%) almost double that of the group charged with less serious (Class-C) felonies (7%). Those who were recommended for ROR were also a little more likely to be charged with a Class-C felony. The percentage of Class-C felonies in each recommendation category is not given in the table but can be calculated: 32 percent of recommended JOs were charged with a Class-C felony (183 out of 575), compared to 26 percent of not-recommended JOs (18 out of 68). - 47 -

ROR-Recommendation System for Juvenile Offenders, Part I: Validation Study

New York City Criminal Justice Agency Table 12 Failure to Appear1 By Bail Amount (High or Low) and Recommendation Category 1996-1997 (14 months) (Juveniles released on bail at earliest release) Recommendation Category

Bail Amount Posted at First Release Low Bail ($150 to $1500) High Bail ($1501 to $50,000) Column Total

Recommended

Not Recommended

(J5)

(J6)

No Recommendation: Policy exclusions (combined) (J7)

Row Total

5 (8%) (base=63)

0 (base=7)

0 (base=2)

5 (7%) (base=72)

6 (12%) (base=49)

2 (67%) (base=3)

1 (25%) (base=4)

9 (16%) (base=56)

11 (10%) (base=112)

2 (20%) (base=10)

1 (17%) (base=6)

14 (11%) (base=128)

8 cases without a JO recommendation stamp were not included in this table. The number (and percentage) within each cell represents the number (and percentage) who failed to appear. The base within each cell is the number who failed to appear plus the number who did not fail to appear.

- 48 -

ROR-Recommendation System for Juvenile Offenders, Part I: Validation Study

New York City Criminal Justice Agency Table 13 Failure to Appear1 By Arraignment Affidavit Charge Severity and Recommendation Category 1996-1997 (14 months) AT-RISK ONLY Affidavit Charge Severity Class-B Felony Class-C Felony Column Total

Recommended J5

Not Recommended J6

(10%)

(base=392) 9

(5%)

(base=183) 50

(9%)

(base=575) Difference in percent between Class B and Class C

(32%)

(base=50) 4

1 1 2

Row Total

(13%)

Difference in percent between J5 and J6 22

(base=451)

(100%)

(base=1)

(29%)

(base=68)

(11%)

(base=9)

(22%)

(base=18) 20

No Recommendation J7

(7%)

(base=202) (20%)

(base=10)

(11%)

(base=653)

8 cases with charges of other severity classes and 44 additional cases without a valid JO recommendation category were not included in this table.

- 49 -

ROR-Recommendation System for Juvenile Offenders, Part I: Validation Study

However, charge severity did not account for the relationship between recommendation and FTA. Within each severity class, recommendation predicted FTA as well as it did for the sample as a whole.

For JOs charged with a Class-B felony, being “recommended” versus “not

recommended” made a difference in the FTA rate of 22 percentage points (10% of recommended JOs failed to appear, compared with 32% of not-recommended JOs). For JOs charged with a Class-C felony, being “recommended” versus “not recommended” made a difference in the FTA rate of 17 percentage points (5% of recommended JOs failed to appear, compared with 22% of not-recommended JOs). The difference made by recommendation, controlling for severity class, is reported in the right-hand column of Table 13. The difference that severity class made, controlling for recommendation, is reported in the bottom row of the table (a difference of 5 percentage points for recommended JOs; 10 for notrecommended JOs). Comparing these two sets of differences in percentage points, one can see that the difference made by recommendation was the greater of the two. This indicates that while charge severity did exert an independent effect on FTA, recommendation exerted a greater effect.

- 50 -

ROR-Recommendation System for Juvenile Offenders, Part I: Validation Study

Replication of FY 1992 Configuration Analysis The preceding sections show that the ROR-recommendation system for juveniles predicted failure to appear with some success, and that this relationship did not disappear when release type, bail amount, or charge severity were held constant. These findings constitute the first step in validating the ROR-recommendation system. The next step is to replicate the analyses done on the FY 1992 data set that were used to develop the system. In this way we can determine if the predictive power of the individual components of the ROR-recommendation system equals their predictive power for the FY 1992 arrests, and if the proportion of correct predictions made by the system as a whole matches that of the earlier analyses. The earlier analyses can be conceptualized in two parts. First, a configuration analysis was done in order to identify the criteria to be included in the recommendation system. From the results obtained, it was then possible to prepare classification tables showing the proportion of JOs who would be correctly classified using the system. The task now is to repeat those analyses on the current (validation) data set, and compare the results with the FY 1992 results. The configuration analyses will show if the individual criteria used in the ROR-recommendation system predicted FTA as well for the 1996-1997 cohort as they did for FY 1992; and the classification tables will show if the proportion of correct predictions made by the system was as high for the more recent cohort as it was for the earlier one. Configuration analysis is a technique for identifying a hierarchy of factors that affect the dependent variable â&#x20AC;&#x201D; in this case, FTA. The procedure begins by dividing a sample into groups with different outcomes (such as FTA rates) based on one factor, then subdividing the groups again based on some other factor, and so on until no further factor can be found with an effect on the dependent variable.

The factor with the strongest statistically significant effect on the

- 51 -

ROR-Recommendation System for Juvenile Offenders, Part I: Validation Study

outcome is used at each stage as the basis for the division. The overall FTA rate for the at-risk JOs in the FY 1992 cohort was about 8 percent. Over 30 variables were examined to determine whether they had any effect on this rate. The factor with the strongest statistically significant relationship to FTA turned out to be school attendance. For those with either full- or part-time school attendance that was verified, the FTA rate was only 6 percent. For juveniles who said they were not in school, and that response was verified, the FTA rate was 21 percent. The first division, then, consisted of two groups with vastly different FTA rates based on verified school attendance. (For a complete description of this process, and a list of the independent variables that were tested, see Winterfield, 1996.) That left a large group whose response to the school attendance question could not be verified, and for them there was one additional factor that affected FTA rate: expecting someone at arraignment, which is a way to measure the juvenileâ&#x20AC;&#x2122;s sense of social support. JOs with an unverified school status who expected someone at arraignment had an FTA rate of 10 percent, compared to 16 percent for those who did not expect anyone. The second division, then, consisted of two groups with different FTA rates based on expecting someone at arraignment. These results are illustrated in Figure 3. Note that this figure resembles Figure 1 (page 8), which illustrated how school attendance and expecting someone at arraignment are used for assigning recommendation categories. Figure 3 shows the same criterion items with numbers filled in to show the distribution of responses for the FY 1992 arrestees, and the proportion in each box who failed to appear at some time prior to sentencing, or prior to final disposition if there was no conviction (or prior to the final appearance date if the case was still pending at the close of data collection).

- 52 -

ROR-Recommendation System for Juvenile Offenders, Part I: Validation Study

New York City Criminal Justice Agency Figure 3 Configuration Analysis Results For Juvenile Offenders Arrested in FY 1992 FY 1992 JOs who were at risk of failure N = 1196* 8.4% FTA

School Attendance

Yes Verified N=646 6% FTA (n=39)

No verification (yes or no), or Conflicting statements made by contact and defendant N=397

No Verified N=39 21% FTA (n=8)

Expects Someone at Arraignment Yes N=304 10% FTA (n=30)

Total: Yes verified + Yes N = 950 7% FTA (n=69)

No N=93 16% FTA (n=15)

Comparing FTA rates: 17% is greater than 7% by a factor of 2.4

Total: No verified + No N = 132 17% FTA (n=23)

* The individual branches sum to 1,082, excluding 114 cases that are missing data on school attendance, expects someone at arraignment, or both. FTA was counted up to the sentencing appearance (or to final disposition, if not convicted). Case-based FTA rates were obtained by dividing the number of cases with a failure to appear by the total number of at-risk cases. Adapted from Winterfield, Laura A., Developing a Release-On-Recognizance System for Juveniles Arraigned in New York City Adult Courts. CJA, May 1996.

- 53 -

ROR-Recommendation System for Juvenile Offenders, Part I: Validation Study

The numbers in the four branches are summed in the unconnected boxes at the bottom to show the combined FTA rates. For the shaded boxes on the left, the combined FTA rate was 7 percent. These juveniles said they were in school, and this response was verified; or their school attendance was not verified and they were expecting someone at arraignment. For the unshaded boxes on the right, the combined FTA rate was 17 percent. These juveniles said they were not in school, and this response was verified; or their school attendance was not verified and they were not expecting anyone at arraignment. If CJA recommendations had been assigned to these juveniles, these would be the FTA rates for “recommended” defendants (7%) and “not recommended” defendants (17%) (except that policy exclusions would subtract some defendants from both groups). The next step was to repeat the same analysis on the 1996-1997 JO cohort. This would tell us if school attendance and expecting someone at arraignment predicted failure to appear just as well for the validation cohort as they did for the FY 1992 cohort. Figure 4 displays these results. There are several comparisons worth noting between the FY 1992 and 1996-1997 results: ·

Verified school attendance did not predict the FTA rate quite as well for the 1996-1997 cohort. For FY 1992 arrests, the FTA rate for juveniles with “No verified” school attendance was more than three times the rate for those with “Yes verified” school attendance (21% compared to 6%, a difference of 15 percentage points). For the 1996-1997 arrests, juveniles with “No verified” school attendance were about twice as likely to fail to appear (19% compared to 9%, a difference of 10 percentage points). For FY 1992 cases, this variable had a much stronger relationship to FTA than did expecting someone at arraignment, but the two factors were more equally related to FTA in the 1996-1997 cohort.

- 54 -

ROR-Recommendation System for Juvenile Offenders, Part I: Validation Study

New York City Criminal Justice Agency Figure 4 Configuration Analysis Results For Juvenile Offenders Arrested in FY 1996-1997 (14-month cohort) 1996-97 JOs who were at risk of failure N = 705* 10.7% FTA

School Attendance

Yes Verified N=382 9% FTA (n=34)

No verification (yes or no), or Conflicting statements made by contact and defendant N=266

No Verified N=21 19% FTA (n=4)

Expects Someone at Arraignment Yes N=209 11% FTA (n=22)

Total: Yes verified + Yes N = 591 9% FTA (n=56)

No N=57 25% FTA (n=14)

Comparing FTA rates: 23% is greater than 9% by a factor of 2.6

Total: No verified + No N = 78 23% FTA (n=18)

* The individual branches sum to 669, excluding 36 cases that are missing data on school attendance. FTA was counted up to the sentencing appearance (or to final disposition, if not convicted). Case-based FTA rates were obtained by dividing the number of cases with a failure to appear by the total number of at-risk cases.

- 55 -

ROR-Recommendation System for Juvenile Offenders, Part I: Validation Study

Expecting someone at arraignment was a better predictor of FTA for the 1996-1997 cohort than it was for the earlier cohort. For the group with school attendance status that was not verified, not expecting someone at arraignment more than doubled the likelihood of FTA for the 1996-1997 cohort (11% if expecting someone compared to 25% if not expecting someone, a difference of 14 percentage points). For juveniles arrested in FY 1992, this factor increased the likelihood of FTA, but did not come close to doubling it (10% if expecting someone, compared to 16% if not expecting someone, a difference of 6 percentage points).

The combined elements of the JO recommendation system produced similar results, even though the contribution of each factor changed. In both cohorts, those who fulfilled the criteria for the “not recommended” release category were more than twice as likely to fail to appear as those who fulfilled the criteria for the “recommended” release category. (Note that these results are a little different from the results for the recommendation system itself shown in Tables 8 and 9, primarily because in the data presented earlier, policy exclusions were removed from the “recommended” and “not recommended” categories. When the J7 [no recommendation] categories were excluded, the difference in FTA rates between “recommended” and “not recommended” increased.)

The FTA rates were slightly higher for the 1996-1997 JOs, especially for those who did not meet the criteria for a positive recommendation (in the unshaded boxes). The rates in the shaded boxes (“recommended”) were only slightly higher for the 1996-1997 cases. Since a large majority of JOs fit the “recommended” criteria in both cohorts, the overall rates did not rise as much in 1996-1997 as did the rates for “not recommended” juveniles. Table 2 showed that the overall FTA rate rose from 8 percent for FY 1992 to 10 percent for FY 1997.

- 56 -

ROR-Recommendation System for Juvenile Offenders, Part I: Validation Study

To summarize: we were able to replicate the configuration analysis of FY 1992 arrests with results that validated the ability of the individual components of the ROR-recommendation system to predict FTA for a more recent cohort of JOs. The contribution of the individual components shifted, but the predictive power of the combined elements matched (even slightly exceeded) their predictive power in the original set of arrests, as measured by the difference in percentage points between the FTA rates for recommended and not-recommended JOs. Classification Tables The purpose of a classification table is to evaluate a risk-assessment instrument (or any other predictive model) in terms of the percentage of correct predictions it made for a given set of data. In the present research, prediction is made in terms of FTA. A “recommended” JO is predicted not to fail to appear for any scheduled court appearances; this is a negative prediction in terms of FTA. A “not recommended” JO is predicted to fail to appear for at least one scheduled court appearance; this is a positive prediction with regard to FTA.

When the

prediction is compared to the actual outcome, there are four possible combinations: ·

True negatives: recommended for ROR and did not fail (predicted not to fail, and did not fail);

False negatives: recommended for ROR and did fail (predicted not to fail, but failed);

True positives: not recommended for ROR and failed (predicted to fail, and failed);

False positives: not recommended for ROR and did not fail (predicted to fail, but did not fail). Table 14 compares classification tables for FY 1992 with the results for the 1996-1997

arrests. Both parts of the table present data on the number and percent of true and false negatives, and true and false positives. The number of total correct predictions (true negatives plus true positives) is given in the row under the heavy double line: 84 percent correctly predicted for FY 1992, and 85 percent for 1996-1997. - 57 -

ROR-Recommendation System for Juvenile Offenders, Part I: Validation Study

New York City Criminal Justice Agency Table 14 Classification Tables for FY 1992 and 1996-1997 JO Cohorts Predicted Outcomes 1996-1997 (14-month cohort) J6 J5 J6 Not Recommended Recommended Not Recommended Predict FTA Predict no FTA Predict FTA False Positives (FP) True negatives (TN) False Positives (FP) 49 530 109 7.6% 81.7% 10.1% True positives (TP) False negatives (FN) True positives (TP) 20 50 23 3.1% 7.7% 2.1% 132 580 69 12.2% 89.4% 10.6%

FY1992 Actual Outcome No FTA

FTA

Totals Total Correct predictions Total Incorrect predictions Sensitivity Specificity RIOC Base for %

J5 Recommended Predict no FTA True negatives (TN) 881 81.4% False negatives (FN) 69 6.4% 950 87.8% TN + TP = 904 (83.5%)

TN + TP = 550 (84.7%)

FN + FP = 178 (16.5%)

FN + FP = 99 (15.3%)

25.0% 89.0% 14.1% N = 1082 at-risk cases (114 missing data on criterion variables were excluded from the at-risk sample, N=1196)

28.6% 91.5% 20.1% N = 649 at-risk cases (56 missing JO recommendation category, or with no recommendation due to exclusionary factors, were omitted from the at-risk sample, N=705)

- 58 -

ROR-Recommendation System for Juvenile Offenders, Part I: Validation Study

False positives and false negatives were more evenly distributed in the validation sample (the 1996-1997 cohort) than in FY 1992, but the differences are small. For the FY 1992 cohort, the proportion of juveniles who were predicted to fail but did not fail (false positives, 10%) was higher than the proportion who were predicted not to fail but did fail (false negatives, 6%). For the 1996-1997 arrests, false positives and false negatives were both about 8 percent. This indicates that the system provides a fairly even balance between the two types of prediction error. Three other measures of the ROR-recommendation system’s predictive power are given at the bottom of the table: sensitivity, specificity, and relative improvement over chance (RIOC). These measures were all calculated from the data in the cells of the classification tables. Sensitivity is the percent of failures correctly predicted ( non-failures correctly predicted

(

TN ). TN + FP

TP FN + TP

). Specificity is the percent of

The RIOC score is a measure of the model’s

improvement over random classification, relative to the greatest improvement over random classification that would be possible given the FTA rate and the proportion of juveniles who were recommended. The RIOC score can range from 0 percent relative improvement to 100 percent relative improvement (see Appendix E for the calculation and interpretation of the RIOC score). On all three measures higher percentages indicate better predictive power. The ROR-recommendation system scored slightly higher for the 1996-1997 cohort on each of these measures.

Deleting the policy exclusions — which were not excluded from the

calculations for the FY 1992 cases because the full recommendation system was not applied to them — may have raised the predictive efficiency of the system for the validation sample, for reasons that are unclear. The ROR-recommendation system cannot accurately predict which specific individuals will

- 59 -

ROR-Recommendation System for Juvenile Offenders, Part I: Validation Study

fail to appear. It merely identifies individuals who belong to a group with a higher likelihood of failure than others. A juvenile who was not recommended for ROR had only a 29 percent likelihood of failure, but because that likelihood was so much higher than the predicted failure rate for someone who was recommended for ROR, the recommendation was not to release. Therefore, it is useful conceptually to treat the “not recommended” category as a prediction of failure, even though one should keep in mind that it is really only a prediction of a higher likelihood of failure than would be predicted for a recommended defendant.16 The classification table is thus a measure of the extent to which the ROR-recommendation system falls short of a 100-percent FTA rate for not-recommended JOs, and a zero FTA rate for recommended JOs.

Or, one could say that it is a measure of the extent to which the

recommendation system falls short of predicting which specific individuals will fail. The system did well on all these measures, but it predicted non-failure (specificity: 92%) much better than failure (sensitivity: 29%). That is because most released JOs did not fail, making a prediction of non-failure the most likely occurrence regardless of recommendation. Even so, the RIOC score indicates that there was an improvement over predictions made by chance, and that the amount of improvement was 20 percent of the maximum improvement that would be possible (given the FTA rate and the number of juveniles predicted to fail).

For this reason, the classification tables presented here are calculated differently from those given in the computer output from the logistic regression models discussed below. SPSS, the software used to do the statistical analyses, counts only a probability of greater than 50% as a prediction of failure. Therefore, the computer-generated classification tables showed that no one was predicted to fail, and the percent correct predictions was 89% with no false positives. Based on the way the recommendation system is designed to be used, however, the classification tables presented in this report give a more realistic assessment of the prediction/outcome relationship. Visher et al. (1991) recommend the kind of classification analysis used in this report because of the practical requirements of a risk-assessment instrument, “which must frequently ignore the probabilistic nature of its statements and act as if it was answering yes or no to the question of a subject’s future failure.” (p. 345)

- 60 -

ROR-Recommendation System for Juvenile Offenders, Part I: Validation Study

Logistic Regression Model The final analysis done to validate the ROR-recommendation system was to estimate a logistic regression model in which the dependent variable was failure to appear, and the independent variable was the recommendation category.

Logistic regression is a type of

statistical analysis most appropriate for dichotomous dependent variables such as FTA, where the only values are “yes” and “no.” In the model shown in Table 15, the dependent variable (FTA) was coded “0” for “no” and “1” for “yes,” where “yes” indicates that there was at least one FTA for the case, and “no” indicates there was none. The model is statistically significant at the .0001 level, which means that the probability is less than one in 1,000 that the relationship between recommendation category and FTA could have occurred by chance. That statistic is reported in the bottom row of the table, as the significance level of the model chi-square. (See Appendix C for a discussion of the meaning of significance tests when the data set is not a random sample.) Contrast coding was used because the independent variable was categorical rather than continuous. This technique enters each category of a variable in a logistic regression equation as though the categories comprised separate dichotomous variables.

Although only a single

variable (recommendation) was entered as the independent variable, with contrast coding each category appears in the table with a coefficient, significance level, and odds ratio. These statistics are all relative to the “reference category,” which in this instance is J5, “recommended.” This means that the FTA rate for each of the other categories was compared statistically to the FTA rate for recommended JOs. There is no logit coefficient, significance level, or odds ratio for the reference category because it cannot be compared to itself. A disadvantage of logistic regression is that the coefficient (or “logit”) is difficult to

- 61 -

ROR-Recommendation System for Juvenile Offenders, Part I: Validation Study

New York City Criminal Justice Agency Table 15 Logistic Regression Analysis Predicting Failure to Appear1 Application of the JO Recommendation System to 1996-1997 JO Arrests (14 Months) AT-RISK ONLY N=6602 Independent Variable JO ROR RECOMMENDATION J5 Recommended (Reference Category) n=580 J6 Not Recommended n=69 J7A No Recommendation because of: Bench Warrant n=2 J7B No Recommendation because of: No NYSID n=2 J7C No Recommendation because of: Murder charge n=6 J7D No Recommendation because of: Incomplete interview n=1 Constant

Logit Coefficient

Significance Level

Odds Ratio

—

1.465

.0000

4.327

2.361

.0969

10.600

2.361

.0969

10.600

-4.841

.7463

.008

-4.841

.8949

.008

-2.361 .0001

model chi-square=25.590

Dependent variable = Failure to Appear 0 = no 1 = yes A positive logit coefficient and an odds ratio greater than 1.0 indicate that the recommendation category is associated with a higher FTA rate than the reference category (“recommended”). A negative logit coefficient and an odds ratio less than 1.0 indicate that the recommendation category is associated with a lower FTA rate than the reference category. A significance level less than .05 is considered statistically significant.

1 2

FTA was counted up to the sentencing appearance (or to final disposition, if no conviction). 45 cases were excluded from the analysis because of missing data.

- 62 -

ROR-Recommendation System for Juvenile Offenders, Part I: Validation Study

interpret. Only the signs of the coefficients have a meaning that is readily understood. A positive coefficient indicates a positive relationship to the dependent variable, and a negative coefficient indicates a negative relationship.

Accordingly, the two negative coefficients in the

table indicate that the FTA rate was lower (compared to the rate for recommended JOs) for those assigned a J7C category (No recommendation because of murder charge) or a J7D category (No recommendation because of incomplete interview). The other three coefficients are positive numbers, indicating that JOs in those categories had FTA rates that were higher than recommended JOs. The next column presents the significance level for each logit coefficient. The smaller the significance level the better: the larger the number, the larger the probability that the effect was merely a chance occurrence. By convention .05 is usually the largest probability level that is considered statistically significant. The only significance level that meets the .05 criterion in this table is for the J6 category (Not recommended), which is significant at the .0000 level. So, the large difference in FTA rates between “recommended” and “not recommended” that has been reported repeatedly in this paper has now been shown to be statistically significant. The other recommendation categories contained so few defendants that it would be virtually impossible for their coefficients to be statistically significant, since statistical significance is partly a function of the number of cases. The number of cases in each category is given in the left-hand column under the category designations. “J7A” (No recommendation because of bench warrant) and “J7B” (No recommendation because of no NYSID) each had only two cases; “J7D” (No recommendation because of incomplete interview) had one case; and “J7C” (No recommendation because of murder charge) had six. The total of seven cases in the J7C and J7D categories had no failures to appear among them, which is why their FTA rates

- 63 -

ROR-Recommendation System for Juvenile Offenders, Part I: Validation Study

were lower than the recommended (J75) category, as indicated by their negative coefficients. But the negative relationship was not statistically significant; with so few cases, it is meaningless. The final column gives an odds ratio for each category of the recommendation system. The odds ratio represents the increase or decrease in odds of a failure to appear for each recommendation category, compared to the reference category. The odds ratio is derived from the logit coefficient (it is the antilog of the logit), but unlike the coefficient, odds ratios can be more easily interpreted. The odds ratio for the J6 (Not recommended) category was 4.3; this means that the odds of a JO failing to appear were 4.3 times greater if he or she had not been recommended for ROR (compared to the odds for a JO who was recommended). Data presented earlier showed that a JO who was not recommended was about three times more likely to FTA than one who was recommended. This may sound inconsistent with the previous paragraph — which is it, three times or four times more likely to FTA? — but it is not. The earlier sections reported probabilities, which should not be confused with odds. Divide the likelihood (probability) of FTA for the not-recommended group (29%) by the likelihood of FTA for the recommended group (9%), and you will obtain the result that the likelihood of FTA for the not-recommended group is 3.2 times greater.

The odds of failure to appear for the not-

recommended group were 29 to 71 (i.e., out of every 100 JOs, 29 had an FTA and the remaining 71 did not), or .4. The odds of FTA for the recommended group were 9 to 91, or .1. Thus, the odds increased from .1 to .4, or by a factor of 4 (rounded off) while the probability increased from .1 to .3, or by a factor of 3 (rounded off). These are two different ways of expressing the same mathematical relationship. Note that the odds ratios for the “no recommendation” categories J7A and J7B were 10.6, which indicate a huge increase in the odds of FTA for these two groups, compared to the

- 64 -

ROR-Recommendation System for Juvenile Offenders, Part I: Validation Study

recommended category. However, these relationships were not statistically significant; they resulted from the fact that in each category one of the two defendants failed to appear, producing an improbably large FTA rate of 50 percent for each group.

With only a few additional

defendants in either group, that percentage would change drastically. Note also that the odds ratios for the other â&#x20AC;&#x153;no recommendationâ&#x20AC;? categories, J7C and J7D, were less than 1.0, which indicates that the odds of FTA decreased for defendants in those categories, relative to recommended defendants. This is consistent with the negative coefficients. However, as has already been pointed out, those relationships also were not statistically significant. In conclusion, the ROR-recommendation system for juvenile offenders was validated using a variety of measures to test the systemâ&#x20AC;&#x2122;s ability to predict failure to appear for a cohort of juveniles arrested in 1996 and 1997. By every measure, the recommendation system performed as well on the validation cohort as it did on the cohort of FY 1992 arrests used to develop the system. The probability of failing to appear was over three times greater (and the odds of failing were over four times greater) for a juvenile who was not recommended for ROR than for a juvenile who was recommended.

- 65 -

ROR-Recommendation System for Juvenile Offenders, Part I: Validation Study

VI. DISCUSSION Data presented in this report have shown that the ROR-recommendation system successfully identified a small percentage of juveniles arrested in 1996 and 1997 who were at higher risk of failure to appear than others, and that the rest â&#x20AC;&#x201D; well over 80 percent â&#x20AC;&#x201D; were at very low risk of failure.

However, the recommendation system does not appear to have

influenced judges to release more low-risk juveniles on recognizance at arraignment. It has been noted that while 52 percent of juveniles were released on recognizance at arraignment in FY 1992, prior to implementation of the recommendation system, that proportion dropped to 38 percent in FY 1997, after implementation. This is cause for concern, not only because some juveniles may be detained at arraignment who could be released without much fear that they would not return, but also because detention in Criminal Court has implications for what happens later during processing of the case. The Significance of Release in Criminal Court The Criminal Court release decision is strongly associated with both prosecution in Supreme Court and with release in Supreme Court. Figure 5 illustrates this relationship. The square-cornered, unshaded boxes on the left divide the 1996-1997 cohort into cases that ended in Criminal Court (top, n=556) and cases that were transferred to Supreme Court (bottom, n=432). The shaded boxes break down the Supreme Court release decision by what happened in Criminal Court. The unshaded, round-cornered boxes on the right represent the defendants who were released at any point in either court prior to sentencing and thus were included in the at-risk sample. Cases that ended in Criminal Court (top left) began as Juvenile Offender cases, but they were quickly dismissed or transferred to Family Court as Juvenile Delinquents. (The breakdown

- 66 -

ROR-Recommendation System for Juvenile Offenders, Part I: Validation Study

New York City Criminal Justice Agency Figure 5 Likelihood of Release Prior to Sentencing Shown Separately for Criminal Court and Supreme Court Cases Cases with appearances in Criminal Court only (no transfer to Supreme Court) n=556

Ever at risk: n=420 (77%) base=543 No opportunity for release: Dismissed or transferred to Family Court at arraignment n=13

Detained through Criminal Court outcome n=123 (23%) base=543

Released prior to at least one Supreme Court appearance following the first n=129 (93% of those released in Criminal Court; base = 138)

No opportunity for release in Supreme Court: Transferred to Family Court at the first Supreme Court appearance (n=2); or SC appearance history missing (n=1)

Cases that were transferred to Supreme Court n=432

Total ever at risk n=705 (72%) base=975

Ever at risk: n=285 (66%) base=432

Released in Criminal Court n=141 (33%)

Not released in Criminal Court n=291 (67%)

(Released prior to at least one Criminal Court appearance)

Released prior to at least one Supreme Court appearance following the first n=144 (51% of those not released in Criminal Court; base=285) [n=72 released pre-disposition; 25% of 285]

No opportunity for release in Supreme Court: Transferred to Family Court or sentenced at the first Supreme Court appearance (n=2); or SC appearance history missing (n=4)

Detained through Supreme Court outcome n=141 (49%) base=285

- 67 -

(Released prior to at least one Criminal Court or Supreme Court appearance) [n=213 (49%) released prior to final disposition]

ROR-Recommendation System for Juvenile Offenders, Part I: Validation Study

of dispositions for cases that ended in Criminal Court was given in Figure 2.) The release rate for these defendants (follow the arrowà all the way to the right) was high: 77 percent. These 420 cases, in which the defendant was at risk in Criminal Court and which were not transferred to Supreme Court, constitute over half of the entire at-risk sample of 705 cases. Release in Criminal Court, then, was strongly associated with a decision not to prosecute as a JO. Conversely, detention in Criminal Court was strongly associated with transfer to Supreme Court: 25 percent of released JOs, as opposed to 70 percent of detained JOs, were prosecuted in Supreme Court. (These percentages are not given in Figure 5, but can be calculated from the numbers in the boxes.) The release rate for Supreme Court cases was lower than for cases that ended in Criminal Court. For all JO cases prosecuted in Supreme Court, the likelihood of being released prior to sentencing was 66 percent (“Ever at risk,” lower right).

If post-disposition releases are

discounted, the rate of release in either court for Supreme Court cases was only 49 percent. However, this overall likelihood was greatly affected by whether the defendant had been released in Criminal Court. The square-cornered box at the bottom left representing Supreme Court cases is subdivided into those who were released in Criminal Court and those who were not (shaded, round-cornered boxes). This breakdown shows that two-thirds of the defendants whose cases were transferred to Supreme Court had never been released in Criminal Court, and the right-hand shaded boxes show that these defendants were also much less likely to be released in Supreme Court. A JO who had been released in Criminal Court was almost certain to be released in Supreme Court: 93 percent of them were. For a JO who was never released in Criminal Court — the majority — those chances were cut almost in half, to 51 percent. Furthermore, many of those releases did not

- 68 -

ROR-Recommendation System for Juvenile Offenders, Part I: Validation Study

occur until the defendant pled guilty (often upon entering an alternative-to-incarceration program). If post-disposition releases are discounted, the release rate for those who had not been released in Criminal Court was only 25 percent. Further research would be required to determine if being detained in Criminal Court had any independent causal effect on either prosecution in Supreme Court or on the release decision once the case was transferred. Presumably many of the same considerations influence the release decision by both courts, and some of these factors — such as the seriousness of the charges and the strength of the evidence — also affect whether a case is transferred to Supreme Court. These data merely establish that there is a positive relationship between detention in Criminal Court and prosecution in Supreme Court, and that a juvenile detained in Criminal Court is likely to remain in custody until the case is disposed. The Problem of Bias Introduced by Using a Sample of Released Defendants This research was done using a subset of arrests that included only juveniles who had been released at some point prior to sentencing, and who were thus at risk for failure to appear. The rationale for this research design was discussed under “Creation of the At-Risk Sample” in section II. As explained there, comparing FTA rates for recommended versus not-recommended JOs is meaningful only if every defendant in the analysis had an opportunity to fail to appear for at least one scheduled court date. It would not mean anything to find a low FTA rate if the cases included defendants who were in detention until sentencing. Selecting only released defendants for the analysis was therefore necessary, but it raises questions about how well the results of the analysis would apply to detained defendants. Over half of the defendants who were recommended for ROR in this study were detained at arraignment; about a quarter were never at risk. If they had been released, would their FTA rate

- 69 -

ROR-Recommendation System for Juvenile Offenders, Part I: Validation Study

have been as low as the rate for recommended JOs who were released? This is an important question because of the hope that judges will use the recommendation system as a guide for releasing more defendants.

Presumably, the reason that they were not released was that

arraignment judges considered them to be poor risks in spite of the CJA recommendation. An ideal way to test this would be experimentally: release every JO at arraignment for a specified period of time, thus eliminating all sample bias because every defendant could be included in the analytic sample (except for a few cases disposed at arraignment). While such a research design would be the most straightforward way to eliminate the possibility of sample bias, it would result in the release of many high-risk juveniles, and it is hardly a feasible option. However, there are statistical ways to test for sample bias. For an analysis of adult cases undertaken as part of a CJA project to revise the ROR-recommendation system for adults, Qudsia Siddiqi tested for bias introduced by using a sample of released defendants and defendants whose cases were not disposed at arraignment. Preliminary results show that sample selection bias resulting from these two factors did not appear to be a problem in predicting FTA for an at-risk sample of adults (Siddiqi, 1999, Appendix).17 In Part II of this research, we will address the problem of sample selection bias by analyzing possible differences between detained and released JOs who were recommended for ROR. All JOs who were recommended for ROR shared the same community-ties characteristics whether they were detained or released. The detention of so many recommended juvenile offenders is a clear indication that considerations other than the community ties upon which the recommendation is based were factors in the courtsâ&#x20AC;&#x2122; decisions. The task will be to attempt to

Possible sample selection bias introduced by cases disposed at Criminal Court arraignment is much less of a problem for JOs than it is for adults because a true JO case must be disposed in Supreme Court. Only 13 cases (1%) of the 1996-1997 JO cohort were disposed at Criminal Court arraignment.

- 70 -

ROR-Recommendation System for Juvenile Offenders, Part I: Validation Study

identify factors that led to the detention of recommended JOs, and to assess the relationship of those factors to FTA. If it turns out that JOs who were recommended for ROR and detained at arraignment were no different — in any way relevant to failure to appear — from recommended JOs who were released, then we will be able to conclude that the risk of failure for the recommended detainees was no greater than for their counterparts who were released. Meanwhile, the preliminary finding that no significant bias was introduced in predicting FTA for adults by using a sample of released defendants increases our confidence in the present analysis for juvenile offenders. Could the Courts Do Just As Well Without the JO Recommendation System? It may be somewhat disconcerting to note that, prior to implementation of the recommendation system, the courts made an “incorrect prediction” only 8 percent of the time. This might be the conclusion drawn from the observation that the FTA rate for the at-risk FY 1992 sample was 8 percent. Everyone who was released was presumably a good risk in the eyes of the courts, and this judgement turned out to be incorrect for only the 8 percent who failed. If the courts were making the correct release decision in the remaining 92 percent of the cases, this appears to indicate that they were doing better without the CJA recommendation. Classification tables presented earlier in this report showed that the CJA recommendation was correct 84 percent of the time for the FY 1992 arrests. (The corresponding figures for 1996-1997 are 89 percent correct predictions made by the courts — based on the 11 percent FTA rate — and 85 percent correct predictions made by the recommendation system.)

Why then have a JO

recommendation system at all? The fact is that the courts really did not make correct predictions 92 percent (or 89 percent) of the time. Those figures count only the false negatives as incorrect predictions (predicted not

- 71 -

ROR-Recommendation System for Juvenile Offenders, Part I: Validation Study

to fail, but failed). However, the accuracy rates calculated for the JO recommendation system also take false positives into account (predicted to fail, but did not fail). For every juvenile who is detained, the (implicit) prediction is that he or she would fail to appear if given an opportunity. We cannot know how many false positives — juveniles in detention who would not have failed to appear — may be hidden in the courts’ decisions. If we counted only false negatives in calculating the accuracy rate of the JO recommendation system (8% for the 1996-1997 cohort, 6% for FY 1992), we would obtain a higher proportion of correct predictions — 92 percent for the 1996-1997 cohort, and 94 percent for FY 1992. Note that higher detention rates mean less opportunity for failure, as well as (perhaps) less failure. False negatives (failures) could be eliminated entirely simply by detaining everyone, but this would vastly increase the number of false positives. Under that scenario, there would be no failures but there would be many detained juveniles who would not have failed if they had been released. The ROR-recommendation system provides an objective, statistically validated method for balancing the costs of releasing some JOs who then fail to appear against the costs of detaining some who would not have failed.

With the recommendation system, it may well be

possible to increase the release rate without raising the FTA rate, and thereby lower the number of detained juveniles who could have been released to their parents’ or guardians’ custody. For this reason, we believe the recommendation system provides the courts with a valuable tool to assist them in judging risk of failure to appear — one that the courts could not do just as well without.

- 72 -

ROR-Recommendation System for Juvenile Offenders, Part I: Validation Study

Plans for Further Analysis Although the ROR-recommendation system for juveniles has been successfully validated, further research is needed to gain a better understanding of factors affecting release and failure to appear. A second report will be issued in order to address these closely related issues. Some of the topics to be covered in Part II will include: 路

analysis of FTA through final disposition rather than through sentencing, to find out if the different circumstances of post-conviction release necessitate different predictive models for pre- and post-disposition FTA;

路

consideration of issues of timing, including time at risk, the timing of the first release, the timing of the first failure to appear; and the length of time it took juveniles to return to court after issuance of a bench warrant;

路

examination of the effect of bail amount on length of time in detention and eventual release;

路

analysis of factors other than CJA recommendation that may affect release, and the relationship between those factors and FTA. Following the completion of Part II of this study, CJA hopes to follow up by initiating

discussions with judges to improve our understanding of how they use the CJA recommendation in making their release decisions, and what factors are contributing to the higher detention rates for juveniles that have been observed since the mid-1990s.

- 73 -

ROR-Recommendation System for Juvenile Offenders, Part I: Validation Study

APPENDIX A JUVENILE OFFENSES Offense

Penal Law

Aggravated sexual abuse in the first degree Arson in the first degree Arson in the second degree Assault in the first degree Burglary in the first degree Burglary in the second degree Kidnapping in the first degree Attempted kidnapping in the first degree Manslaughter in the first degree Murder in the second degree Attempted murder in the second degree Rape in the first degree Robbery in the first degree Robbery in the second degree Sodomy in the first degree

130.70 150.20 150.15 120.10 (1) (2) 140.30 140.25 (1) 135.25 110/135.25 125.20 125.25 (1) (2) 125.25 (3)** 110/125.25 130.35 (1) (2) 160.15 160.10 (2) 130.50 (1) (2)

Felony Class B A B B* B C A B B A A B B B C B

*Assault in the first degree was upgraded from Class-C to Class-B felony in November 1996. **But only where the underlying crime is also a JO offense.

- 74 -

Defendantâ&#x20AC;&#x2122;s Age 14, 15 14, 15 14, 15 14, 15 14, 15 14, 15 14, 15 14, 15 14, 15 13, 14, 15 14, 15 14, 15 14, 15 14, 15 14, 15 14, 15

ROR-Recommendation System for Juvenile Offenders, Part I: Validation Study

APPENDIX B

New York City Criminal Justice Agency PROCEDURAL MANUAL FOR JUVENILE OFFENDERS

Planning Department Aida Tejaratchi October 1995

- 75 -

ROR-Recommendation System for Juvenile Offenders, Part I: Validation Study

APPENDIX C Use of Tests of Statistical Significance in Cohort Studies The use of tests of statistical significance for some of the analyses in this report may be questioned on the grounds that the data set was not a random sample. The data set included every arrest of a juvenile offender during a 14-month period during 1996 and 1997 that resulted in a Criminal Court arraignment; this is a cohort rather than a sample, random or otherwise. Tests of statistical significance are used to measure the likelihood that the results found in a sample are actually representative of the population from which the sample was drawn. The conventional significance level of .05 was used here, which is interpreted to mean that there is a less than 5 percent chance that the â&#x20AC;&#x153;statistically significantâ&#x20AC;? effect found in the sample data would have been found if there were really no such effect in the population. If we had drawn a sample of JOs arrested during the 14-month time frame of the study, rather than including every one, we could interpret statistically significant results from our sample to mean that there is a better than 95% chance that those results are representative of all JOs arrested (and arraigned in Criminal Court) during that time. When the data set is a cohort or some other set that includes every member rather than a true sample, there is some ambiguity surrounding the interpretation of statistical significance. Some statisticians argue that statistical inference is irrelevant, and that only descriptive statistics should be employed for such data sets. According to this position, the data should be treated as a population; there is no need for statistical inference because the population parameters are known. However, social scientists routinely use inferential statistics to analyze data from groups that are not random samples. Cohort studies, cross-national studies employing data from sets of

- 86 -

ROR-Recommendation System for Juvenile Offenders, Part I: Validation Study

countries studied at one point in time, and urban studies that collect data from all American cities with populations of a certain size are examples. This approach treats the data set as though it were a random sample, even though it is not. The legitimacy of doing this is a matter of debate, but not doing it leaves researchers vulnerable to the criticism that an apparent effect may have no “statistical significance” — whatever that may mean for data that do not constitute a sample. The method used for most of the analysis in this report is crosstabulation, and the results are presented in the form of tables without any tests of statistical significance because we can be certain, without a significance test, that the results are representative of the cohort. That is to say, the findings could not be the result of sampling error because we did not draw a sample. Nevertheless, we reported the significance level for the logistic regression (Table 15) in order to establish that the model is statistically significant, whether one considers this demonstration necessary or not. Reporting the significance level for each coefficient also serves a heuristic function in showing that categories with very small numbers of cases do not produce statistically significant coefficients — which is the same point made numerous times throughout this report that percentages based on table cells with very small numbers of cases are meaningless. The attraction of significance tests for the present research, and for cohorts in general, is that one would like to be able to generalize beyond the specific population that was studied. Often a statistically significant finding for a cohort is interpreted as evidence that the finding can be generalized to other cohorts, although it is unclear how wide the circle of generalizability might be. Sometimes the wider circle is conceptualized as a “super population,” and the cohort as the equivalent of a random sample drawn from this super population.1

For a discussion of this issue see Berk, et al., 1995, pp. 421-458. The authors are not proponents of this approach. Rather, they believe that neither dispensing with statistical inference nor treating the data set as a random sample is completely satisfactory for “apparent populations,” which they define in a way that could include cohorts. Instead,

- 87 -

ROR-Recommendation System for Juvenile Offenders, Part I: Validation Study

By this rationale, the tests of statistical significance presented in this report can be interpreted as estimates of the likelihood that the social processes that produced these results would produce the same results in any other cohort drawn from a “super population” of JOs arrested and arraigned in New York City. The inference does not allow us to generalize beyond a population of New York City JOs arraigned in Criminal Court, but it does allow us to generalize beyond the point in time when this “sample” was drawn. The crucial assumption is that the “social processes” do not change from one point in time to another — an assumption that may be questionable, but is more likely to be valid for time periods that are close together than for those that are widely separated. An alternative (or an adjunct) to using statistical inference for cohorts is to attempt to replicate the analysis for other time points.2 In fact, this study does replicate the analysis of the FY 1992 data set that was done in developing the ROR-recommendation system for JOs. Findings for one cohort that are replicated for the other provide mutually supporting evidence that neither cohort was an unrepresentative sample of juvenile offenders in New York City.

they propose using a method known as Bayesian analysis for such data sets. However, they concede that “for many applications, the numerical conclusions from Bayesian analysis will be identical, or nearly identical,” to conventional statistical inference. 2 This is the method advocated by Kenneth A. Bollen (1995) in his response to Berk, et al.

- 88 -

ROR-Recommendation System for Juvenile Offenders, Part I: Validation Study

APPENDIX D Procedures for Checking Data Errors Attempts were made to verify the JO status of any defendant with discrepancies regarding date of birth or CJA recommendation: those who seemed to meet the age and charge criteria for JO status, but who were assigned an adult recommendation category; and those who were recommended as JOs but apparently did not meet a criterion for age or charge. Sources used for checking such discrepancies included computerized data from the Office of Court Administration (OCA), other cases for the same defendant in the CJA database; and the CJA Interview Report. Court calendars were also occasionally useful in confirming JO status if, for example, there was a calendar notation such as â&#x20AC;&#x153;JVFOâ&#x20AC;? (Juvenile Violent Felony Offender). In deciding whether to keep a questionable case in the file, we considered the weight of the evidence from all of these sources. If we found confirmation for a date of birth that would mean that the defendant was not within the age range for a JO, the case was eliminated from the analysis even if a JO recommendation stamp had been issued.1 On the other hand, if we could confirm a JO-eligible date of birth, we retained the case even if the arrest charge was not a JO offense, or if the CJA recommendation was something other than a juvenile category. Once case selection was final, we carefully examined the court appearance histories of the 988 cases in the file, focusing especially on release status. Release status throughout court processing prior to the sentencing appearance was of paramount importance to this research because a defendant was at risk for failure to appear only if he or she was released from custody

JO status is based on age at the time of the offense, not at arrest. The date of the offense was not available for this research, so it is possible that some defendants who were 16 or older at arrest were in fact true JOs because the offense had been committed earlier. Excluding them might have resulted in a slight undercount of the number of JOs, but the exclusion is appropriate for a study attempting to validate the JO recommendation system. The JO recommendation system was designed to apply to 13-, 14-, and 15-year olds, so it is unclear if the system would work well for older youths.

- 89 -

ROR-Recommendation System for Juvenile Offenders, Part I: Validation Study

prior to at least one scheduled court appearance. We used Office of Court Administration (OCA) electronic records and paper court calendars to check release status, and we also used supplemental computerized data provided by the Department of Juvenile Justice (DJJ) to track bail-making at DJJ’s juvenile detention centers. If a defendant posted bail and was released from DJJ custody in the time between two court dates, we counted him as at risk for the second appearance even though he was in custody leaving the first appearance. Failure to appear for a scheduled court date triggers the issuance of a bench warrant for the defendant’s arrest. Consequently, we used a disposition of “warrant ordered” as the indicator of FTA, and we checked the release status prior to the first warrant ordered in each case. (Stayed warrants were not counted.) If the defendant was in custody prior to the supposed warrant, and if we could find no record of a bail-made date from DJJ that would explain how the defendant came to be released, then we did not count the warrant as a failure to appear. Conversely, we checked each “order to produce” to ascertain that the defendant was in fact in custody prior to the appearance. An order to produce indicates that the defendant did not appear for the scheduled court date, but the responsibility for the failure to appear lies not with the defendant but with the agency in whose custody he was being held — for a juvenile, DJJ. We did not count an order to produce as a failure to appear unless the defendant had been released prior to that court appearance. We also searched for other open cases for some defendants in the data file. A defendant who had been released on the sample case — and was thus apparently at risk for the next scheduled court appearance — would actually not be at risk if he or she had been rearrested in the interim, and was in custody on the rearrest at the time of the next appearance on the sample case. It was not feasible to check other open cases for every defendant in the data file because of

- 90 -

ROR-Recommendation System for Juvenile Offenders, Part I: Validation Study

the enormous time and effort involved, but we did check every case with a bench warrant on any court date in order to avoid over-counting failures to appear. If a defendant was found to have been in custody on another case at the time a warrant was ordered, the event was not counted as a failure to appear. The error checks carried out on this data set were much more extensive than what was done on the FY 1992 data set used to develop the JO recommendation system. The 1996-1997 data set was compiled specifically for the purposes of this research, allowing us to tailor it to our needs, whereas the FY 1992 data set had been compiled for broader purposes. Some differences in the findings for the two data sets may be attributable to the different data-checking procedures.

- 91 -

ROR-Recommendation System for Juvenile Offenders, Part I: Validation Study

Appendix E Calculation and Interpretation of Relative Improvement Over Chance (RIOC) Scores1 The Relative Improvement Over Chance (RIOC) score is a measurement of the improvement of a statistical model’s classification over chance, relative to the maximum improvement over chance that would be possible given the base rate and the selection ratio. It is calculated by dividing the actual improvement over chance by the maximum possible improvement over chance. The score can range from 0 (or 0%), indicating no relative improvement, to 1 (or 100%), indicating maximum possible improvement. In this study, the base rate refers to the observed FTA rate, and the selection ratio refers to the percent predicted to fail. Although the statistical model predicts only a probability (29%) that a juvenile who is not recommended will fail to appear, the practical requirements of the riskassessment instrument necessitate assigning individuals to the “recommended” and “not recommended” categories in a deterministic fashion. Thus, “recommended” is treated as a prediction of non-failure, and “not recommended” is treated as a prediction of failure. The issue is how well these assignments accord with observed outcomes, and to what extent they represent an improvement over predictions generated randomly (Visher et al., 1991, pp. 344-345). Understanding the Meaning of the RIOC Score The classification table for the 1996-1997 data set (adapted from Table 14) is given below so that the reader can follow the discussion more easily. It shows that the proportion of correct predictions made by the JO recommendation system was 84.7 percent. The proportion of correct predictions that would be expected by chance (given the FTA rate and selection ratio) was 80.9

Portions of this description of the RIOC score, and the formula for calculating the score, were taken from Loeber and Dishion (1983) as adapted by Siddiqi (1999, Appendix). - 92 -

ROR-Recommendation System for Juvenile Offenders, Part I: Validation Study

percent (not shown in the classification table). The actual improvement over chance, then, was the difference between 84.7 and 80.9, or 3.8 percentage points. (The calculations for the percent expected by chance and other statistics mentioned in the following paragraphs are shown at the end of this appendix.) Classification Table for 1996-1997 JO Cohort Predicted Outcome J6 Not Recommended Predict FTA J5 Recommended Predict no FTA Total observed

FTA True positives (TP) 20 3.1% False negatives (FN) 50 7.7% 70 10.8%

Total Correct predictions Total Incorrect predictions Sensitivity Specificity RIOC

TN + TP = 550 (84.7%)

Observed Outcome No FTA False Positives (FP) 49 7.6% True negatives (TN) 530 81.7% 579 89.2%

Total predicted 69 10.6% 580 89.4% 649 100%

FN + FP = 99 (15.3%) 28.6% 91.5% 20.1%

Within the constraints of the number of juveniles who failed (70) and the number of juveniles predicted to fail (69), the most efficient prediction would result from a scenario in which the 70 who were observed to fail included all 69 of those who were not recommended (predicted to fail). This would reduce the number of false positives (predicted to fail but did not fail) to zero (from the actual number, 49), but would still leave one false negative (predicted not to fail but did fail), because there was one fewer prediction of FTA than there were actual FTAs. This one incorrect prediction could not be eliminated given the total number of FTAs and the total number of juveniles not recommended for ROR. This best-case scenario would result in correct predictions for 99.8 percent of the cases. Therefore, the maximum correct prediction would rep-

- 93 -

ROR-Recommendation System for Juvenile Offenders, Part I: Validation Study

resent an improvement over chance of 18.9 percentage points (the difference between 99.8 and 80.9). As stated, the RIOC score represents the actual improvement over chance (3.8 percentage points) as a proportion of the maximum possible improvement over chance (18.9 percentage points). The RIOC score is thus 20.1 percent (3.8 divided by 18.9). This means that the predictions made by the ROR-recommendation system increased the number of correct predictions over chance by about 20 percent of the maximum possible increase, within the constraints of the existing FTA rate and selection ratio. The RIOC score is Relative because a small absolute increase over chance may result in a large RIOC score if the maximum possible increase is also small. In this study an absolute increase over chance of 18.9 percentage points would have been sufficient to produce a RIOC score of 100 percent, because 18.9 was the maximum possible, given the base rate and selection ratio. Conversely, the 3.8-percentage-point absolute increase over chance found in this research would have resulted in a RIOC score much smaller than the 20 percent that was actually found if the maximum possible increase had been larger. For example, if it had been possible to increase the correct predictions by 50 percentage points, the RIOC score would have been only 8 percent (3.8 divided by 50). The RIOC score is thus more informative than the absolute increase-over-chance (IOC) measure in that the RIOC, unlike the IOC, does not penalize a model if not much improvement is possible within the constraints of the given FTA rate and selection ratio. In this study, the possibilities for improving over chance were limited because the proportion of correct predictions expected by chance alone was already very high (because of the very low FTA rate, which made it most likely that a prediction of non-failure would be correct for anyone). The RIOC score of 20 - 94 -

ROR-Recommendation System for Juvenile Offenders, Part I: Validation Study

percent for the data in this study thus indicates that the efficiency of the system is much better than the 4 percent absolute increase over chance would seem to imply. The RIOC score should not be the only measure considered in assessing the adequacy of a risk-assessment model because, as pointed out above, it takes the selection ratio as a given. It would be possible to obtain a high RIOC score for a recommendation system that produced a very low proportion of correct predictions if the selection ratio differed greatly from the FTA rate. For example: In a sample of 100, if 50 were predicted to fail and only 10 failed, the selection ratio would be much larger than the FTA rate. If all 10 of the observed FTAs were correctly predicted (not recommended), then the RIOC score would equal 100%, but the total correctly predicted would be only 60%. There would be no false negatives, but the proportion of false positives would be extremely high, 40 percent. In that instance, depending on policy priorities, researchers would probably conclude that the risk assessment instrument was too restrictive. In this research, however, the number predicted to fail (not recommended for ROR) was nearly identical to the number who did fail (69 and 70 respectively, even though they were not all the same individuals), and the proportion predicted correctly was quite high (85%). Furthermore, false positives and false negatives were evenly balanced and very low (both about 8%). In this context, a RIOC score of 20 percent offers the assurance that the model represents a significant improvement over random assignment, while at the same time indicating that the improvement falls far short of the best possible match between predictions and observations.

- 95 -

ROR-Recommendation System for Juvenile Offenders, Part I: Validation Study

Calculating the RIOC Score The formula used for calculating the RIOC score is as follows: RIOC =

IOC % MC - % RC

x100

where RC = random correct (percent expected to be predicted correctly by chance alone) MC = maximum correct (best possible percent predicted correctly) IOC = improvement over chance (the difference between the percent predicted correctly and the percent expected to be predicted correctly by chance alone) Calculation of RC: The percent expected to be correctly predicted by chance alone (RC) is the sum of the true positives (TP) and true negatives (TN) expected to be correctly predicted by chance alone. Expected TP

Expected TN

((Total N observed as failing / Total N) x (Total N predicted as failing / Total N)) x 100 ((Total N observed as not failing / Total N) x (Total N predicted as not failing / Total N)) x 100 Expected TP + Expected TN

Working out the calculations for the validation data set from the classification table produces the following results for RC: The percentage of true positives expected by chance alone: (Expected TP) = ((70 / 649) x (69 / 649)) x 100 = (.108 x .106) x 100 = 1.14 % The percentage of true negatives expected by chance alone: (Expected TN) = ((579 / 649) x (580 / 649)) x 100 = (.892 x .894) x 100 = 79.74 % RC = 1.14 + 79.74 = 80.88% expected to be classified correctly by chance alone Calculation of MC: The maximum correct prediction was calculated by formulating a new 2x2 table. When the number of defendants who were observed as failing to appear is greater than the number who - 96 -

ROR-Recommendation System for Juvenile Offenders, Part I: Validation Study

were predicted to fail, the true negative cell should be used as the starting point for the calculation of MC. Otherwise, the true positive cell should be used as the starting point. For the 19961997 cohort, the number observed as failing was 70, which was greater than the number predicted to fail (69), so the true negative cell was used to begin the calculation of MC. The number of defendants who were observed as not failing was 579, so the maximum number of defendants who could be correctly classified as not failing was 579. This number is entered in the true negatives cell. The row and column totals are fixed, and are entered from the classification table. Predicted Behavior FTA

No FTA

Column Totals (Observed)

TP ?

Observed Behavior FTA No FTA FP ?

FN ?

TN N=579 89.2% N=579 89.2%

N=70 10.8%

Row Totals (Predicted) N=69 10.6% N=580 89.4% N=649 100%

With the true negative cell and the totals filled in, the numbers for the rest of the cells are fully determined. They can then be entered as shown. Predicted Behavior FTA

No FTA

Column Totals (Observed)

Observed Behavior FTA No FTA FP TP 0 N=69 0% 10.6% TN FN N=579 N=1 89.2% 0.2% N=70 N=579 10.8% 89.2%

Row Totals (Predicted) N=69 10.6% N=580 89.4% N=649 100%

The maximum correct prediction can now be calculated by summing the maximum true positives (10.6%) and maximum true negatives (89.2%), so MC = 99.8 percent.

- 97 -

ROR-Recommendation System for Juvenile Offenders, Part I: Validation Study

Calculation of IOC: The IOC statistic is the difference between the percentage of defendants who were classified correctly (84.7%, from the classification table) and the percentage of defendants who could be expected to be classified correctly by chance alone (RC = 80.9%). Therefore, IOC = 3.8. Calculation of RIOC: Recall that the RIOC formula is: RIOC =

IOC % MC - % RC

x100

Substituting the values for IOC, MC, and RC that have now been calculated, the result is: RIOC = 3.8 / (99.8 - 80.9) x 100 = (3.8 / 18.9) x 100 = .201 x 100 = 20.1%

- 98 -

ROR-Recommendation System for Juvenile Offenders, Part I: Validation Study

BIBLIOGRAPHY Berk, Richard A., Bruce Western, and Robert E. Weiss, 1995. “Statistical Inference for Apparent Populations.” In Sociological Methodology, vol. 25, American Sociological Association, Oxford: Basil Blackwell, pp. 421-458. Bollen, Kenneth A., 1995. “Apparent and Nonapparent Significance Tests.” In Sociological Methodology, vol. 25, American Sociological Association, Oxford: Basil Blackwell, pp. 459-468. Loeber, R., and T. Dishion, 1983. “Early Predictors of Male Delinquency: A Review,” Psychological Bulletin, 94 (1):68-99. Metchik, Eric, 1987 (December). Recommending Juvenile Offenders for Pretrial Release, New York City Criminal Justice Agency. National Council on Crime and Delinquency (NCCD), 1998 (January 1). Interim Summary Evaluation Report, “Chapter 4: New York City,” pp. 80-183. New York City Criminal Justice Agency, Semi-Annual Report: Second Half of 1994. Phillips, Mary T., 1997 (October). Implementation of a Release-On-Recognizance Recommendation System for Juvenile Offenders Arraigned in New York City Adult Courts: A Report on Arraignment Outcomes — The First Eight Months. New York City Criminal Justice Agency. Siddiqi, Qudsia, 1999 (forthcoming). Prediction of Pretrial Failure to Appear and Alternative Pretrial Release Risk-Classification Schemes in New York City. New York City Criminal Justice Agency. Visher, Christy A., Pamela K. Lattimore, and Richard L. Linster, 1991. “Predicting the Recidivism of Serious Youthful Offenders Using Survival Models.” Criminology 29(3):329-366. Winterfield, Laura A., 1996 (May). Developing a Release-On-Recognizance System for Juveniles Arraigned in New York City Adult Courts, New York City Criminal Justice Agency.

- 99 -