An Impact Evaluation of a Neighbourhood Crime Prevention Program: Does Safer Commune Make Chileans Safer? Inder J. Ruprah
Inter-American Development Bank Office of Evaluation and Oversight Working Paper: OVE/WP-09/08 November, 2008
Electronic version: http://www.iadb.org/ove/DefaultNoCache.aspx?Action=WUCPublications@ImpactEvaluat ions
Inter-American Development Bank Washington, D.C. Office of Evaluation and Oversight, OVE An Impact Evaluation of a Neighbourhood Crime Prevention Program: Does Safer Commune Make Chileans Safer? Inder J. Ruprah*
* The author is from the Office of Evaluation and Oversight of the Inter-American Development Bank. The author thanks Maria Francisca HenrĂquez, Pavel Luengas and Luis Marcano for their valuable inputs. The findings and interpretations of the author do not necessarily represent the views of the Inter-American Development Bank. The usual disclaimer applies. Correspondence to: Inder Ruprah, e-mail: inderr@iadb.org, Office of Evaluation and Oversight, Inter-American Development Bank, Stop B-760, 1300 New York Avenue, NW, Washington, D.C. 20577.
ABSTRACT Safer Commune is a neighbourhood crime prevention program in Chile. It has failed according to some critics who cite as evidence the rising crime rates and fear of crime in municipalities with the program. This is incorrect. Valid empirical evidence would be the crime rates that would have been observed without the program. Such an impact evaluation – using double difference propensity score method- reveals that the program has reduced high crimes particularly of two types of crimes namely battery and theft. Thus, high crimes would have been 19% higher in the communes without the program; the program has made Chileans safer. Active participation in the program by local residents has reduced insecurity and increased security; it reduced the fear of crime. However, with very low active participation in the program the scale of the effect is low. These positive evaluative findings suggest that an expansion of the program but simultaneously enhancing co-production of order through mechanisms to encourage local resident participation would have high returns.
INTRODUCTION Crime and the fear of crime have become important issues of public concern. Chile is no exception. In response to rising reported crime rates and increased fear of crime the government has initiated a series of policy reforms and introduced a number of crime prevention programs. 1 In this paper we evaluate the effectiveness of a specific program, Safer Commune. The program that began in 2001 originally consisted of a typical project menu of Crime Prevention through Environmental Design program. 2 Such programs aim to reduce crime and fear of crime by attempting to prevent crime opportunities in urban spaces in part by encouraging local community participation in the program. The Safer Commune Program was based on the fact that crime is highly geographically concentrated in a small number of high crime communities. However, within communities most places have little to no crime and most crime is highly concentrated in a relatively small number of places (as predicted by routine activity theory and offender search theory, see Sampson and Wooldridge (1987). This suggests that making changes of opportunity blocking type to these places would prevent criminal events by making crime more difficult and risky hence less rewarding. 3 Another central tenet, common to situational crime prevention programs, was that the effectiveness of the program would depended critically on co-production between public agencies and local organisations to repel disorder and reduce crime and the fear of crime. Specifically, the design feature assumed that the most effective opportunity blocking local projects would best be chosen by empowering the local community leaders to design and implement crime prevention strategies. In addition, based on a merger of social capital theory with collective efficiency, i.e. trust among neighbours, kinship/friendship ties, voluntary associations and neighbourhood activism, the program assumed that direct participation in the program by municipality’s residents would reduce their fear of crime. The program has failed according to some commentators. They cite as evidence of failure the increased crime rates in municipalities with the program. 4 They call for stricter punishment measures framing the discussion in terms of 1 A comprehensive and detailed description and analysis of the government’s policies and programs can be found in: The Report of The Forum of Experts (2004). 2 For individual meta-evaluations of interventions like closed circuit television surveillance, street lighting, neighbourhood watch that are examples of the types of projects financed by Safer Commune see Cambell Collaboration at: www.Cambellcolaboration.org. 3 More recently, since 2005, the program began to recognise that family risk factors have a major influence on crime hence has begun to include family risk mitigation measures in addition to situational risk factors. 4 See for example Libertad y Desarrollo (2005).
1
“prevention” vs. “punishment” as two extremes on a continuum of soft vs. tough response to crime. However, both are crime prevention interventions. Crime prevention is not defined by the type of intervention but by its consequences: the number of criminal events. The efficacy of any crime prevention program has to be judged by if it causes a lower number of crimes than those that would have occurred without the program. Thus, the evidence offered by the critics of the program is not credible evidence against the efficacy of the program. The relevant evidence would be the observed crime rates relative to the crime rates that would have been observed if the program had not existed, i.e. impact estimation. The latter approach attempts to determine the casual effect of a program by estimating the part of the observed change in an outcome of interest that can be attributed to the program. This point has often been made generally regarding evaluations of public programs (see Imbens and Wooldridge 2008) and crime prevention programs specifically (see Sherman et al 1997). However, impact evaluations of crime prevention programs in Latin America are practically zero (see Morrison 2007). There have been a number of evaluations of Safer Commune. However, they have been almost exclusively process evaluations although some have ended by a call for an impact evaluation. This is the task of this paper. As a prelude to the findings of the impact calculations of Safer Commune program, this evaluation reveals that the program has been successful in reducing certain types of reported crimes and has been successful in reducing the fear of crime relative to the counterfactual of no program. The rest of this document is structured as the following. In Section II we discuss the different measures of the problem that the program was designed to tackle and we briefly discuss the policy response and set of specific programs. In the third section we detail the design features and the efficacy of the delivery system of the Safer Commune program. In the fourth section we present the impact calculations of the program. The final section consists of a discussion of the main findings. THE POLICY PROBLEM AND POLICY RESPONSE Problem The evolution of the general policy problem, in the limited sense 5 of the size of the problem, can be gauged from two types of indicators; subjective indicators 5
“Limited” because we do not explore the factors that could account for the level, composition and change in crime. This would take us away from the primary objective of this paper that is to determine the effectiveness of Safer Commune program through impact estimations. However, for the socio-economic determinants of crime see Nuñez et al (2003) and Benavente et al (2006); for
2
(importance placed on the issue and the level of fear of crime); and objective indicators (reported crime or detention rates). All empirical measurements of crime –subjective and objective- of crime show that crime has increased. Since 2001 the percentage of citizens that place crime as one of the three most pressing policy issues has steadily risen (Chart 1). While it was placed as the fourth most important policy issue in 2002, by 2004 and also in 2005 it was considered the most important problem. The relative importance reflects an increased level of the fear of crime; the percentage of habitants with high level of fear has increased by 7 percentage points from 2002 to 2005 (Chart 2).
0
20
16
Percentage of Population who Identify Crime as one of the Three Main Problems
2
14
3
(%)
12 10
1
Ranking
8 6 4
4
Ranking
18
Chart 2: Fear of Crime
Percentage of People - High Level of Fear of Crime (%)
Chart 1: Relative Importance of Crime as a Problem
5 6 7
0
8
19 89 19 94 19 95 19 96 20 00 20 01 20 02 20 03 * 20 04 20 05 * 20 06
2
Source: CEP (see Appendix A)
20 19
Fear
18 17 16 15 14 13 12 11 10 2000
2001
2002
2003
2004
2005
2006
2007
ADIMARK (see Appendix A)
The importance placed on this issue also corresponds to the sharp increase in reported crime rates. In Chart 3 are shown the reported crime rates (index with 2001 set at zero for each type of crime). Except for homicides all other types of crimes have risen. Reported High Crime increased by 145% from 2001 to 2005. By 2005 High Crime rates were 2.565 per 1000 habitants. The largest increases were for larceny and aggravated robbery and the smallest increases were for rape. Homicides fell. Data from victimisation surveys (2003 and 2005) in contrast show consistently higher levels (perhaps reflecting under-reporting of crime) but a reduction in crime from 43 percent to 38 percent for the same years.
the fear of crime see Dammert (2002), Dammert and Malone (2003) and Luengas and Ruprah (2008), For studies on diverse aspects of on crime in Chile see the papers in the Center for Studies on Citizen Security of the University of Chile: http://www.cesc.uchile.cl/serie_estudios.html
3
Chart 3: Change in Reported High Crime Rates 300
Reported Crime (index, 2001=100)
High Crimes Aggravated robbery
250
Robbery through intimidation or threat Larceny
200
Robbery through force Theft
150 Battery Homicide
100
Rape
50 2001
2002
2003
2004
2005
Source: CSD/MI, see Appendix A.
The Policy Response In response to the increasing public concern and rising crime rates the number of government plans and initiatives to deal with crime has grown exponentially since the mid-1990s. Government policy had a three- pronged basis: (i) increased public expenditure on Order and Security; (ii) reformed the Penal Code and strengthen security institutions; and (iii) created a number of local community based crime preventive measures. In this section we briefly review these policy responses but concentrate on describing the Safer Commune Program. The de facto increasing importance placed by the government on crime can be gauged by the increased public expenditure on Public Order and Security (see Chart 4), as a percentage of public expenditure it increased by one percentage point and in per capita terms it almost doubled from 1999 to 2005. The number of police per habitant has also increased (see Chart 5). The distribution of police per habitant and crime rates across regions over time is highly positively correlated.
4
Chart 4: Budgetary Expenditure on Order and Security 110
7.0
6.0 80 Per Capita
5.5
70 5.0 60 4.5
50 40
No. per 100,000 Habitants
6.5 % of Central Government Exp.
90
205 As % of Central Government Expenditure
100
US.$ Per Capita
Chart 5: Number of Police Per Capita
2000
2001
2002
2003
2004
195
190
185
4.0 1999
200
180
2005
2000
Source: CSD/MI , see Appendix A
2001
2002
2003
2004
2005
2006
Source: CSD/MI, see Appendix A
The penal code reform in 2000 changed the country’s judicial system from one characterised as a written-inquisitorial to an oral –adversial system. In its implementation two characteristics stand out. First, unlike other reforms in LAC where reform was applied simultaneously throughout the country in Chile it followed a process of pilot programs and then gradually implemented throughout the country. Second, the new system began without having old cases transferred to it (See Marangunic and Folesong 2004, and Blanco et al 2005). From 13% municipalities with the new system by 2005 about 87% of the total number of 346 municipalities were using the new system. Institutional change included the creation, in 2000, of the Citizen Security Division within the Ministry of the Interior. The division was charged for developing and implementing anti-crime policies and programs including Safer Commune program. A number of community based crime prevention programs were also introduced. Vulnerable Neighbourhood (later renamed as Safer Barrio) program started in 2002 and was gradually extended to cover more municipalities that suffer from high levels of crime associated with drugs. 6 This program is aimed at neighbourhoods with high prevalence of drug related crimes. Plan Precinct, began in1998 on an experimental basis –a pilot- in southern Santiago and thereafter has gradually been expanded. It consists of neighbourhood police patrolling the streets and police participation in local neighbourhood committees. The other main program was the Safer Commune program.
6
The origin of the program was an ad hoc intervention in September 2001, in La Legua, where drug related crime had reached very high levels.
5
Safer Commune The Safer Commune programs historical roots can be traced generally to the neighbourhood security sub-program of 1993 when the Ministry of the Interior invited social organisations to present security projects, and specifically to a pilot project designed by Paz Ciudadana –an NGO- and the Municipality of Santiago. Paz Ciudadana also participated in the workshops coordinated by the Interior Ministry, which included staff from the Ministry and the Division of Social Organisations, to elaborate the program “Safe Municipality- Compromise Hundred” from which was born the Citizen Security Division in the Ministry of the Interior and the program it manages, the Safer Commune Program. The program in turn created local Citizen Security Councils at the municipality level. The Safer Commune program’s design and implementation can be described by its objectives, institutional framework including mechanisms to empower local communities, budget, typology of projects financed, coverage, targeting mechanism, and public opinion of the program. Objectives of Safer Commune In terms of objectives, the program according to Mazano et al (2006), originally defined them as “brake the rise in crime and reduce the feeling of fear” ( freely translated from “frenar al alza de la delincuencia y disminuir la sensación de temor”), although in 2003 they was redefined as “strengthen the local management capacity to prevent crime and reduce fear” (freely translated from “fortalecer las capacidades de gestión locales para previnar el delito y disminuir temor miedo y el delito”). In practice the program has failed to operationalise the objectives in evaluable terms hence clearly relate activities to numerical values of expected outcomes. The lack of objective expected outcomes has led to a confusion regarding what the program is about amongst the general public and a disconnection between the political discourse and the reality of the program. The absence of defined objectives combined with a lack of a coherent communication policy may account for the fact that the general population is unsure what the program is about. Of respondents who claim to know the program only 1% considers it to be a crime prevention program and 15% as a citizen security program, similar levels to respondents who confess they do not know the program. Chart 6 in which is shown the percentage of respondents that identified the purpose of the program from a pre-defined list.
6
Chart 6: Public Interpretation of the Objective of Safer CommuneSafer Commune
32.2% 32.7%
Improved vigilance 21.6%
Community org.
28.0% 16.9% 15.8%
Community education
14.0% 15.1%
Citizen security
7.1% 1.3% 1.1%
Crime dentention
Reduce school drop out
W/o knowledge With knowledge
13.4%
Poverty program
0.6% 0.3%
0.0%
5.0%
10.0%
15.0%
20.0%
25.0%
30.0%
35.0%
40.0%
Source: National Survey of Urban Security of 2005
Institutional Framework and Citizen Participation In terms of the institutional framework the Citizen Security Division of the Ministry of the Interior manages the Safer Commune program. It operatives a competitive fund which finances projects at the municipal level, and to mitigate local institutional weakness provides technical professionals to the local communities. They work in the locality. At the local level are created Citizen Security Committees. The committees are presided by the local mayor and are supposed to be composed of representatives of local government, citizens, police, and representatives of other programs. The Committees are responsible for a diagnostic of the local problem, the development of a local strategy that is presented to the Competitive Fund, and upon approval –see targeting below- the implementation of the strategy and the specific projects. For these activities a technical professional, which is financed by the program, helps the committee and the commune.
7
In practice the institutional framework has encountered problems (see Manzano et al 2006). The institutional framework is best described as work-in-progress with the basic institutional framework been seen as unsatisfactory. At the central government level the Citizen Security Division that reported to an undersecretary of the Ministry of the Interior has encountered problems with a high turnover in directors. With the objective of giving a higher profile to crime and to enhance intra-government coordination, in 2006 the new administration of president Bachelet proposed overhauling the institutional framework. Originally the idea was to create a new Ministry of Citizen Security, but with a change in June of the Minister of the Interior the proposed law was modified to create an Undersecretary of Citizen Security in the Interior Ministry. At the local level, as mentioned above strengthening local capacity in crime prevention was one of the objectives of the Program. Manzano et al (2006) point out that the tension for the Citizen Security Division between intervening directly or coordinating the interventions at the central government is compounded at the local level given that municipalities are not legally charged with law and order. Thus in the earlier stages of the program it was an add-on to the functions of local government and took time to be mainstreamed into the activities of the municipalities. Further, the municipalities have not obtained possible synergies by generally failing to coordinate activities with other programs like Vulnerable Neighbourhood when they coexist in a municipality. The Citizen Security Committees have not been free of problems. Initially their membership included only local government officials and the police hence coproduction was absent. However, gradually local community participation increased. The strategy of enhancing local empowerment remains a problem partly due to a low standing with residents of the entities involved. Local residents performance rating of the entities involved in the program is biased towards low ratings for the Ministry of the Interior, Municipalities, and Citizen Security Committees although the opposite holds for the police (see Chart 7). Second, there is a low level of individual participation and relatively little knowledge of the various entities or processes of the program (see Chart 8).
8
Chart 7: Public Opinion about Entities involved in Safer Commune 60%
Chart 8: Citizen Participation in Safer Commune Program has participated in the execution of a project
Police Ministerio de interior
50%
know of the competitive fund to finance projects
Las municipalidades 40%
know of the neighbourhood security plan
Community Citizen Security Committees
30%
know of the security diagnostic of the neighbourhood
20%
know of the existence of citizen security council participated in any meeting to discuss security
10%
recived information about the program
0% Very good
Good
Regular
Bad
Very bad
0%
5% 10% 15% 20% 25% 30% 35%
Source: National Survey of Urban Security of 2005.
Little knowledge about the program could be taken as an example of communication failure and the remedy of public campaigns to fill the knowledge gap appears to be an obvious policy solution. However, low opinion of, and participation by, citizens in the program is more problematic in terms of the policy response. Low participation is a common stumbling block of programs that attempt to enhance coproduction i.e. cooperative work of the crime prevention program, neighbourhood residents, and other non-governmental organisations to control crime and increase safety. If, as the program assumes, the link between community context and crime requires direct citizen participation (see Dammert 2002) then this problem needs to be studied further and policy adjusted appropriately to enhance participation. Community participation is generally high in Chile, about 67% of citizens reported that they participate in some type of local organisation or local chapter of a national organisation or engage in local activities (see Table 1 in which participation is separated into passive vs. active, and general vs. crime specific). Participation in Safer Commune, however, is low, about 16% and only 3% with active participation.
9
Table 1: Measures of Local Resident Activism General
Passive
Active
Crime Specific
Local Organisations Membership (52%)
Neighbourhood Safety Committee (2%)
Neighbourhood Association Membership (8%)
Safer Commune. Passive participation (13%) \1
Social Integration Activities (21%) Safer Commune. Active \2 participation (3%) \1
Total Net Participation
56%
40%
Safety Activities (19%) Total Net Participation
58%
29%
67%
\1 Participation in safer commune is from question 151 in the victimisation survey of 2005, where passive participation includes the answers: received information of the program; know of the existence of Citizen Safety Council; know of the diagnostic of security; and know of the security plan, and active participation aggregates the answers: participated in a meeting on citizen security and participated in a safety project. Source: National Survey of Urban Security of 2005
Nonetheless, high local engagement suggests that an interpretation that residents do not care or are unmotivated would be mistaken given the numbers for social integration activities and safety activities. The latter figures may be picking up the de facto mechanisms that link collective efficacy to community safety and order, a potential that Citizens Security Council’s have not been able to tap. It also points to an opposite direction of change to that taken by the program. Recently, the program has created zone coordinators that will be responsible for number of municipalities hence will help to the extent that crime rates are spatially auto correlated, and also the program has formally began working with the national mayors association. However, returns to a focus in the opposite direction namely at the neighbourhood level may be higher. Budget, Coverage and Typology of Projects The budget comes from ordinary fiscal resources and from a loan from the InterAmerican Development Bank. Specific projects of the program are financed by a competitive fund that is often co-financed from other sources. In terms of the typology of projects financed by the program these, as classified by the Division, include two sub categories: situational (lighting, green areas, sports infrastructure, and security equipment). In 2005 the Division started using a three way classification of the projects: community infrastructure; strengthening the community; and supporting local management. In practice the total budget has increased each year reflecting an increased coverage of municipalities from 13 in 2001 to 54 by 2005. Coverage increased but represented a falling average expenditure per municipality per year. Safer
10
Commune expenditure is complemented by co-financing from other public sources that has hovered around 35% of the total expenditure per project. Chart 9. Safer Commune: Expenditure Distribution by Type of Projects
Psychosocial Family 2% Psychosocial Children and Youth 2%
Psychosocial Strenthening Neighbourhood 1% Green Areas 11%
NeighbourhoodStrengt hening 11%
Social Locations 7%
Family 6% Security Equipment 8%
Sport Infrastructure 8% Children and Youth 27% Lighting 17%
Source: Ministry of the Interior
A typology of projects of the program is shown in Chart 9; about 51% of projects were typical infrastructure projects (lighting, green areas, security equipment etc.); about 11% went towards strengthening local organisations; and psychosocial interventions represented about 10%. Targeting In terms of targeting design the Safer Commune program went through two steps. During the first period the ranking of municipalities was based on the following formula: robbery with violence (weight 20%), robbery with force (20%), intra-family violence (20%); prevalence of drug consumption (10%); and poverty incidence (35%). In the second period the ranking was based on three indicators: social vulnerability, population, High Crime Rates, with weights of 50%, 30% and 20% respectively (see Moya and Serra 2002).
11
Chart 10. Safer Commune: Targeting and Coverage Coverage
18
Number of Munciplalties
16 14
Safer Commune Number of Municipalities Lowest High-Crimes incidence Decile 2 Decile 3 Decile 4 Decile 5 Decile 6 Decile 7 Decile 8 Decile 9 Highest High-Crimes incidence Total
2001 2003 2005
12 10 8 6 4 2 0 Lowest Decile 2 HighCrimes incidence
Decile 3
Decile 4
Decile 5
Decile 6
Decile 7
Decile 8
Decile 9
35 34 34 34 34 35 34 34 34 34 342
2001 2003 2005 0 0 1 0 0 0 0 0 0 0 1 3 1 4 2 3 5 4 3 5 5 1 7 10 3 9 13 2 11 16 13 42 54
Highest HighCrimes incidence
Sources: Own calculations
In practice, although the formal targeting mechanisms were more sophisticated then just using the criteria of incidence of high crime, de facto the program has increasingly targeted municipalities that rank high in crime incidence (see Chart 10). This is to be expected as there is a high correlation between, and a similar ranking of, municipalities using risk and reported crime criteria. Public’s Evaluation of the Program Before discussing the impact of the program it is important to note the lay opinion of the program. In the Victimisation survey of 2005 given the choice between satisfied and unsatisfied with the program the majority (57%) of respondents said they were unsatisfied. A more nuanced response regarding ranking the performance is when respondents were given a choice of a seven categorical response. As can be seen in Chart 11 the distribution of the ratings is biased towards the higher end of negative rating (7).
12
Chart 11. Safer Commune: Public Opinion of the Program
do not know-no response
7
Ranking Categories
6
5
4
3
2
1 0%
5%
10%
15%
20%
25%
30%
35%
40%
Source: National Survey of Urban Security of 2005.
OUTCOMES EVALUATIVE STRATEGY AND FINDINGS In this section we present the evaluative strategy and the efficacy of the program on the two expected outcomes; a reduction in reported crime rates and the fear of crime. The treatment for reported crime is defined as the existence of the program in the municipality. For the fear outcome the treatment is defined as the active participation of the residents of the municipality in the program. Evaluative Strategy The effects of the program are measured by the impacts of the program. For comparison purposes we also include na誰ve calculations. Na誰ve evaluations compute the change in the variable of interest (in our case crime indicators) over time in the municipalities treated (beneficiaries of the program). Na誰ve evaluations potentially can use two standards: the change in outcome over time with respect to baseline values; and progress made towards the targeted values. The program did not have numerically defined targets hence the evaluation uses the change in reported crime rates over time.
13
The impact approach is attempts to determine the proportion of the observed change that is attributable to the program. It is conceptually simple and intuitive. The impact evaluation approach compares the average value of the variable of interest with the average value of that variable that would have been observed if the program had not existed. In practice, for a non-random designed program like Safer Commune, this involves constructing a comparison group of municipalities who were not beneficiaries of the program but are similar in all other aspects to the municipalities that were beneficiaries of the program. The difference in the average value of the crime indicator of the treated municipalities relative to the average value of the crime indicator of the comparison group composed of nontreated similar municipalities is taken to be the impact of the program. 7 The treated (beneficiaries) and non-treated comparison group are determined through a propensity score matching approach using a logit regression. From the propensity scores households are matched through an algorithm to determine the specific treated and comparison group. 8 The success of this method depends critically on the construction of the comparison and treated groups used in the comparison. Thus to be confident that the groups are similar in all relevant observable characteristics other than treatment requires defining the statistical equivalent to the concept of "similar". We use "t" test compare the means of the individual covariates of the two groups, the Hotelling joint significance test of the set of means of the covariates is equal between two groups) and, although we use only mean values for the impact calculations, the Kolmogorov-Smirnov "D" statistic (i.e. based on the entire distribution to determine whether two independent samples have been drawn from the same population) plus a chart of the distribution of the propensity score of unmatched and matched groups to determine acceptable level of similarity. FINDINGS Impact on Reported Crime
7
For cross section data, in our case the data for fear of crime, the calculation is a single difference between treated and comparison group, for panel data, in our case for reported crime, with two points in time the calculation is a double difference. Formally, the single difference is defined as the difference in the outcome indicator between the treated and comparison group i.e.: SD = (Om1t - Om1nt); the double difference is defined as the difference in the change in the outcome indicator over time between the treated and comparison group, i.e.: DD = (Om1t - Om1nt) + (Om0t - Om0nt) where O is the outcome of interest, m indicates that mean value of the outcome, 0 is at base-line and 1 is end-line time, t and nt is treated and comparison group. 8 See Caliendo et al (2005) for a discussion of the different algorithms. In this paper we report only a specific exercise although different algorithms were used. No substantial difference in the results was observed.
14
A declared objective of the program was to reduce crime. In this section we report the estimated effects of the program on reported high crime rates both at the aggregate and for the six types of crimes (aggravated robbery, robbery through threat, larceny, theft, battery, homicide, and rape) that enter into the country’s definition of high crime. Reported high crime rates have increased in the program’s beneficiary municipalities by 48%. However, that increase has been slightly lower than the non-beneficiary municipalities where high crime rates increased by 51% with the ensuing slight fall in the ratio of high crime between beneficiary to nonbeneficiary from 1.68 to 1.65. Nonetheless, using this approach leads to the conclusion that the program has failed. Such a conclusion would be erroneous. The relevant evidence would be the observed crime rates relative to the crime rates that would have been observed if the program had not existed. For that we need impact calculations. The treatment is defined as the existence of the program in the municipality. The method used is double difference- propensity score. Impact calculations depend critically on successful matching of treated and nontreated groups. As noted above impact calculations depend critically on the matching quality. The matching variables, the estimated values of the coefficients from the logit regression and calculations of the matching quality at the level of the individual covariates through their mean values, reduction in bias between unmatched and matched data, individual and joint mean tests are presented in Table 2. The bias was substantially reduced. The “t” test of equality of individual means for matched data holds for practically all the variables although not for the unmatched data. In the table is also given two joint means tests; the equality of joint means hypothesis is not rejected.
15
Table 2: Crime; Regression Coefficients, and Treated and Comparison Groups Means’ Similarity Tests. Variable ln (density of population 2003) Gini Index Municipality Investment 2003 Penal Process Reform 2003 Rate Robbery with Violence or Intimidation 2003 Rate Robbery with Force 2003 Rate Theft 2003 Rate Lesions 2003 Incidence of Poor 2003 Region Dummy of Dispersion 2003 Birth Rate 2003 Rate Robbery with Surprise 2003 Rate of Homicide 2003 Rate of Rape 2003 Infantile Mortality Rate 2003
Coefficient 1.415*** [0.29] 13.87*** [4.65] 0 0 0.00302** [0.0012] 0.0227*** [0.0053] -0.00174*** [0.00058] -0.00189 [0.0015] 0.00460* [0.0025] -10.64** [4.26] -0.139 [0.17] -1.223 [1.07] 0.198** [0.085] -0.00573 [0.0056] 0.0974 [0.065] 0.0171 [0.033] -0.776*** [0.28]
UnMatched Matched UnMatched Matched UnMatched Matched UnMatched Matched UnMatched Matched UnMatched Matched UnMatched Matched UnMatched Matched UnMatched Matched UnMatched Matched UnMatched Matched UnMatched Matched UnMatched Matched UnMatched Matched UnMatched Matched UnMatched Matched
Treated Control % Bias Bias 6.8545 2.9843 205.8 4.5674 4.5428 1.3 99.4 0.46514 0.46411 1.6 0.48063 0.46586 23.1 -1335.3 0.12917 0.19283 -59.9 0.13389 0.13579 -1.8 97 1017.4 1824.9 -98.5 1801.4 2002.3 -24.5 75.1 335.87 53.239 153.7 160.2 122 20.8 86.5 937.31 582.94 60.2 1158 1078.4 13.5 77.5 556.7 390.87 45.8 691.9 574.29 32.5 29.1 534.74 472.5 38 520.7 477.57 26.4 30.7 0.11973 0.22588 -128.4 0.16066 0.14802 15.3 88.1 9.5902 7.5575 60.3 6.8 5.4286 40.7 32.5 0.04918 0.12832 -28 0.2 0.28571 -30.3 -8.3 16.848 14.995 41.8 17.39 17.843 -10.2 75.6 132.59 17.478 95.9 60.4 35.286 20.9 78.2 2.0164 2.0841 -1.9 1.7 1.5714 3.6 -90 12.639 8.5265 43.3 9.4 4.5714 50.8 -17.4 5.4672 5.8376 -22.8 5.58 5.2143 22.5 1.3
t 15.57 -0.13 0.11 0.23 -3.99 -0.13 -8.16 -1 14.49 0.73 3.87 0.15 3.6 0.46 2.44 0.64 -7.8 0.65 4.57 1.27 -1.75 0 2.85 -0.08 9.29 0.89 -0.11 0.08 2.58 3.6 -1.49 0.52
p>t 0 0.899 0.912 0.818 0 0.896 0 0.336 0 0.478 0 0.881 0 0.657 0.015 0.532 0 0.526 0 0.227 0.082 1 0.005 0.939 0 0.391 0.912 0.936 0.01 0.004 0.138 0.613
Bal. Y/N N Y Y Y N Y N Y N Y N Y N Y N Y N Y N Y N Y N Y N Y Y Y N N Y Y
; *,**, ***, significant at 10%,5%, and 1% respectively Joint Means Statistical Tests 2-group Hotelling's T-squared = 230.95413 F test statistic: ((21-16-1)/(21-2)(16)) x 230.95413 = 3.0388701 H0: Vectors of means are equal for the two groups F(16,4) = 3.0389 Prob > F(16,4) = 0.1456
A second level of tests is based on the entire distribution of the propensity scores before and after matching (see Table 3). The charts of the propensity score before and after matching plus the Kolmogorov – Smirnov test of equality of distribution function after matching show that the distribution of the propensity score between treated and comparison group are not different from each other. Thus the matching seems to be successful.
16
Table 3: Crime Propensity Score Distribution and Similarity Tests Propensity Score of Unmatched Data \1
Propensity Score of Matched (treated and comparison groups) \1
kdensity
0
0
20
.5
D en sity
Density 40
1
60
1 .5
80
kdensity
0
.2
.4
.6
.8
1
0
Pr(benef) beneficiaries
non beneficiaries
.5 Pr(benef) treated
Statistical Tests of Equality of the Propensity Score Distributions
Two-sample Kolmogorov-Smirnov test for equality of distribution functions: D P-value Corrected Untreated: 0.1364 0.823 Treated: -0.1364 0.823 Combined K-S: 0.1364 1 1 \1 Matching algorithm used is Neighbour, no replacement, calliper 0.5
17
1 control
In Table 4 are shown the impact calculations of the effects of the program on reported crime rates. Table 4: Reported Crime Impact Calculations
I ndic a tor H ig h C r im e R ob. V. or I . R o b. S ur pr ise B atte r y Th ef t Le sion s H om ic ide R a pe
N a ïve D iff e re nc e 33 8. 41 74. 17 8 .6 7 15 8. 26 46. 55 42. 51 0 .2 9 3 .9 3
M a tch ed -4 58.2* 29.9 18 -2 93.4* -1 33.2* -7 5.8 0 .7 -4.9
Ne a re st- ne ighbo r 1, ca lipe r (0 .0 5) M a r gina l E ff ec t [ 95 % C on f. I nte rv al] /1 -1 8. 5% - 2, 484 .0 0 -9 8. 92 18. 7% -5 3.20 9 3.64 30. 5% -3 4.33 4 8.27 -2 7. 0% - 1, 032 .6 7 -3 6. 71 -2 0. 2% - 355 .5 7 -2 3. 67 -1 5. 0% - 284 .6 7 8 .0 0 41. 2% -1. 17 4 .7 1 -5 4. 4% -1 5.00 2 .5 0
/1 based on bias corrected error from bootstrapping with 500 repetitions. Matching algorithm used was nearest neighbour, no replacement, and calliper of 0.05, the treated group consists of 10 municipalities and the comparison group consists of 11 municipalities.
The estimated statistically significant effect is a reduction in 18.5% in high crime rates, a reduction in battery (a marginal effect of 27%) and a reduction in theft (a marginal effect of 20%) The implication is that without the program high crime, theft and battery would have been 19%, 20% and 27% higher in the treated municipalities. Impact on the Fear of Crime A second objective of the program was to reduce the fear of crime. We measure fear of crime, as the self-reported feeling of security in the municipality. Security is measured by a seven- ordered category Lickert scale where category 1 is very insecure and 7 is very secure. 9 The treatment is defined as the active participation of individuals living in the municipality in the Safer Commune program. The method used is single difference propensity score. The data used is the country’s Victimisation Survey of 2005. Similar to the crime estimations Table 5 shows the matching variables, the estimated values of the coefficients from the logit regression and the matching quality at the level of the individual covariates mean values and joint means. Matching at the mean values appears successful.
9
In this exercise we take the municipality as the geographical space as the unit of analysis. For an impact analysis at the level of the neighbourhood see Luenguas and Ruprah (2008), although that paper focuses –using multinomial regressions- is to determine the variables associated with the fear of crime.
18
Table 5: Fear of Crime, Regression Coefficients, and Treated and Comparison Groups Means’ Similarity Tests. Variables Municipality High Crimes victimization rate per 100 habitants Urban Poverty rate, 2006 Urban unemployment rate, 2006 Municipality relative per capita urban income, 2006 Age Person experienced a dangerous situation at home Perceived Socioeconomic status - Low Occasional and Informal employment Wage earner: Administrative worker Without difficulties to participate in citizenship security activities Participation in any organisation Participation in the Neighborhood Council Participation in the Neighborhood Security Council Participation in safety activities Participation in activities to improve the neighbourhood Passive participation in Safer Chile
Coefficients 0.023* [0.012] 2.304* [1.193] 3.979 [3.257] -0.427** [0.204] 0.006* [0.003] 0.285* [0.148] 0.381*** [0.146] -0.543* [0.293] 0.421*** [0.146] 0.357* [0.205] 0.387*** [0.126] 1.673*** [0.151] 1.443*** [0.226] 0.486*** [0.122] 0.409*** [0.121] 4.411*** [0.202]
Sample Unmatched Matched Unmatched Matched Unmatched Matched Unmatched Matched Unmatched Matched Unmatched Matched Unmatched Matched Unmatched Matched Unmatched Matched Unmatched Matched Unmatched Matched Unmatched Matched Unmatched Matched Unmatched Matched Unmatched Matched Unmatched Matched
Treated 19.35 19.26 0.14 0.14 0.08 0.08 0.88 0.88 45.34 44.05 0.29 0.30 0.15 0.15 0.03 0.03 0.20 0.23 0.13 0.09 0.75 0.71 0.38 0.26 0.17 0.10 0.33 0.32 0.35 0.33 0.92 0.90
Control 20.00 19.23 0.14 0.14 0.08 0.08 0.84 0.89 45.52 44.03 0.30 0.28 0.25 0.15 0.03 0.02 0.19 0.19 0.07 0.08 0.70 0.73 0.19 0.27 0.04 0.10 0.29 0.31 0.31 0.31 0.28 0.88
Bias -12 0.6 -5 3.8 -5.8 6.9 12.8 -6.1 -1.1 0.2 -0.7 4.2 -25 -2.4 4.5 5.9 2.5 8.6 18.1 3.5 11.2 -5.4 42.5 -1.4 43 0.2 7.7 2 7.1 3.6 171.1 6
% Bias reduct. 95 24.7 -18.3 52.2 85.8 -526.9 90.6 -31.9 -244.8 80.6 51.6 96.7 99.6 74.3 49.6 96.5
t -2.51 0.08 -1.05 0.55 -1.21 1.01 2.65 -0.80 -0.22 0.02 -0.14 0.60 -4.93 -0.37 1.00 0.92 0.53 1.17 4.28 0.52 2.32 -0.76 9.86 -0.20 12.49 0.02 1.66 0.28 1.52 0.51 30.81 0.99
p>t Bal. Y/N 0.012 N 0.933 Y 0.296 Y 0.582 Y 0.227 Y 0.313 Y 0.008 N 0.424 Y 0.828 Y 0.983 Y 0.887 Y 0.551 Y 0.000 N 0.712 Y 0.315 Y 0.360 Y 0.596 Y 0.241 Y 0.000 N 0.601 Y 0.020 N 0.448 Y 0.000 N 0.843 Y 0.000 N 0.981 Y 0.097 Y 0.781 Y 0.128 Y 0.613 Y 0.000 N 0.324 Y
Statistical Tests of Joint Means of the Covariates
2-group Hotelling's T-squared = 9.4914189 F test statistic: ((788-28-1)/(788-2)(28)) x 9.4914189 = .32733492 H0: Vectors of means are equal for the two groups F(28,759) = 0.3273 Prob > F(28,759) = 0.9997 Note: the number of observations in the sample used are 15242 including 520 beneficiries; PsudoR-squared 0.49; robust standard errors in brackets where the standard errors are clustered at the munciplalty level; *,**, ***, significant at 10%,5%, and 1% respectively;
Further, similar to above, matching appears successful as the distribution of the propensity score between treated and comparison group can be said to be statistically similar (see Table 6).
19
Table 6: Fear of Crime; Propensity Score Distribution and Similarity Tests Propensity Score after matching
Propensity Score before matching
Nearest Neighbor 1 (no replacement) - Caliper: 0.01 30
2.5
2
Density
Density
20
1.5
1 10
.5
0
0 0
.1
.2
.3
.4
.5
.6
.7
.8
.9
1
0
Propensity Score Treated - Common Support
.1
.2
.3
.4
.5
.6
.7
.8
.9
1
Propensity Score Untreated - Common Support
Treated Matched
Untreated Matched
Statistical Tests of Equality of the Propensity Score Distributions Two-sample Kolmogorov-Smirnov test for equality of distribution functions D P-value Corrected Untreated 0.0658 0.182 Treated -0.0168 0.894 Combined K-S 0.0658 0.361 0.328
However, generally no statistically significant impact on the fear of crime was found except for category 4 (see Table 7). However, before we can conclude that active participation in the Safer Commune program has no effect on the feeling of security in the municipality, it must be noted that the coefficients of insecurity categories (1 to 4) are generally negative (reduction in insecurity) while the coefficients for security categories (5 to 7) are positive. Taking this pattern into consideration we repeat the impact exercise but for two sub-aggregates: (i) insecurity (categories 1 to 4); and (ii) security (categories 5 to 7). We find that a statistically significant –at the ten percent level- negative for insecurity and by definition positive for security.
20
Table 7: Fear of Crime Impact Calculations
Indicator Na誰ve Difference 1 Very Insecure 0.01 2 -0.02 3 -0.02 4 -0.01 5 0.03 6 0.00 7 Very Secure 0.00
Indicator Categories 1-4 Categories 5-7
Na誰ve difference -0.02 0.03
Matched Difference 0.01 -0.02 0.00 -0.05* 0.02 0.03 0.01 Matched difference -0.06* 0.07*
Marginal Effect [95% Conf. Interval] /1 16.1% -0.03 0.04 -26.1% -0.05 0.02 -1.9% -0.04 0.05 -20.4% -0.11 -0.01 4.5% -0.04 0.09 24.4% 0.00 0.10 41.7% -0.01 0.04 Nearest-neighbor 1, caliper(0.01) Marginal effect 10.3% -12.2%
[95% Conf. Interval] /1 -0.14 0.01 -0.01 0.14
/1 based on the bias corrected error from bootstrapping (with 500 repetitions); the matching alogrothim used was nearest neighbour with replacement, and a caliper of 0.01, the sample of treated and comparison groups are 393 observations each. Where* statistically significant at the 10% level.
The evidence obtained through impact analysis suggests, that contarary to a na誰ve evaluation, the program has contributed to the reducion in high crime and the fear of crime. DISCUSSION Safer Commune program started out as a typical crime prevention public program in 2001. The program has come under a barrage of criticism including that it is soft on crime and that it has had no effect. Lay opinion about the effectiveness of the program and of the local and national entities in charged with the program is low. So is the knowledge by citizens of what the program is about. There is low participation by local residents in the program through the specially created local Citizens Crime Committees. Lack of knowledge of the program is indicative of a communication failure by the agencies running the program. The remedy of public campaigns to fill the knowledge gap appears to be an obvious policy solution. Low participation in the program is, however, more problematic. Local resident participation through specially created Citizen Security Councils was a critical design feature of the program. It may account for the lack of major impacts of the program. High participation was assumed to ensure appropriate local projects were chosen and avoid capture by non-representative interest groups. It was assumed participation of local residents directly and/or through neighbourhood organisations would unleash coproduction with public agencies that would
21
produce order and increase safety beyond just the projects financed. Direct participation was also assumed to reduce the fear of crime through local empowerment. This remains the key challenge for the program. However, effort in resolving problems of implementation of the program including the adjustment of the institutional framework, changing the communication strategy, or adopting measures to enhance local participation are only worthwhile endeavours if the program has any impact on crime and the fear of crime. Some critics assert that the program has failed regarding its fundamental objectives –reduce crime and fear of crime- citing as evidence the rising crime rates and the fear of crime in municipalities with the program. This assertion implies that the effort to improve the institutional framework to improve the delivery system of the program is not worthwhile. This is incorrect. Valid empirical evidence would be the crime measure that would have been observed without the program. Such an impact evaluation – using propensity score method- reveals that the program has reduced high crimes, specifically battery and theft. The program has made Chileans safer. Further, active participation in the program by local residents has decreased feelings of insecurity in the municipality and increased the feeling of security. The program has made Chileans feel safer, but with a very low active participation rate, the scale of the effect is small. These positive findings suggest that an expansion of the program and simultaneously enhancing coproduction of order through mechanisms to encourage local resident participation would have high returns.
22
REFERENCES Benavente J. M., and E. Melo, “Determinantes Socio Económicos De La Criminalidad En Chile Durante Los Noventa” Serie Documento De Trabajo, N. 233, University of Chile, October 2006. Blanco R, R. Hutt, and H., Rojas, “The Reform to the Criminal Justice in Chile: Evaluation and Challenge”, the Loyola University International Law Review, 2005. http://www.cejamericas.org/doc/documentos/cl-
blanco-reform-criminal-justice.pdf Caliendo M., and S. Kopeining “ Some Practical Guidance for the Implementation of Propensity Score Matching”, Discussion Paper 485, DIW, April 2005. Dammert, L. (2002). “Participación Comunitaria en prevención del delito en América Latina. ¿De qué participación hablamos?” In: Cuadernos del Centro de Estudios del Desarrollo, Santiago, Chile. Dammert L., and M. Malone, “Fear of Crime or Fear of Life? Public Insecurities in Chile”, Bulletin of Latin American Research, Vol. 22, No. 1, 2003. Dammert L., “Participación comunitaria en prevención del delito en America Latina: De que participación hablamos? Undated. Forum of Experts, “Diagnóstico de la Seguridad Ciudadana en Chile”, Documento de Trabajo No. 1,. Division de Seguridad Ciudadana, Ministerio del Interior. April 2004. Fundación Paz Ciudadana, “Índice Paz Ciudadana-Adimark” Santiago, D July 23, 2002e, December 27, 2002, July 17, 2003, June 29, 2005. Imbens G., and J. Wooldridge “Recent Developments of Program Evaluation” Working Paper 14251, NBER, August 2005. Instituto Nacional de Estadísticas, “Encuesta Nacional Urbana de Seguridad Ciudadana, Resultados Nacionales”, Ministerio del Interior, April 2004. Libertad y Desarrollo“ Delincuencia 1990-2005: Evaluación Critica y
Propuestas”, Serie Informe Político, , No.91, August 2005, Libertad y Desarrollo Luenguas P., and I. Ruprah “Fear of Crime: Does Trust and Community Participation Matter? Working Paper OVE/WP-08/08, July 2008. IADB. Manzano, l., A. Mohor, J.P. Piña, D. Piñol, C. Rodriguez, and R. Sepúlveda, “Registro de Lecciones Apprendidadas Por El Programa de Seguridad y Participación Ciudadana 2001-2005”, 27 January 2006.
Marangunic A., and T. Foglesong, “Charting Justice Reform in Chile”, Vera Institute of Justice, 2004. http://www.vera.org/publication_pdf/254_498.pdf Ministerio del Interior, “Programa de Seguridad y Participación Ciudadana (SEG. Y PAC)” Gobierno de Chile, Ministerio de Hacienda Dirección de Impuestos, June 2003. Ministerio del Interior, División de Seguridad Ciudadana, “Documento de Política Nacional de Seguridad Ciudadana”, October 2004. Morrison A., “Alternative View Paper: Violence and Crime in Latin America”, August 2007.
http://www.google.com/search?hl=en&q=alternative+view+pa per+andrew+morrison+2007 Moya J. and D. Cistermes, “ Influencia de Factores de Riesgo Social en el Origen de Conductas Delincrenciales” Serie Estudios, Ministrio del Interior, Junio, 2002 Nuñez J., Rivera J., Villavicencio X. and Molina O., “Determinantes Socioeconómicos Del Crimen en Chile” Estudios de Economía, Universidad de Chile, June 2003. Sampson R., and J. Wooldridge “Linking the micro and macro-level dimensions of life style-routine activity and opportunity models of predatory victimisation” Journal of Quantitative Criminology, Vol 3, No. 4, December 1987. Seguridad Ciudadana, Comuna de Penco, Octubre 2004. Documento de Trabajo No. 1, Comuna de Macul, Octubre 2004. Documento de Trabajo No. 2, Comuna de Valparaíso, Octubre 2004. Documento de Trabajo No. 3, Comuna de Villa Alemana, Octubre 2004. Documento de Trabajo No. 4.
Sherman, L., and D. Gottfredson, D. MacKenzie, J. Eck, P. Reuter, and S. Bushway “Preventing Crime: What Works, What Doesn’t What’s Promising. A Report to the United States Congress.” National Institute of Justice . Undated
Appendix A Page 1 of 1
APPENDIX: DATA The study uses information from different sources. descriptions are summarised in the Table below.
The sources and their
Table A1: Data Sources Variables
source
Reported crime
CSD/MI (Citizen Security Division of the Ministry of Interior) The raw information is received by the Division monthly from the police. It is homologised by the division. The data is published quarterly. (www.seguridadciudadana). Reported crimes are classified as high crimes (delitos de mayor connotatcion social) and are disaggregated into: Aggrevated robbery (robo con violencia); robbery with threat (robo con intimadacion); larceny (robo con surprisa); robery with force(robo con fuerza); theft (hurto);battery (lesiones); homocide (homocidio); rape (violaciones)
Victimisation
Chile’s National Survey of Urban Security of 2003 and 2005
Fear of Crime
MINTER/INE: Chile’s National Survey of Urban Security of 2005. The survey’s field work was carried out September to December of that year and with a reference period of the previous 12 months. The domain was urban and rural urban population of population aged 15 years and over in 92 municipalities of the 312 that exist. The number of individuals interviewed was 10,359,219. ADIMARK.
Importance Crime
of
CEP Public opinion polls by the Centro de Estudios Politicos. See: http://www.cepchile.cl/bannerscep/bdatos_encuestas_cep/base_datos.php
Sociodemographic features of municipalities
CASEN, University of Chile.
Program details (coverage and type of projects)
CSD/MI (Citizen Security Division of the Ministry of Interior)