Background Paper International Workshop on Governance Indicators New Delhi, April 20-22
Governance Indicators for Pro-Poor and Gender-Sensitive Policy Reform
1
I. Introduction In light of emerging global standards for governance there is an increasing demand to measure various aspects of democracy, human rights and governance. This demand has resulted in a tremendous growth of governance indicators, which are used to measure the performance of governments, institutional quality and people ’s perception. Traditionally, such indicators have been limited to quantitative or process indicators that describe the institutional inputs that produce governance outcomes. An example of this measurement is the number of people who come out to vote in a country ( ‘voter turnout ’). In recent years, however, there has been an increased focus on creating qualitative or performance indicators that provide assessments on the quality of governance. An example of this type of indicators is people ’s perception on how free and fair an election was conducted.1 Despite the increased demand and growth of governance indicators there remain serious concerns regarding the quality and purpose of these measurements. An extensive desktop mapping carried out by the UNDP Oslo Governance Centre (OGC) in 2003 showed that there exists a plethora of governance indicators used by inter-governmental institutions, developing agencies, NGOs and academic institutions.2 These indicators are almost exclusively applied as country-ranking instruments intended for business investment, donor allocation, civil society advocacy or academic purposes. The most cited indicators include the Freedom in the World Survey (Freedom House), Governance Matters I-III reports (World Bank Institute) and the Corruption Perceptions Index (Transparency International).3 The UNDP mapping and subsequent Governance Indicators: A Users ’ Guide4 showed several limitations with present governance indicators:
1
For more information on different types of governance indicators and how to use them see:
http://www1.worldbank.org/publicsector/indicators.htm and http://www.undp.org/oslocentre/docs04/UserGuide.pdf 2
The mapping has been electronically published as a UNDP guide entitled “Sources for Democratic
Governance Indicators ”. The guide contains over 50 publicly available sources along UNDP governance practice lines. 3
For more information on these sources see http://www.undp.org/oslocentre/docs04/UserGuide.pdf
4
Joint UNDP-European Commission publication:
http://www.undp.org/oslocentre/docs04/UserGuide.pdf.
2
Lack of pro-poor and gender sensitive indicators x
A review of the 50+ sources demonstrated that there was virtually no pro-poor or gender sensitive focus in the measurements. The indicators used by Freedom House, Transparency International and the World Bank Institute, for example, do not provide information on how poverty groups in developing countries are faring (i.e. access to public services, court system, corruption encounters, human rights capacity, etc.). If democratic governance is to be a framework for poverty reduction there clearly needs to be a pro-poor and gender sensitive emphasis on the measuring tools used for governance.
x
The conspicuous lack of pro-poor and pro-gender governance indicators is related to the lack of disaggregated data. The UNDP mapping found that the majority of the sources were highly aggregated measurements and did not provide disaggregate information on particular groups such as the poor, ethnic minorities, and women.
Current indicators are not operational x
The majority of governance indicator sources rank countries on a single aggregate scale or on a few governance cluster indexes (e.g. corruption, freedom of expression, public service delivery etc.). Freedom in the World Surveys for instance, classify countries as “Free ”, “Partly Free ”, and “Not Free ” while the Governance Matters Reports arrange countries on a -2.5 to +2.5 point scale. Although these indices simplify a complex subject into an easily understood rating, they are of very limited use to stakeholders in countries. If development country X is ranked as “Partly Free ” and 0.21 on these indices it does not provide policy makers and other stakeholders with much information on where or what type of action needs to be taken. The same could be said for the Corruption Perceptions Index where a ranking of 95 (out of 133) does not tell the same development country X if corruption is more widespread in the private or public sector or whether corruption is more concentrated in certain parts of the country.
3
Methodological issues: x
The Users ’ Guide produced by UNDP and the European Commission demonstrates that there are several methodological issues with most intercountry indicator sources. The guide shows that governance indicators often fail to disclose what and how they are measuring. Moreover, even when indicator producers are transparent about the methodological process they are often misapplied by users. An example of this is the Corruption Perception Index, which is sometimes wrongfully used to monitor a country ’s effort to combat corruption.5
x
There are severe methodological limitations with measuring governance across countries using the same scale. The World Bank Institute, for example, states that great caution should be used for country comparison on the Governance Matters Index as the margin of error is often very high. 6
x
Most of the governance indicator sources tend to assume that key concepts, such as human rights, corruption, freedom of expression, etc., are defined in the same way in the multiple sources from which data are drawn These data are often aggregated into single measures. Since the assumption of common definitions is not valid, the use of such aggregate indicators, even for intercountry comparison, should be made carefully, allowing for large margins of error.
x
The Guide also shows that many of the indicator sources rely on so called expert groups for their rankings. Freedom House and Transparency International, for example, rank countries on the basis of a few experts ’ perception on the state of governance. Apart from the political implications, there are serious methodological questions regarding the validity and reliability of using such expert opinions as representative assessments for entire countries. Moreover, from UNDP ’ perspective there is a clear lack of poverty focus when using these types of expert groups.
5
The index measures the perception of corruption in business transaction in a given country, which
does not necessarily say anything about the country ’s effort to combat corruption. Moreover, countries are measured on an absolute scale, which means that country Y can climb on the ranking ladder without a change in score if another country performs worse. 6
The WBI Governance Matters Reports provide the margins of error for countries
http://www.worldbank.org/wbi/governance/pubs/govmatters3.html
4
II. UNDP Project Governance Indicators for Pro-Poor and Gender-Sensitive Policy Reform Overview Recognizing the need for more meaningful and operational measurements of democratic governance, the UNDP Oslo Governance Centre (OGC) proposes to assist 6-8 pilot countries in producing non-ranking governance indicators to be used for propoor and gender sensitive policy reform. As demonstrated in the sections above there is a clear need for governance indicators that are: x
Poverty and gender oriented: need to disaggregate governance data for these and other marginalized groups at both national and local levels.
x
policy-action oriented: the indicators need to be operational and inform the stakeholders where specific policy action is needed
x
nationally owned and participatory: the indicators and data need to be produced in the country through a broad stakeholder consensus.
x
methodologically sound: the indicators and data collection process needs to be transparent and accountable. Greater attention needs to be paid to the issues raised in the Governance Indicators: A Users’’ Guide.
The project is to benefit developing countries who will be the primary owners and users of the governance indicators. The objective is to provide governance indicators as a tool to create institutions and processes that can be more responsive to the needs of ordinary citizens, including the poor and women. In particular, there is a need to assist developing countries produce disaggregated data on governance, which can help stakeholders to identify where policy change is needed, especially with regards to marginalized groups. The indicators would serve as baselines from which government and stakeholders can measure progress or regression using time series. There is a clear need for an institutionalized system for collecting data and producing indicators that could help policy makers not only to identify problems, but also to monitor the effectiveness of actions taken to respond to those problems.
5
Framework for National Indicators The mapping and the Users’’ Guide show that cross-country indicators are of limited use to developing countries. There is a clear need for indicators that are more relevant and useful to the national and sub-national context. At the same time there needs to be a set of indicators that reflect basic dimensions of governance. The project is therefore conceived to produce two sets of indicators: 1. Core governance indicators: these would reflect universal aspects of governance that are relevant in all countries at both national and local level. The project will compartmentalize these core indicators along the UNDP service lines of governance, which cover the following areas: -parliamentary development, -electoral systems and processes, -justice and human rights, -E-governance and access to information, -decentralization, local governance and urban/rural development, -public administration reform and anti-corruption Stakeholders in each pilot country will identify the core indicators under each of the governance areas. Upon completing the pilot testing it should be possible to compare (not rank) core indicators among the pilot countries.7 2. Satellite indicators: will be indicators and data specific to the pilot country context. The stakeholders will identify governance issues that are particular to the country and discuss specific measurements for these issues. The satellite measurements are indicators that cannot be captured by global (inter-country) sources. Disaggregation: Both core and satellite indicators will need to be disaggregated to highest degree possible across space (national and local levels) and population (gender, age, minorities and other vulnerable groups).
7
The core indicators will often require different measurements (for example, voter turnout may be a
misleading indicator in countries where voting is mandatory).
6
Pilot Strategy The governance indicators in each of the pilot countries will be demand driven through UNDP’’s Country Office and their partnership with the host government, statistical office, academic research institutions and other relevant stakeholders. The UNDP Country Offices are well situated to coordinate the project and identify the appropriate actors and partners. As the indicators are intended for pro-poor governance reform it will be crucial to have the relevant government actors involved from the beginning. The indicators are meant to provide empirical data for policy. The challenge will be to get policy makers to make good use of the indicators. The participation of the statistical office in each pilot country will ensure sustainable capacity development and represents the most suitable entity to create national/subnational indicators for policy reform. In addition, it will be necessary to collaborate with a more specialized national (or regional) academic institution for developing the indicators. The UNDP project recognizes that there are several ongoing governance indicators initiatives in many developing countries and does not seek to replicate these efforts. The objective of the project is actually to coordinate and make use of the multiple indicator sources being developed. Key to this objective will be partnerships with other indicator actors both at national and global level. Pilot framework Although the pilot testing in the countries will develop differently it is possible to identify a common framework for the pilot exercise. The following steps will be taken in each pilot country: x
Mapping of available governance indicators in the country, which will include local, national and international sources of data.
x
Identify ongoing and pipeline efforts to produce governance data. This can also be complimented by an assessment on national capacity to collect governance data.
x
Establish National Steering Committee for the pilot project comprising of representatives from national and local governments, parliament and local assemblies, civil society, academia, private sector, the National Statistical Office and other relevant stakeholders.
7
x
The Steering Committee will identify core and satellite governance indicators and a timeline to pilot test these indicators. The Committee will also agree on a suitable national institution to collect and analyze the data (ideally NSO and specialized governance institution).
Project outputs: x
Governance indicator assessment from 6-8 pilot countries
x
Handbook on developing governance indicators at country level (to incorporate lessons learned and new methodology from pilot testing)
x
Core indicators that can be used in new pilot countries
x
Added value to several development frameworks that require data on governance such as the Common Country Assessment (CCA), the United Nations Development Assistance Framework (UNDAF), Poverty Reduction Strategy Papers (PRSPs), and Human Rights Based Approaches (HRBAs).
8