awild

THE DEMOCRATIZATION OF GOVERNANCE ASSESSMENTS Alexandra Wilde

Governance assessments, and democratic governance assessments in particular, have not been around for long. It is only since the late 1990s that comprehensive democratic governance assessment frameworks have been developed and implemented systematically. This chapter examines the principal influences on assessment approaches' tools and methods. The most dynamic period in their development has been since the early 2000's, which have witnessed a simultaneous surge in the production of donor-driven, government-driven, and citizen-driven assessments. There are shared and divergent reasons for the propagation of assessments among these actors, but for many, assessments represent an important instrument for enhancing accountability and transparency, maximizing the effectiveness of development programmes, and providing an empirical basis for policy reform.

A key challenge in developing countries is to understand how a democratic governance assessment can strengthen democratic development. There are two camps of thought on this issue. The first sees democratic governance assessments as an exogenous aid-conditionality instrument, in which the assessment provides an incentive to reform. For example, democracy promoters argue that the Millennium Challenge Account, which is based on countries passing a governance and democracy test, has been highly effective in bolstering democratization in recipient countries. The second sees democratic governance assessments as an endogenous transformative process that should be guided by the same democracy and good governance principles it seeks to assess (e.g., participatory, transparent, representative). This latter view is said to provide more potential for local capacity development and local ownership.

This chapter charts the contributions to the thinking and practice on the content of governance assessments (what to measure, and which tools to use) to the more recent concern with the process of assessment, which has been influenced by social accountability thinking, the rights based approach, and international commitments to national ownership of development.

Evolving normative and operational imperatives, as well as innovations in methods and tools since the mid 1990s, have had a considerable impact on how governance assessments are designed and implemented to the extent that there have been three distinct waves of assessment.

The first wave of governance assessment was heavily influenced by the work of academics fixated on producing instruments that seek to evaluate the progress of democratization in regions such as Latin America and Eastern Europe, and the democracy support that was given by the donor community. In the democracy promotion field, first wave indicators are rather crude measures that are oriented towards academic research. Furthermore, most of these indicators are based on an institution-centred concept of democratic governance as opposed to a people-centred concept of governance. Much activity in producing these measures was not particularly suitable for use by development practitioners, the chief criticism being an overuse of quantitative measures and the over-simplification of the governance or democracy concept by insisting on minimalist definitions. At the same time, the theoretical insights that donors took away toward the end of the first wave were two approaches that have continued to be influential in subsequent years. One is the Results-Based Management approach, the other the Logical Framework Analysis. Both contributed to keeping donor agencies focused on assessing results.

The second wave of governance assessment built on the first, but was particularly influenced by an increasing demand from donors for instruments that can be used to determine development assistance decisions. This has resulted in the production of a number of global indices that are used to rank countries' performance, including instruments produced by the World Bank, and Transparency International, among others. New aid conditionality programmes designed by organizations such as the Millennium Challenge Corporation, the Ibrahim Foundation, the World Bank, the European Commission, and other bilateral donors, are using these tools. Many are 'naming and shaming' indices and have been criticized for lacking transparency in the way they are compiled and for being used to make decisions for which they are ill-suited. The second wave also includes assessments that combine and integrate democracy, governance, and poverty reduction dimensions. Carothers (2009) writes extensively on the divide that has existed between democracy and development specialists and how these fields are often treated as entirely distinct disciplines. This can also be seen in the body of work in which separate stand-alone assessments have developed over time for governance, democracy, and human rights, but where the divisions between the three gradually dissipate.

The third wave builds on the innovations from previous generations in terms ofthe content ofgovernance assessments, but turns its focus to the process of assessment. It does this through its emphasis on principles and actions that fortify local ownership, strengthen domestic accountability relationships, and develop local capacities. The Africa Peer Review Mechanism, which is based on regionally chosen indicators and an implementation process that emphasizes domestic ownership, is an important example of this most recent generation. The third wave of assessments is also heavily influenced by the aid effectiveness agenda (the Paris Declaration on Aid Effectiveness and the Accra Agenda for Action) and the rights-based approach. The latter is processoriented and underlines domestic accountability relationships and capacity development. Finally, the third wave is marked by the prominence of social accountability instruments, such as citizen-led assessments, surveys of the public (i.e., of ordinary citizens), and using measurement tools that empower citizens in the process of measurement. John Keane describes this as "monitory democracy", an approach marked by public scrutiny and public control of decision makers through surveys, focus groups, deliberative polling, online petitions, and audience and customer voting. All these are citizen-driven accountability mechanisms

1

The Democratization oof Governance Assessments

that in many ways can be more powerful checks than traditional accountability institutions, such as parliament, the judiciary, or the media (Keane, 2009).

First wave: democracy programming indicators and academic datasets The first wave began in the 1970s with the democratic transitions in southern Europe — Greece, Spain, and Portugal — and continued in the 1980s in Latin America. By 1990, the democracy bug had infected more than 60 countries throughout Africa, Asia, Latin America, and Europe, all of which underwent some form of democratic transition (Huntington, 1991). The fall of the Berlin Wall in 1989 and the spread of democracy in the former Soviet Union and Latin America were significant landmarks towards the end of this wave. At this time, capitalism and liberal democracy were increasingly espoused as ideal systems. In the context of US foreign policy, democracy promotion was not an explicit area until 1994, but various activities that existed before then could be thought of as democracy-related assistance, such as the post-World War II Marshall Plan and human rights programmes in Latin America in the 1980s.

The "democracy promotion industry", as Carothers (1999) describes it, came into being in the 1980s. Along with the US, European democracy aid also expanded with programmes in Latin America and later in Africa and Eastern Europe. There was also a huge increase in the number of international democracy organizations starting in earnest in the early 1980s. Freedom House, which was founded in 1941 but did not begin producing its annual assessments on the State of Freedom in the World until 1978 (the same year, incidentally, that Human Rights Watch was founded), Amnesty International which was established in 1961, and the German government-funded party foundations, Friedrich Ebert, Friedrich Neumann, and Konrad Adenauer foundations in the 1960s, are the early exceptions. The real mushrooming began in the 1980s and included the creation of the National Endowment for Democracy in 1983, which provides funding for the International Republican Institute, and the National Democratic Institute, the Westminster Foundation for Democracy in the United Kingdom in 1992, the European Initiative for Democracy and Human Rights (EIDHR) in 1994 (which has since been replaced by the European Instrument for Democracy and Human Rights (also EIDHR) in 2007, the founding of the International Institute for Democracy and Electoral Assistance (IDEA) in 1995, and the development of its State of Democracy project in 2002, and then more recently the Netherlands Institute of Multiparty Democracy in 2000, among others.1 It is important to acknowledge that the first wave of assessments was dominated by academics who were employed by stakeholders, such as the international democracy organizations, rather than the official donor agencies. It was the academic community that was responsible for building large multi-country, time series democracy and politics datasets. Landman (2009) outlines the various strategies that scholars have used to measure democracy since the 1960s, including categorical measures (democracy versus non-democracy), scale measures (e.g., a rating on a 1 to 10 scale), objective measures (e.g., voter turnout and party share of the vote), hybrid measures of democratic practices, and perceptions of democracy based on mass public opinion surveys. Cheibub et al. (2009) also note variation in how democracy is measured based on different notions of democracy, i.e., continuous or categorical? And if categorical, dichotomous or polychotomous?

Although political data (and statistics, in particular) are far behind social and economic comparative data, Munck (2009) notes that the current period is unprecedented in terms of the production of data on politics. A sampling of some of this production includes the Polity data series which began in 1978 and is still widely used; the six-fold classification of regimes by Cheibub which was first published in 1996 and then revised in 2009 and renamed the Democracy and Dictatorship Data (Cheibub et al., 1. The EIDHR is a European Union programme that aims to promote and support human rights and democracy worldwide. Since its inception in 1994, the EIDHR has been the principal vehicle for extending support to civil society activities in the promotion of human rights and democracy

2009); the Vanhanen Polyarchy Dataset; Przeworski's Democracy and Development Extended Dataset (data from 1946-2002), and Freedom House's State of Freedom (previously known as the Gastil Index) with data from 1972 to the present, among others.

However, despite this impressive collection of data, there is a significant challenge to be overcome in making it relevant and usable beyond academia.

Yet another influence on the first wave of indicators is the contribution of surveying to democracy and governance data. The last two decades have seen a significant growth in the range and scope of public opinion surveys that are based on adult population samples. They measure the public's views of the quality of democracy in their own country, and assess the performance of their government leaders, institutions, policies, confidence and trust in government institutions, and satisfaction with democracy, patterns of political behaviour and civic engagement, and social and political values. Norris (2008) maps the major cross-national and time-series surveys of public opinion, which include the Global Barometers (Europe, Latin America, Africa and, more recently, the Asia and Arab barometers), the World Values Survey, the Pew Global Attitudes, and the Gallup World Poll. The surveys are an important addition to the data sources for democracy and governance assessments, because they capture the differences that exist between de jure rules and de facto outcomes on the ground. While subjective data has often been thought considered inferior in the measurement of other variables and phenomena, in the democracy and governance field subjective indicators can often be superior precisely because of this property.

With the increase in democracy-building support in the second half of the 1990s, demand grew for evaluation instruments and assessment tools that would not only show whether the assistance was having an impact on democratization, but would also explain the distinctive features of democratic development, the role of political institutions and actors in that development, and shed light on such issues as the sequencing of democracy support interventions. Results-based management emerged as the predominant mode for managing donor resources in the late 1990s and it is within this framework that work to develop

2

The Democratization oof Governance Assessments

democracy and governance indicators that could be used in donor programming began. At the heart of results-based management is a focus on a time-bound project or programme and the application of logical framework thinking, which is essentially a results chain that links inputs to outcomes and impacts. It was quite a significant leap at that time to think about development in terms of outcomes and impacts, rather than inputs, activities, and outputs.

The Canadian International Development Agency (CIDA) and United States Agency for International Development (USAID) introduced suggested performance indicators for the democracy assistance 'sector' in the late 1990s. The 1996 CIDA guide, Indicators for Programming in Human Rights and Democratic Development: A Preliminary Study, and the USAID Handbook of Democracy and Governance Program Indicators (1998) remain, more than a decade later, important reference documents on democratic governance indicator development. This is because they provide a considerable number of illustrative or example indicators based on an extremely inclusive conceptualization of democracy and governance that had not previously existed.

That being said, there are useful things to be borrowed from the general evaluation field, particularly in its knowledge of, and experience with, participatory methods. Cracknell (2000) stated that, "the trend towards the use of participatory methods in monitoring and evaluation is undoubtedly the most significant change currently taking place in the field of evaluation". The evaluation discipline has experimented with participatory techniques since the late 1980s. UNDP (1997) provides a useful summary of participatory evaluation. It "involves the stakeholders and beneficiaries of a programme or project in the collective examination and assessment of that programme or project. It is people-centred: project stakeholders and beneficiaries are the key actors of the evaluation process and not the mere objects of the evaluation". Its key characteristics include: •

Drawing on local resources and capacities.

•

Recognizing the innate wisdom and knowledge of end-users.

•

Demonstrating that end-users are creative and knowledgeable about their environment.

•

Ensuring that stakeholders are part of the decision-making process.

•

Using facilitators who act as catalysts and who assist stakeholders in asking key questions.

Crawford and Kearton (2002) note that even though participatory evaluations have been widely practiced in various development fields, this is not the case in the democracy and governance sector, which has obvious complexities and sensitivities for engaging stakeholders such as the government, donors, and non-government actors together. Consequently, the notion of multi-stakeholder participation in governance assessments only occurs later in the evolution of governance assessments, and is a third wave attribute that is examined later in this chapter.

Second wave: aid conditionality and naming and shaming indices Second wave assessments are characterized by quantitative, multi-country, ranking, and aggregated/composite measures. The UNDP Governance Indicators: A Users' Guide (2nd edition, 2006a), presents a flourishing quantity of global indices that are used to 'name and shame' countries for their performance against a range of democracy and governance measures. Beyond naming and shaming, many of these indices are used as a basis for decisions on official development assistance and for access to other financial resources. Each year the US Millennium Challenge Corporation and the World Bank allocate billions of dollars of assistance using governance indicators. Bilateral donors are increasingly doing the same as well as new entrants, such as the Ibrahim Foundation, who give grants based on performance against predefined governance indicators. Some international organizations, such as the World Bank, play a leading role both in the supply and use of cross-country governance ratings. The European Commission is also a relatively new user of governance ratings with the introduction of its Governance Incentive Tranche for African, Caribbean and Pacific Countries, which supplements ratings with qualitative assessments. Arndt and Oman (2008) note that the last 15 years have seen a veritable explosion of interest in the quality of governance in the developing world and an equally significant increase in the use of quantitative governance indicators (Arndt and Oman, 2008).

Unlike the first wave of assessments (the datasets produced largely by academia), these indices are increasingly being built with the development community (mostly donors) in mind so that they can pluck out one or two indicators that will give them the information they need to quickly make decisions. They are also more usable than the first generation insofar as they integrate democracy, governance, and human rights concepts — all of which together, rather than individually, have become increasingly relevant to development practitioners. This group is not interested in democracy datasets based on procedural or minimalist definitions of democracy, but wants measures that are able to include all the economic, political, social, and cultural dimensions of governance, democracy, and human rights together. This has focused attention on the concept of governance and to questions of definition — how broad or narrow should the concept be defined? What attributes are most important? What are the critical dimensions or components of the attribute, etc.? Democracy and human rights, while still tricky concepts, can be anchored in theoretical frameworks, whereas governance is entirely 'under-theorized' and is often overloaded "where governance is all good things" (Grindle, 2004). This makes it even more of a challenge to develop an assessment around it. For example, the Worldwide Governance Indicators measure the perceived quality of six dimensions of governance for 213 economies, but are not based on a theoretical construct of what constitutes good or bad governance. Governance is, in a way, a catch all for everything good and it suffers greatly from concept validity. The normative aspect is important, as the Governance Indicators: A Users' Guide highlights: all assessments have a normative bias otherwise it is not possible to judge what is good or bad or if something is improving or deteriorating.

Many actors on the development stage began to broaden human rights, democracy, and governance concepts so that they merge into one people-centred meta-concept. For example, UNDP began to incorporate a political dimension in its public

3

The Democratization oof Governance Assessments

management work in the late 1990s and renamed the area Democratic Governance. This area includes human rights, governance (in all spheres), and promotion of democratization. Donors also began to develop instruments that focus on cross-cutting concepts, such as power to understand reform dynamics in which formal and informal sources of power are mapped and stakeholder interests and incentives are analyzed (World Bank, 2009).

Why did quantitative cross-country comparative indices become so popular? With the last decade's international consensus that governance matters for development and economic growth, the development community and donors in particular need broad brush instruments that will say something about what is happening on the ground on these subjects. It also became widely recognized that corruption in particular (a key governance concern), is a considerable obstacle to the delivery of effective development assistance. Furthermore, donors and, importantly, pro-reform citizen groups and non-governmental actors in developing countries demand tools that can be used as an incentive to improve the situation by naming and shaming and horse races (countries want to out-perform each other). Transparency International's Corruption Perceptions Index is an excellent example of this.

The Corruption Perceptions Index is an annual ranking of countries according to perceptions of corruption in the public sector. It is an aggregate index that combines different sources of information about corruption. It has played a huge role in raising awareness of the problem of corruption, of catalyzing dialogue with government, and of galvanizing support for measures to fight corruption. The index has great influence both in academic research and in anti-corruption efforts (Andersson and Heywood, 2009). It is a so called "index of indices", similar to the World Bank's Worldwide Governance Indicators. The Bank's approach to corruption measurement is similar to methodology applied by Transparency International, and the two indices correlate well (Sereide, 2006). These two instruments and the Freedom House Index enjoy huge popularity in their use and reference.

Criticisms of governance ranking/rating instruments both in academic and policy circles are widespread, and many users now have access to a growing information base that explains how these indicators should be used and what their limits are.

Furthermore, these kinds of indices have been challenged for normative, legitimacy, and methodological reasons: some governments in developing countries regard the indicators as biased, as pro-liberalization, pro-market and not relevant to understanding the governance, democracy, or human rights reality in their countries.

Two examples of the use of second wave indicators, the Millennium Challenge Corporation and the Ibrahim Foundation, are detailed in the following paragraphs to illustrate the use of second wave governance assessments and their characteristics (i.e., aggregate/composite measures based on a multidimensional concept of governance). The Millennium Challenge Corporation is governed by a CEO and a board of directors comprising the Secretaries of State and Treasury, the Administrator of USAID, the US Trade Representative, the CEO of the Millennium Challenge Corporation, as well as four non-governmental representatives.

It administers the Millennium Challenge Account, which is a bilateral US development assistance programme established by President Bush in 2002.

The Millennium Challenge Account is independent of all other agencies that administer US aid and is intended to be different from other US aid programmes in that it asks eligible countries to submit proposals based on national development priorities.

When the proposals are approved, the grantee government enters into a 'compact' with the Corporation that includes benchmarks for success. One of the aims of the Corporation's selection processes is to draw attention to and reward countries for democratizing reforms in that eligibility for Millennium Challenge Account funds serve as an incentive. The criteria for country eligibility is based on a number of cross-country measures of the quality of governance, including several constructed by the World Bank in its Worldwide Governance Indicators, including especially measures for voice and accountability.

The Millennium Challenge Corporation has three 'democracy indicators' in its eligibility criteria: 1. Political Rights (as measured by Freedom House, Freedom in the World), covering a country's performance on the quality of the electoral processes, political pluralism and participation, government function and accountability, transparency, and fair political treatment of ethnic minorities.

2. Civil Liberties (as measured by Freedom House, Freedom in the World), covering freedom of expression and belief, associational and organizational rights, rule of law, protection of human rights, personal autonomy, individual and economic rights, and the independence of the judiciary.

3. Voice and Accountability (as measured by the Worldwide Governance Indicators), covering the ability of domestic institutions to protect civil liberties, the extent to which citizens are able to participate in the selection of government, and the independence of the media.

Research undertaken by the Center for Global Development assert that there is a Millennium Challenge Corporation 'effect', in that the indicator-based country selection process has proven to be a powerful incentive for reform in candidate countries (Herrling et al., 2009). The brochures produced by the Corporation also speak of its impact. It states that Freedom House has begun reporting that "they have held discussions with a much wider range of countries since the Millennium Challenge Corporation began using their analysis as part of the selection process", and that "governments who have previously dismissed ratings as irrelevant or illegitimate have now requested meetings to learn more about the methodology and the issues driving their scores" (Millennium Challenge Corporation, 2007).

4

The Democratization oof Governance Assessments

The Mo Ibrahim Foundation was established by Dr. Mohamed (Mo) Ibrahim, founder of Celtel International, as an African initiative to recognize achievement in African leadership and to stimulate debate on good governance across sub-Saharan Africa and globally. The Foundation publishes the Ibrahim Index of African Governance and has an annual African Leadership Award, a monetary prize ($5 million) awarded to an African leader based on the findings of the Index. Nelson Mandela, former president of South Africa, provides one of the testimonials on the Mo Ibrahim Foundation's website and has this to say about the Foundation: "Mo Ibrahim has a vision to promote and recognize good governance that will drive Africa's political and economic renaissance. He has established the Mo Ibrahim Foundation to develop criteria for good governance, stimulate public debate and challenge the continent's leaders to set the global benchmark on this issue. Most specifically, the Foundation will provide a unique prize to recognize good governance by African leaders who have left office after a successful period in power. It is appropriate that this will be the largest prize in the world. Nothing is more important".

The Ibrahim Index of African Governance was first published in 2007 and calls itself "one of the most comprehensive assessments of governance in sub-Saharan Africa". It assesses the provision of public goods and services delivered to citizens by government and non-state actors. It consists of 84 indicators grouped into four main categories: 1. Safety and rule of law, i.e., personal safety (crime), rule of law, accountability and corruption, national security.

2. Participation and Human Rights, i.e., political participation, political rights and civil liberties, gender equality and women's' rights.

3. Sustainable economic opportunity, i.e., public management, private sector, infrastructure, environment and the rural sector.

4. Human development, i.e., health and welfare and education.

The Ibrahim Index is a composite and, like the Corruption Perceptions Index, is an index of indices, utilizing data from 23 external institutions. It goes much further than the Worldwide Governance Indicators and certainly Freedom in the World in its inclusion of social and economic indicators in its conception of governance. The problem with the Ibrahim Index is that by including indicators of both outcome and process, there is no way of differentiating between the two, or between cause and effect.2

Third wave: a focus on process The third wave includes assessments that emphasize process. The focus on process is made on the basis that an inclusive, transparent (i.e., more democratic) process will have positive transformative effects for local stakeholders. It sees the assessment process itself as an entry point for empowering local stakeholders through access to information and by providing a mechanism for engagement and participation in dialogue on governance and democracy issues in their countries.

The first and second wave assessments had little concern for the points of intersection between an assessment and local actors.

Assessments in these cases are based on data generated by external experts or qualitative studies undertaken by external researchers. There is limited interest in using or building on local capacities. Nor is assessment ownership a priority, especially in cases where those that are undertaking the assessment are using it for making their own decisions rather than viewing the assessment as a tool for local groups to use.

So, in contrast to the two previous waves, the third wave of governance assessments is characterized by: 1. 'Bottom-up', citizen-initiated approaches that emphasize dialogue and social accountability.

2. See Art Kraay's comments on the Ibrahim Index given at the Brookings Institution meeting "How to Rank Good Governance: The Mo Ibrahim Prize for Achievement in African Leadership" in Washington D.C on October 25 th, 2007

2. The application ofbasic principles ofthe rights-based approach (especially strengthening accountability relationships).

3. The influence of the aid effectiveness agenda with its emphasis on national ownership of development and capacity development.

A useful example of a process-focused approach is the State of Democracy assessment initiated by the International IDEA to strengthen the demand for good governance. This tool is based on a carefully thought through framework of qualitative and quantitative democracy indicators that citizens can use to examine their own governance systems. Its main purpose is to contribute to the process of democratization by raising public awareness about what democracy involves, and public debate about what standards of performance people should expect from their government; providing systematic evidence to substantiate citizens' concerns about how they are governed; contributing to public debate about ongoing reform, and helping to identify priorities for a reform programme. The assessment process is locally led and owned, as local researchers undertake the assessment which is then disseminated widely and the results subjected to local multi-stakeholder dialogue. Since 2000, the State of Democracy assessment has been used in a broad range of developing and developed countries including Bangladesh, El Salvador, Italy, Kenya, Malawi, New Zealand, Peru, South Korea, Australia, Bosnia-Herzegovina, Ireland, Latvia, Mexico, Chile, Mongolia, the Netherlands, the Philippines, Nepal, Bangladesh, Sri Lanka, India, and Pakistan.

Another example is the Urban Governance Index, supported by the United Nations Human Settlement Programme (UNHABITAT). The index aims to catalyse local action to improve the quality of urban governance, and measures local governance across five dimensions: effectiveness, equity, participation, accountability, and security. Local indicators are adapted by cities and

5

The Democratization oof Governance Assessments

their partners to respond directly to their unique contexts and needs. The Urban Governance Index promotes stakeholder participation and local ownership of the process to help ensure the data collected are locally relevant and used in decisionmaking. The assessment process has seven main steps, beginning with sensitizing local leaders on the importance of measuring progress in improved urban governance, developing locally appropriate indicators (i.e., definition, selection criteria, linkages with other indicators), defining benchmarks and targets, assigning scoring and weighting to the indicators and sub-indices and the proposed formulae for the local adaptation of the Urban Governance Index, field testing in cities, collecting data on a periodic basis (the data on the indicators is collected through stakeholder meetings where all key urban actors are present), and integrating findings in urban policy development.

Finally, two international programmes for supporting country-led assessments include the OECD supported Metagora programme and UNDP's Global Programme on Democratic Governance Assessments. Metagora was an international project focusing on bottom-up methods, tools, and frameworks for measuring human rights and democratic governance. It was launched in February 2004, under the auspices of the OECD/PARIS21, and concluded its operations in 2009. The programme made a significant contribution to local governance assessments through piloted innovative activities lead by local institutions and invariably engaging national statistics offices. Activities included a statistics-based assessment of the rights of indigenous peoples in the Philippines, a survey in Mexico of all forms of ill-treatment of persons by police bodies, a pilot survey in South Africa focused on the realization of democracy and human rights in the land reform process, coding, matching, and analysis of NGOs' data on human rights violations in Sri Lanka, and a database on the realization of the right to education conducted by the Palestinian Central Bureau of Statistics in collaboration with research institutions and NGOs (see also Chapter 13).

The UNDP Global Programme on Democratic Governance Assessments supports country-led, nationally-owned governance assessment initiatives in more than 30 countries, including Mongolia, Morocco, Nicaragua, Nigeria, Senegal, Malawi, Mexico, Indonesia, Egypt, and Chile. This programme is a significant catalyst of third wave assessments. It provides support to local entities for facilitating multi-stakeholder engagement on governance measurement, for capacity development initiatives related to governance measurement, and for initiatives that enhance the policy relevance of governance indicators (i.e., country contextualization).

Bottom-up approaches Third wave governance assessments stress a bottom-up approach. This is an alternative methodology that begins with the local population's most pressing governance and democracy needs and concerns. Then local citizens assess how well those needs and concerns are being met. This is the demand side of democratic governance, in that the approach focuses on the voice and capacity of citizens to directly demand greater accountability and responsiveness from public officials and service providers. At the centre of this idea is social accountability, a concept that relies on civic engagement in which ordinary citizens and/or civil society organizations participate directly or indirectly in holding officials to account (Malena et al., 2004). Social accountability is a close relative of the rights-based approach, because the obligation of government officials to be accountable to citizens derives from an idea of citizens' rights, such as the right to official information, to freedom of assembly, and freedom of expression.

Furthermore, the concept of social accountability underlines the right of citizens to expect and ensure that government acts in the best interests of the people (World Bank, 2004).

It is the participation of citizens that distinguishes social accountability from conventional mechanisms of accountability. In many cases, citizens, communities, and civil society organizations do not merely participate in social accountability activities, but initiate and control them (as with the International IDEA's State of Democracy assessment). While many participatory approaches focus exclusively on the individual community or at the micro-level, social accountability mechanisms expand opportunities for participation at the macro-level. This may include, as Chapters 11 and 12 demonstrate, citizen involvement in the analysis and/or formulation of national or local budgets or linking the findings of local level participatory monitoring and evaluation exercises to budgetary, administrative, or governance issues at higher levels of the public service delivery chain (World Bank, 2004). These social accountability instruments are governance assessments in themselves, but also represent important information sources for broader and more comprehensive governance assessment initiatives. The key contribution of social accountability tools and techniques is in providing citizens with access to information. But there are important prerequisites for effective use of social accountability mechanisms, notably an active civil society that has the political space and resources to assist citizen action, and a vibrant media that can (and is willing to) work with NGOs and citizens groups in communicating governance deficits.

Turning now to the rights-based approach, this deals with the monitoring not only of outcomes, but also process and is essentially concerned with the accountability relationship that exists between duty-bearers, (state and government actors, including parliaments, ministries, local authorities, judges, police, and teachers), rights-holders (individuals, citizens, citizen groups, and non-government actors) and the capacities or lack thereof between those parties in fulfilling obligations and asserting rights (OHCHR, 2003).3 It also emphasizes that it is impossible to evaluate development without first paying attention to how it was accomplished (Decker at al., 2005).

Rights-based approaches require active and meaningful participation in the assessment process by communities, civil society, vulnerable groups such as indigenous peoples, women and others. Particular attention is given to addressing discrimination, ensuring equality, equity, and participation of vulnerable groups in the assessment process and collection of disaggregated data.

3. OHCHR (2003): What is a rights-based approach to development?

Disaggregated data is of huge importance for governance assessments. Development data need to be disaggregated, as far as

6

The Democratization oof Governance Assessments

possible, by race, religion, ethnicity, language, sex, and other aspects. First and second wave governance assessments provided little or no disaggregated data for vulnerable groups, whereas third wave governance assessments attempt to capture and reflect the potentially different impacts that the mechanisms and processes of governance have on various social groups. It is even possible to capture sensitivity to vulnerable groups beyond using the same but disaggregated governance indicators for vulnerable groups as the rest of the population, by choosing governance indicators that measure a governance practice specifically targeted at a vulnerable group (e.g., percentage of prisons that have special facilities for pregnant prisoners) or having vulnerable groups themselves indicate, through participatory techniques (e.g., survey methods), the governance issues that are of special interest to them (UNDP, 2006b).

However, the rights based approach's usefulness for improving and shaping the evolution of how governance assessments are done exists more in potential than realization. Nevertheless, the basic principles of a rights based approach have already had a significant impact on the UNDP programme of governance assessments, especially in its emphasis on multi-stakeholder participation, disaggregated data for vulnerable groups, and capacity development.

Aid effectiveness: national ownership and capacity development While the main emphasis in recent years has been on strengthening bottom-up approaches, this chapter would be incomplete without a reference to the changes that have taken place also at macro level, and specifically in the way donors approach their partners in recipient countries. Most important is the effort to improve aid effectiveness, as enshrined in the Paris Declaration on Aid Effectiveness (2005) and, more recently, through the Accra Agenda for Action (2008). These two declarations have ushered in a new approach by donors to governance assessments, particularly in sharpening commitments to country ownership and alignment with national systems. This agenda aims to improve the quality of the delivery, management, and use of official development assistance to maximize its development impacts. The agenda proclaims that 'ownership of development' by developing countries is a prerequisite for development success, and argues for a shift in the way donors support development through aligning programmes with country policies and systems, and to apply programmatic instruments, such as budget support (Meyer, 2009). The five basic principles of the Paris Declaration are: 1. Ownership. Partner countries exercise effective leadership over their development policies and strategies and co-ordinate development actions.

2. Alignment. Donors base their overall support on partner countries' national development strategies, institutions and procedures.

3. Harmonization. Donors' actions are more transparent, collectively effective and harmonized with each other.

4. Managing for Results. Managing resources and improving decision-making with a focus on results.

5. Mutual Accountability. Donors and partners are accountable for development results.

Moreover, the Accra Agenda for Action reinforced donor commitments made in the Paris Declaration to national ownership, alignment, and harmonization with national development plans, and capacity development. It went further by calling on donors to "jointly assess the quality of country systems in a country-led process using mutually agreed diagnostic tools" (paragraph 15c).

During a two-day conference in London in 2009, governance specialists from development agencies, civil society, and academia discussed why, how, and when donors assess governance in partner countries, and the impact of these assessments. The conference sought to take stock of present practices, stimulate mutual learning, and find ways to cooperate so that assessments are better used by in-country stakeholders working on improving governance. The conference discussed a review of donor approaches to assessments undertaken in 2008. The review identified 11 development agencies having 17 general governance assessment tools in use, plus three under development, and stressed the risk of frequent duplication and overlap between governance assessment tools, along with the need to improve practice with regard to greater reliance on partner country assessment processes (OECD, 2008).

In response to the London meeting and the donor review, the OECD produced 'guiding principles' to improve donor impact, usage, and harmonization of governance assessments. The principles are intended to provide practical advice to governance specialists in donor agencies working on assessments and to donor agency staff working in other sectors who encounter the realities of governance in their work every day (OECD, 2009). They are: •

Build on and strengthen nationally driven governance assessments.

•

Identify a clear key purpose to drive the choice of assessment tools and processes.

•

Assess and address governance from different entry points and perspectives.

•

Harmonize assessments at country level when the aim is to stimulate dialogue and governance reform.

•

Make results public unless there are compelling reasons not to do so.

Many third-wave governance assessments have been fostered by UNDP, the International IDEA, and the New Partnership for Africa's Development's African Peer Review Mechanism at the international level. These complement the spread of local level smaller scale social accountability initiatives. They are characterized by citizen/country-driven assessment initiatives focused on utilizing national capacities and creating dialogue. The donor contribution to third wave assessments lies in its commitment to harmonize its own assessments with country led ones and to support indigenous assessment processes as much as it can. In the light of the Paris and Accra resolutions, however, donors are using the rhetoric of country ownership of assessments, but there

7

The Democratization oof Governance Assessments

remain obvious tensions in reconciling these prevailing development principles (participation, national ownership, capacity development) with an efficient appraisal of democratic governance. In many cases, donors are paying more attention to national consultation in their governance assessment activities, encouraging input to the assessment process, which is different from ownership in which local actors drive the process.

Conclusions The three waves of governance assessment that have been discussed in this chapter reflect in large part the shifts that have taken place in the governance concept itself. Many lessons have been learned and carried over to the next wave.

A conference of democracy indicator experts from academia was convened in Berkeley, California, in 2009 to assess the current state of democracy assessments and governance indicators and to develop an agenda for constructing better indicators. Two 'relevance' challenges were highlighted at the conference. One was a legitimacy challenge related to the need for greater transparency in measurement, i.e., who is doing the measuring and who is funding the research. The other was the challenge of contextual relevance related to the need of the development and policy making community to have indicators for the subnational level as well as measurement approaches that can highlight sub-national trends and differences, i.e., contextually specific indicators. This shows how much academics have come to realize the importance of being able to devise methods that speak not only to what is theoretically interesting, but also to what is practically valuable.

A second lesson that has been learned, if not fully embraced, is that the results-based management approach with its grounding in logical framework analysis is ill-suited for assessing the democracy or governance situation of a country at any level. Crawford argues that the logical framework "produces a narrow project-based, mainly quantitative, analysis, insufficiently grounded in the complexities and nuances of the political context in which such assistance take place" (Crawford, 2003). Another critic is Tom Carothers, who argues that it uses "highly reductionist indicators and is a deeply flawed undertaking that is consuming vast resources, producing little useful insight or knowledge, and introduces distortions into the design and implementation of aid" (Carothers, 1999). In response to this kind of criticism, there has been a gradual shift towards recognizing the importance of more contextualization in assessing governance.

A third lesson is reflected in the shift from cross-country comparisons based on global data sets towards country-led assessments involving local stakeholders. Much of this mirrors the growing influence of civil society organizations and citizens activists, but is also a consequence of the limited policy value provided by these global comparisons. The World Bank itself has cautioned that its governance indicators have substantial margins of error, so that careful empirical scrutiny is needed for borderline cases where the risk of misclassification is especially high (Kaufmann and Kraay, 2002).

Overall, it is appropriate to conclude that the task of assessing governance has moved over a significant intellectual and programmatic territory in the past 20 years. It has clearly moved in the direction of making democratic governance more practical and participatory. This is evident in the various case study contributions that form the main empirical part of this volume.

References 1. Staffan Andersson and Paul M. Heywood, "The Politics of Perception: Use and Abuse of Transparency International's Approach to Measuring Corruption", in Political Studies, vol. 57, Issue 4, (December 2009), 746-767.

2. Christianne Arndt and Charles Oman, "The Politics of Governance Ratings", Draft, (2008).

3 Thomas Carothers, "Democracy Assistance: Political vs. Developmental?" in Journal of Democracy, vol. 20, no. 1, (January 2009), 5-19.

4. Thomas Carothers, Aiding Democracy Abroad: The Learning Curve, (Washington D.C.: Carnegie Endowment for International Peace, 1999).

5. Jose Antonio Cheibub, et al., "Democracy and Dictatorship Revisited", unpublished draft, (2009).

6. Gordon Crawford and Iain Kearton, Evaluating Democracy and Governance Assistance, (United Kingdom Centre for Development Studies, Institute for Politics and International Studies, 2002).

7. Gordon Crawford, "Promoting Democracy from Without - Learning from Within (Part I)", in Democratization, vol. 10, Issue 1, (Spring 2003), 77-98.

8. Merilee Grindle, "Good Enough Governance: Poverty Reduction and Reform in Developing Countries", in Governance, vol. 17(4), (2004) 525-548.

9. Sheila Herrling, et al., From Innovation to Impact: Next Steps foor the Millennium Challenge Corporation, (Washington D.C.: Center for Global Development, January 2009).

10. Samuel P. Huntington, The Third Wave: Democratization in the Late Twentieth Century, (Norman, OK: Oklahoma University Press, 1991).

11. Daniel Kaufmann and Aart Kraay, "Governance Indicators, Aid Allocation, and the Millennium Challenge Account". (World Bank Draft Paper, December 2002).

12. John Keane, The Life and Death of Democracy, (New York and London: Simon and Schuster, 2009).

8

The Democratization oof Governance Assessments

13. Todd Landman, Concepts Matter: Delineating Democracy, Governance and Human Rights. (Stockholm: International Institute for Democracy and Elections Assistance, 2009).

14. Stefan Meyer, "Governance Assessments and Domestic Accountability", in Feeding Domestic Debate and Changing Aid Practices, FRIDE Working Paper 86, (Madrid, June 2009).

15. Millennium Challenge Corporation, The Millennium Challenge Corporation and the Long Term Goal of Deepening Democracy, (November 2007).

16. L. Gerado Munck, Measuring Democracy: A Bridge Between Scholarship and Politics, (Baltimore MD: John Hopkins University Press, 2009).

17. Office of the High Commissioner for Human Rights, What is a Rights-Based Approach to Development? (Geneva, 2003).

18.

OECD, Survey of Donor Approaches to Governance Assessments, (Paris, 2008).

19.

OECD, Donor Approaches to Governance Assessments: Guiding Principles for Enhanced Impact, Usage and Harmonization, (Paris, 2009).

20.

21.

Tina Soreide, Is it Wrong to Rank? A Critical Assessment of Corruption Indices, Chr. Michelsen Institute Working Paper. (Bergen, 2006).

UNDP, Governance and Sustainable Human Development. (New York, 1997).

22.

UNDP, Governance Indicators: A Users' Guide (2nd edition). (UNDP Oslo Governance Centre, 2006a).

23. UNDP, Measuring Democratic Governance: A Framework for Selecting Pro-Poor and Gender-Sensitive Indicators, (UNDP Oslo Governance Centre, May 2006b).

24. UNDP, Fostering Social Accountability: From Principle to Practice, UNDP Practical Guidance Note. (UNDP Oslo Governance Centre, August 2010).

25. World Bank, Citizen Report Card Surveys: A note on the Concept and Methodology in Social Development Notes: Participation and Civic Engagement, Note No. 91, (February 2004) 26. World Bank, Problem-Driven Governance and Political Economy Analysis, Draft Paper, (April 2009)

9

The Democratization oof Governance Assessments

10