Measuring Governance and Service Delivery in Safety Net Programs

S P D I S C U S S I O N PA P E R

Abstract This paper develops a framework to assess organizational performance in the delivery of social safety nets. Specifically, it provides guidance to task teams and program managers for identifying indicators of governance and service quality in targeted cash transfer programs.

The paper identifies governance issues along the results chain of service delivery and suggests policy and performance indicators for assessing program inputs - human resources, financing and resource management; and program activities - operational procedures, MIS and control. It also suggests indicators of organizational performance and the quality of outputs, including demand-side accountability mechanisms.

HUMAN DEVELOPMENT NETWORK

About this series...

Social Protection Discussion Papers are published to communicate the results of The World Bank’s work to the development community with the least possible delay. The typescript manuscript of this paper therefore has not been prepared in accordance with the procedures appropriate to formally edited texts. The findings, interpretations, and conclusions expressed herein are those of the author(s), and do not necessarily reflect the views of the International Bank for Reconstruction and Development / The World Bank and its affiliated organizations, or those of the Executive Directors of The World Bank or the governments they represent. The World Bank does not guarantee the accuracy of the data included in this work.

For more information, please contact the Social Protection Advisory Service, The World Bank, 1818 H Street, N.W., MSN G7-703, Washington, D.C. 20433 USA. Telephone: (202) 458-5267, Fax: (202) 614-0471, E-mail: socialprotection@worldbank.org or visit us online at www.worldbank.org/sp.

Measuring Governance and Service Delivery in Safety Net Programs Gloria M. Rubio

September 2011

NO. 1119

Measuring Governance and Service Delivery in Safety Net Programs

Gloria M. Rubio1 September 2011

Human Development Network The World Bank

1

This paper was jointly commissioned by the Knowledge and Learning Program on Measuring Governance and Service Delivery at the HD Chief Economist’s Office and the Social Protection Strategy and Results Group. Dena Ringold and Laura Rawlings provided valuable guidance for this work. Irvin Rojas (COLMEX) provided important research assistance by collecting and reviewing information on governance assessment tools. Emil Tesliuc and Lucian Pop provided helpful comments and Rodrigo Quintana helped edit and format and gave useful inputs.

1

Abstract This paper develops a framework to assess organizational performance in the delivery of social safety nets. Specifically, it provides guidance to task teams and program managers for identifying indicators of governance and service quality in targeted cash transfer programs. The paper identifies governance issues along the results chain of service delivery and suggests policy and performance indicators for assessing program inputs ─ human resources, financing and resource management; and program activities ─ operational procedures, MIS and control. It also suggests indicators of organizational performance and the quality of outputs, including demand-side accountability mechanisms.

JEL Classification: H11, H83, I38 Keywords: governance, service delivery, safety nets, cash transfers, service quality indicator, performance monitoring

2

Table of Contents

Introduction .......................................................................................................................... 4 1. Governance and social protection .................................................................................. 5 2.1 Governance in service delivery ................................................................................................. 5 2.2 Tackling error, fraud and corruption ......................................................................................... 6 2.3 International development approach to governance measurement ....................................... 7

2. Framework for Measuring Governance and Service Delivery in Cash‐based Safety Nets . 8 3.1 Results Chain in Cash Transfer Programs .................................................................................. 8 3.2 Where Does Governance Come into Play? ............................................................................. 10 3.3 Measuring Governance and Quality of Service Delivery in Safety Nets Programs ................. 11 3.4 How is governance different in cash transfers than health and education? .......................... 12

3. Identifying Governance and Service Quality Indicators for Safety Nets Programs ......... 13 4.1 Human resources .................................................................................................................... 13 4.2 Financing and Resource Management .................................................................................... 16 4.3 Operational procedures, information management and control mechanisms ...................... 19 4.4 Organizational Performance of the Delivery of Cash Transfers .............................................. 23 4.5 Quality of Social Assistance Service Delivery .......................................................................... 24 4.6 Demand‐side accountability in service delivery ...................................................................... 27 4.7 Data Collection ........................................................................................................................ 31

4. Conclusions .................................................................................................................. 32 5. References ................................................................................................................... 33 Annexes .............................................................................................................................. 39 Annex 1. Overview of governance and social protection assessment tools ................................. 39 Annex 2. List of indicators related to social protection issues ...................................................... 51

3

Introduction

Making services work for poor people involves a number of challenges including strengthening the governance of public sector institutions. Specifically, this requires setting the right set of incentives and accountabilities to affect the way provider organizations, their managers, and staff behave, as well as the quality and efficiency with which they deliver services. Thus, how providers are selected, paid, monitored, and held accountable for their performance has gained increased relevance in the service delivery reform debate.

In recent years the international development community has made efforts to define and measure governance in public service provision. It has sought to generate and systematize empirical evidence on the links between service delivery quality and governance. Although this work has focused mainly on health and education, governance and service quality are also relevant in social protection. For example, efforts to strengthen social assistance targeting, administration and service quality can benefit from a conceptual framework to help diagnose, monitor and evaluate the performance of institutions in charge of social assistance services, including their incentives and accountabilities, as well as service quality and efficiency.

As a first step towards measuring governance in social protection, this paper develops a framework to assess organizational performance in the delivery of social protection services with a focus on cash-based social safety nets (social safety nets). In particular, this paper aims to identify a set of governance and service quality indicators that can be applied to targeted safety net programs.

The paper is structured as follows. The first section summarizes existing literature on governance and service delivery relevant for social protection. The second section develops a conceptual framework to measure governance and service quality in the delivery of cash benefits. The next section discusses a potential list of governance and service quality indicators for social safety net programs as well as data collection options. Finally, the last section presents the conclusions.

4

1. Governance and social protection

Governance has been the subject of multiple definitions and applications. There is a vast literature on governance applied to a variety of fields including corporate management, public sector management and international development. More recently, the governance dimension has entered the analytical work and policy debate on human development, particularly in relation to education and health care service delivery and to a lesser extent, social protection. As a result, the literature on assessing the performance of service delivery in social protection is still limited.

This section aims to start filling this gap by drawing from lessons learned more broadly on human development and public sector management, and apply them to the delivery of social safety nets. This section briefly touches upon three areas of governance relevant for social protection in general: (i) governance in service delivery; (ii) tackling error, fraud and corruption; and (iii) governance measurement in international development.

2.1 Governance in service delivery Governance can be defined as “the set of incentives and accountabilities that affect the way provider organizations, their managers and staffs behave, as well as the quality and efficiency with which they deliver services” (Fiszbein, et al., 2011). Recognizing that poor governance and service quality in the delivery of social services is as relevant to achieve human development outcomes as funding has led policymakers and academics to better understand and seek to influence the performance of public service providers (World Bank, 2003a; Brinkerhoff and Bossert, 2008, Savedoff, 2009; Lewis and Pettersson, 2009a and 2009b; Fiszbein, et. al., 2011). In particular, Fiszbein, et al. propose a framework for the selection and measurement of governance and service delivery indicators for health and education.

This framework distinguishes between two measurable aspects of governance: policies and performance. First, the set of policies, or rules of the game, that influence the ways that service providers function. These are the incentive and accountability arrangements that frame service provision. Second, governance materializes as performance – the actual practices and behaviors of providers resulting from policies. Governance performance is measured by the extent to which the governance policies work in practice. Governance policies and performance in turn influence the quality of service delivery.

5

The framework includes five areas in which governance issues play a critical role in health and education: human resources; financing and resource management; procurement and management of critical inputs (textbooks and pharmaceuticals); information; and provider entry.

2.2 Tackling error, fraud and corruption Governance in social protection has been addressed primarily from the operational and institutional perspective of tackling error, fraud and corruption (World Bank, 2007, and Stolk and Tesiluc, 2010). This approach categorizes governance risks related to the misdirection of public funds into three types of problems: error, fraud and corruption (EFC). Error refers to unintentional mistakes of various natures, while fraud and corruption refer to intentional attempts to exploit the system by benefit claimants and program staff respectively.

Safety net programs have inherent design features such as complex eligibility requirements and changing eligibility over time which pose a particular challenge for controlling error, fraud and corruption (EFC). Eligibility is often based on the economic position of the household, which is difficult to assess and hard to verify. Moreover, eligibility may change over time as households’ poverty status varies. Evidence from developed countries suggests a four-pronged strategy to control EFC: prevention, detection, deterrence and measurement (Stolk and Tesliuc, 2009).

A number of actions considered in the strategy can be mapped to the governance framework discussed above. In particular, the EFC strategy emphasizes the need for developing appropriate administrative procedures or rules, including a clear delineation of procedures in the processing and verification of benefit claims and clarity in the allocation of institutional responsibilities across the different levels of government or between government and civil society. Additionally, it suggests the adequate use of IT systems and staff training to help in reducing human error and improve performance in following procedures.

Moreover, the generation of intelligence to detect cases of EFC through data matching and telephone or online grievances and complaints mechanisms, as well as, established review processes of benefit claims, and sanctions are part of the incentive arrangements to deter fraud and corruption. Finally, information campaigns aimed at raising awareness of the rights and obligations under the rules of the social safety nets programs can create social control and accountability mechanism over program implementation. These recommendations coincide with the findings of a review of conditional cash transfer (CCT) programs in Latin American countries which pointed to the need for a better definition of institutional responsibilities; adequate implementation of management information systems; and thorough control and accountability systems (World Bank, 2007).

6

2.3 International development approach to governance measurement Governance has gained importance in the international arena and has evolved from a narrow focus on public sector management to a broader concern for interconnected issues, including the role of formal and informal institutions, security, human rights, and corruption. As a result, bilateral and multilateral development agencies, among other organizations, have developed a number of governance assessment tools (UNDP, 2007 and OECD, 2008 and 2009).

These tools broadly serve three sets of objectives: (i) strategic decisions of donors regarding aid allocation and risk management; (ii) donor-client policy dialogue and joint decisions on aid allocation, performance assessment and operational set-up; and (iii) the promotion and design of domestic governance reform processes. Most of these instruments focus on political systems and public administration. Some address social governance issues including pro-poor spending and access to, and effectiveness of, service delivery.

Annex 1 presents an overview of governance assessment tools by focus areas. It identifies the tools addressing social protection issues, and distinguishes those dealing with broader public sector management topics such as financial management, human resource management (HRM), transparency, accountability and corruption that can be adapted to analyze social protection performance.

Similarly, there are a few initiatives that systematize information on the performance of social protection systems2. Although these are not explicitly used as governance assessment tools, they include some indicators that may be useful for measuring service quality. A number of these initiatives are discussed below. Annex 2 lists all of the indicators in the governance assessment tools related to social protection. 

International Labor Organization Social Security Database offers information on compulsory cash benefit programs (e.g. sickness, maternity, old-age, disability, family allowance, work injury and unemployment) for 124 countries. It generates a statistical database on coverage, contribution rates, expenditures, legal framework, means-tested conditions for benefits, benefit size and duration, and waiting period.



Social Protection Expenditure and Performance Reviews (SPERs) are countrylevel analysis of social protection systems, expenditures, financing and performance in terms of coverage, distributional aspects and administrative efficiency.



European Commission, Mutual Information System on Social Protection provides comparable and regularly updated information on social protection systems legislation in 31 European countries. It includes charts and descriptions of

2

Some of these initiatives originate in Europe´s standard‐setting and coordination efforts of social security systems.

7

social protection systems organization, as well as comparative tables of statutory regulations of core areas (health care, sickness, maternity/paternity, invalidity, oldage, survivors, work injuries and occupational diseases, family allowances, unemployment benefits, minimum protection, long-term care and financing)3. 

Eurostat – European System of Integrated Social Protection Statistics (ESSPROS) is a database using a common framework which enables international comparison of the administrative national data on social protection benefits and financing across European countries.

In sum, governance lenses have not been systematically applied to social protection.

Nonetheless, there are valuable inputs and lessons from different areas in the literature4.

The next section aims at contributing to the knowledge of social protection governance by proposing a measurement framework for cash-based social safety net programs.

2. A Framework for Measuring Governance and Service Delivery in Cash­based Safety Nets Social protection encompasses a range of different types of policies and programs, including social insurance, labor markets, and social safety nets. Although they share a number of common features in their administration, differences in benefits and eligibility criteria exist. Hence, social protection policy instruments involve different operational procedures and corresponding organizational challenges and require tailored governance approaches. As a starting point, the framework developed in this section will focus on cash benefits. In the future, this framework could be further adapted to other types of social safety nets (e.g. social services) and social protection programs.

3.1 Results Chain in Cash Transfer Programs Results chains depict the theory of change behind a program. They describe a logical, plausible pathway of how a sequence of input, activities and outputs ─ which an implementing agency is directly responsible for─ interacts with behavioral processes and external factors to achieve a particular outcome. Results chains help associate what is being done with intended and actual outcomes. They therefore help trace and track program

3

A similar initiative by the Council of Europe is the Mutual Information System on Social Protection of the Council of Europe (MISSCEO). Following the same structure as MISSOC, it produces comparatives tables on social protection systems legislation in 15 states: Albania, Armenia, Azerbaijan, Bosnia and Herzegovina, Croatia, Georgia, Moldova, the Russian Federation, Serbia, “the former Yugoslav Republic of Macedonia”, Turkey and Ukraine as well as Australia, Canada and New Zealand.

4 Other areas in the literature worth exploring in future reviews include research in management – performance management and organizational effectiveness, and services marketing.

8

implementation, link specific program outputs to specific inputs and evaluate whether the program is achieving its intended results.

Figure 1 illustrates a simplified traditional results chain for a cash assistance program. In this case, the links of the results chain involve the following: -

Inputs: include staff from the ministry, institution and/or local government responsible for operating the program; the budget allocated to the program; and the central and local facilities where the program operates.

-

Activities: comprise all operational procedures required for program implementation, including outreach and application, beneficiaries’ selection and enrollment, payment delivery; and periodic eligibility reassessment.

-

Outputs: are the cash transfers delivered.

-

Outcomes: are the ultimate program objectives. For social safety nets programs, the main outcomes are improved household consumption in the short term, and poverty reduction in the long term. They are influenced not only by program outputs but also by other factors like the economy and individual behaviors (foregone income, crowding-out of other transfers, etc.).

Figure 1: Traditional Results Chain for a Cash Delivery Social Assistance Program

Source: Author

9

3.2 Where Does Governance Come into Play?

Governance enters the results chain as an enabling cross-cutting factor first through the availability and performance of inputs, as well as their transformation into outputs through program activities (see Figure 2). Secondly, governance policies and performance influence the quality of service delivery – defined here as program accessibility and the quality of the client and service provider interaction and transactions. This explicit emphasis on the service quality dimension of outputs represents a “missing middle” in the traditional results chain.

Service delivery is a function of supply and demand side factors5. On the supply side, governance policies and performance shape the way organizational providers6 materialize their mandate and manage program inputs and activities to generate outputs and reach the target population through frontline staff.7 Service delivery is also influenced by demandside factors. Through direct influence on service providers and indirect pressure on politicians and policymakers8, citizens and beneficiaries can demand changes in the delivery of a program. Governance policies and performance on access to program information, choice, participation, and independent assessment also have an effect on the demand-side accountability of service delivery.

5

There are also external factors such as technology that influences service quality.

Organizational providers are public, private nonprofit, and private for‐profit entities that actually provide services. In the case of social assistance services, most provider organizations will be within the public sector, for example, a ministry, department, agency or bureau.

7 Following the service provision framework of the 2004 World Development Report (WDR), on the supply side there are two accountability relationships: the compact relationship between organizational providers and the state, and the management relationship between frontline professionals and the organizational provider. Policymakers make resources available and delegate powers and responsibility for collective objectives to service providers. In turn, organizational providers assign frontline workers with tasks and delineated areas of responsibility, furnishing them with the resources to act.

8 “Clients may seek to hold service providers accountable for performance in two ways. Client power connecting clients and providers is the direct, “short” route of accountability. When such client power is weak or not possible to use, clients must use voice and politics in their role as citizens to hold politicians accountable—and politician/policymakers must in turn use the compact to do the same with providers. The combination of the two is the roundabout, “long” route of accountability.” (World Bank, 2003a) 6

10

Figure 2: Governance in Social Assistance Programs

Source: Author

3.3 Measuring Governance and Quality of Service Delivery in Safety Nets Programs Governance can be measured along two dimensions: policies and performance. Governance policies are the rules of the game – both formal/de jure and informal/de facto – that set the incentive and accountability arrangements under which program stakeholders operate.

These can involve national or local level policies, and laws and regulations applicable to the public sector at large, such as budget or civil-service laws. They can also entail program specific rules such as operational manuals, administrative procedures or job descriptions.

The distinction is important as they imply different strategies and stakeholders for the policy dialogue.

Governance performance refers to how the governance policies work in practice - that is, the organizational behavior and practices resulting from the existence and application of a set of rules. Staff absenteeism is an example. It reflects the application of governance policies in human resource management involving a meritocratic civil service, adequate compensation scheme, performance evaluation, and sanctions and rewards system.

11

3.4 How is governance different in cash transfers than health and education?

Cash transfers are transaction-intensive programs that require frequent client-provider contact. In contrast to classroom teaching or curative health, they involve overall better defined and comparatively easier to systematize activities. Examples of transactions include, but are not exclusive to: (i) beneficiary application and selection to the program; (ii) benefit delivery; (iii) grievance and redress mechanisms.

Figure 3 takes a closer look at program activities or operational processes involved in delivering cash transfers. In addition to the implementation steps, there are two crucial cross-cutting components for program operation: management information system (MIS) and control mechanisms. Responsibilities along the implementation process are often shared across organizations and government levels.

The program results chain and the more detailed operational procedures can help identify critical areas for assessing governance in safety nets programs. They refer to program inputs ─ human resources, financing and resource management, and program activities ─ operational procedures, MIS and control mechanisms. These elements should be considered in relation to the organizational performance (intended versus actual) in cash transfer delivery –outputs– and the quality of social assistance services, including demand-side accountability mechanisms in safety nets.

Figure 3: Social Assistance Programs Activities IMPLEMENTATION PROCESSESES

COVERAGE STRATEGY AND PROMOTION

APPLICANTS’ REGISTRATION

BENEFICIARY SELECTION & VERIFICATION

BENEFICIARY ENROLLMENT

PAYMENT

RECERTIFICATION

MANAGEMENT INFORMATION SYSTEMS AND CONTROL MECHANISMS

Source: Author

12

3. Identifying Governance and Service Quality Indicators for Safety Nets Programs The selection of any particular set of indicators has underlying normative assumptions.

There is some evidence from OECD countries on which governance interventions are costeffective measures for reducing EFC (Stolk and Tesiluc, 2010). However, not all governance interventions have a proven impact on organizational performance including service quality, or program outcomes. These indicators should be treated as hypothesis to be tested rather than evidence to follow (Savedoff, 2011).

Existing tools for assessing governance in social protection systems include a number of indicators to draw from. In particular, human resources and financial management governance indicators have been extensively used in the past. Available sources could be used to identify relevant indicators and evaluation criteria. However, most tools are geared towards general public sector performance measurement. Thus, specific data collection for indicators tailored for safety nets will be required.

As explained above, governance and service quality assessment in safety nets programs involves measuring indicators in six areas: a. b. c. d. e. f.

Human resources Financing and resource management Operational procedures, information management and control mechanisms Organizational performance in cash transfers delivery Service quality9 Demand-side dimension of service delivery

4.1 Human resources Social assistance programs involve a number of interactions between frontline staff and beneficiaries, including program outreach and application process, eligibility assessment, applicants’ information verification, beneficiary enrollment, and benefits payment. The rules of the game for hiring, compensating, training, evaluating and rewarding program staff will hence shape their incentives, which in turn, may influence their work performance in providing efficient and client-oriented services10.

9

Although service quality is one of the dimensions of organizational performance, it will be addressed separately to highlight its importance.

10 The impact of human resources management on organizational performance has been studied mostly in the private firms’ context. There is no consensus on the robustness of the relationship between HR systems and firms’ performance, and more work is needed on measuring organizational performance (Rogers and Wright, 1998). The links between HRM governance policies and public organizations performance are still

13

Table 1 shows the main issues in assessing governance in human resource management (HRM) and the corresponding policy and performance indicators. Performance measurements may be associated to more than one governance policy in human resources or in other areas. For example, staff absenteeism may be associated with employees’ motivation and accountability framework determined by retention and compensation policies, as well as with performance evaluation and control mechanisms. As indicated next to each indicator, some are included in existing assessment tools such as the WB Actionable Governance Indicator Initiative (AGI), the Country Performance and Institutional Assessment (CPIA), or the US Anti-Corruption Assessment Framework (ACF). These indicators usually refer to the overall civil service, but can be adapted to gauge human resources performance in a particular sector or program. Annex 1 presents detailed information on the assessment tools, including references to consult the rationale and evaluation criteria for each indicator.

Interestingly, the main focus of available indicators is on hiring, promotion, and compensation practices, and less so on training and delineation of responsibilities. EFC literature, however, recognizes the role of the latter as part of the prevention initiatives.

Thus, for social safety nets programs is important to complement existing HR indicators with measurements of staff knowledge and application of procedures and responsibilities.

being studied, thus indicators discussed in this section should be considered as part of hypotheses to be tested.

14

Hiring, promotion

Issue retention

Duties/Responsibilities

Compensation

Training

Evaluation Rewards/Sanctions

and

Table 1. Human Resources Management Governance Indicators in Safety Nets Programs Policy Indicators Performance Indicators - Average number of qualified (long-listed) candidates per advertised - Quarterly staff turnover rates [AGI] program openings [AGI] - Monthly average staff absentee rate [AGI] - Average 3-year turnover rates among recent recruits by types of position (e.g. Titles, urban vs. urban locations) [AGI] - % of program vacancies filled through advertised, competitive procedures - Ratio of average total remuneration for staff in the highest rank to [AGI] average total remuneration for staff in entry level position [AGI] - Degree in which hiring and promotion are based on merit and - Perception of political influence in staff promotion, transfer and/or performance, and ethical standards prevail [CPIA] appointment [GAC] - Existence of a legislative framework for the civil service regulating recruitment, job security and retention [ACF] - % of staff for whom annual performance evaluations were completed on - Existence of clear rules regarding tenure [ACF] time [AGI] - Existence and clarity of job descriptions/responsibilities and - % of staff performance evaluations falling in each rating category [AGI] administrative procedures - Existence of conduct code or equivalent [ACF] - % of staff receiving the lowest performance rating in two successive - Existence job performance standards years who have left the program within the following year [AGI] -Average staff remuneration as a % of average economic sector wages [AGI] - Degree of correspondence between program staff skills and job profiles - Ratios of average staff to private sector total remuneration by title (e.g. % of staff whose skills match their job profile) [AGI, ROSC] - Degree of flexibility in paying more attractive wages in hard to fill - Level of staff proficiency and compliance with administrative procedures positions (e.g. rural areas, technical specialists) [CPIA] - Degree of correspondence between standard and actual staff workloads - Extent to which pay and benefit levels do not deter talented people from entering the public sector [CPIA] - Perceived effectiveness of the training opportunities offered by - Level of adequacy of civil service compensation to sustain an institutions at improving work performance [GAC] appropriate livelihood [ACF] -Regularity of training for civil servants on rules and procedures - Degree of understaffing to perform required tasks [SGACA] governing recruitment, hiring and promotion [ACF] - Existence, relevance and participation on training aimed at improving - Share of “ghost” employees (those who receive a payment and are not work performance legitimate program personnel) -Existence, type and application of performance evaluation systems - Extent of public employment use to diffuse social tensions at the expense -Existence, type and application of rewards and sanctions tied to job performance of efficiency and effectiveness [SAF]

Note: AGI-WB Actionable Governance Indicators; CPIA-WB Country Policy and Institutional Assessment; ACF- USA Anti-Corruption Assessment Framework, ROSC- IMF Fiscal Module Report on Observance of Standards and Codes; GAC- WBI Governance and Anti-Corruption Country Survey; SGACA - Netherlands, Strategic Governance and Corruption Analysis; SAF- Netherlands, Stability Assessment Framework.

15

4.2 Financing and Resource Management As in any other development intervention, financing and resource management is an enabling factor in program implementation. The rules concerning budget planning and execution, resource allocation across agencies and government levels involved in program implementation, and budget reporting and monitoring matter for organizational performance. In particular, policy and performance indicators in this area are very relevant as social safety nets may not have the same legal status as health, education or pension systems. As a result, their funding availability may be subject to changes in government.

In this regard it is central to identify the legal and administrative mechanisms that ensure budget availability for the program life cycle. In many countries sub-national governments have a significant role in financing social safety nets benefits. Incentives and accountability mechanisms for local governments to deliver these programs are critical since these tend to lack influential constituencies who can pressure governments to pay benefits.

As shown in Annex 1, financial management is included in a number of existing governance assessment tools. In particular, PEFA Public Financial Management Performance Framework is one of the most comprehensive instruments. It includes a comprehensive list of high level indicators, disaggregated assessment dimensions and a set of evaluation criteria for appraising the operational performance of key elements of financial systems. Although their measurement focuses on central governments, legislatures and external audit institutions, a number of PEFA indicators can be adapted to assess sectoral or program level financial management. Data collection and additional indicators specific for social safety nets can be obtained from other tools such as Public Expenditure Reviews (PERs) and Public Expenditure Tracking Services (PETS).

Table 2 presents the main indicators for assessing financing and resources management governance. Many policies and rules in financial management are central government-wide regulations, although their application (policy performance) may vary across institutions. In a few cases, rules and procedures may be sub-national government or program specific.

16

Issue Budget preparation process

Policy-based budgeting

Predictability and control in budget execution

Table 2. Financing and Resource Management Governance Indicators in Social Assistance Programs Policy dimension Performance dimension - Existence and adherence to a fixed budget calendar and approval process - Nature of involvement in the budget preparation by the ministry, - Executed social assistance budget as a percentage of department or agency in charge of social assistance approved budget (aggregate and by budget line) at the - Existence of a clear and comprehensive guidance for the preparation of central and sub-national government levels 11 budget submissions (budget circular or equivalent) - Resources received as a percentage of approved transfers - Timely budget approval by the legislature from the central to sub-national governments for social - Existence of social protection sector strategy with multi-year costing of assistance programs* recurrent and investment expenditure - Timely availability of approved resources in cash and in - Availability and monitoring of social assistance program multi-year kind transferred to frontline offices coverage targets* - Share of local governments that publish quarterly reports - Existence and use of rules-based systems for the distribution of of budget execution* transfers from the central to sub-national governments for social - Frequency and extent of wage (and travel reimbursement) assistance programs arrears for frontline office staff (those with direct contact - Degree of timeliness and reliability of information on transfers from the with clients) central to sub-national governments - Frequency and extent of benefit payments arrears - Regularity of tracking of resources received in cash and in kind by - Effectiveness of payroll controls frontline offices across the country, irrespective of which level of - Effectiveness of internal controls for non-salary government is responsible for the operation and funding of those units expenditures - Extent to which cash flows for social assistance programs are - Frequency and scope/nature of audits performed forecasted and updated - Evidence of effective and timely follow up on internal and - Frequency and transparency of adjustments to social assistance external audit recommendations program budget allocations - Timely publication on the internet of key social assistance - Existence of protection mechanisms of social assistance expenditure in expenditure information at the national and sub-national case of in-year adjustments to budget allocations government levels * -Existence of payroll controls (reconciliation of personnel records and - Availability on internet of a summary of program budget payroll; internal controls and audits) written in clear, simple language [ROSC] - Quality and transparency of procurement regulatory framework and application (for equipment purchasing, selection of payment providers, etc) - Existence of internal controls for non-salary expenditures (expenditure

11

The ministries, departments and agencies concerned for the purpose of this indicator are those which are directly charged with responsibility for implementing the budget in line with sector policies and which directly receive funds or authorization to spend from the Ministry of Finance. Department and agencies that report and receive budgetary funds through a parent ministry should not be considered in the assessment.

17

Issue

Accounting, recording and reporting

External scrutiny and audit Transparency

Table 2. Financing and Resource Management Governance Indicators in Social Assistance Programs Policy dimension Performance dimension commitment controls and rules for processing and recording transactions) - Coverage, independence and quality of internal audit function in social assistance program - Existence of systematic procedures for program management response to internal audit findings - Existence of routine data collection or accounting systems that provide reliable information on all types of resources received in cash and in kind by frontline offices across the country, irrespective of which level of government is responsible for the operation and funding of those units.

- Regularity of report compilation on resources received in cash and in kind by frontline offices across the country - Regularity of bank reconciliations (or with other payment agents – e.g. post office, etc.)

- Regularity of reconciliation and clearance of suspense accounts and advances - Quality and frequency standards of regular budget reports and annual financial statements - Existence and adequacy of auditing policies - Existence of systematic procedures to follow up on audit recommendations - Existence and use of a robust classification for monitoring and tracking social assistance spending - Existence and comparability of social assistance expenditure tracking at central and sub-national levels of government - Existence and thoroughness of policies for public access to key social assistance expenditure information (including transfer distribution rules from the central to sub-national governments) at the national and subnational government levels

Note: ROSC- IMF Fiscal Module Report on Observance of Standards and Codes; *Indicators not included in reviewed existing assessment tools.

18Â Â

4.3 Operational procedures, information management and control mechanisms Social assistance programs require well-defined operational procedures, management information systems and control mechanisms to ensure that the right people get the right benefit at the right time. As depicted in Figure 3, a series of operational processes must be in place to identify and enroll beneficiaries, deliver program benefits to them, eventually reassess their eligibility as well as manage their complaints in case any of the mentionedtransactions went wrong. The rules concerning the definition of clear-cut operating procedures and institutional responsibilities could make a substantial difference in reducing discretionary decisions on eligibility and benefit levels, giving certainty to applicants as to what to expect, and in case things go wrong, how to fix them.

Table 3 includes the main indicators in assessing governance in program operation, information management and control. On one hand, policy indicators in this area assess the existence and pertinence of standard procedures and control mechanisms along operational processes. On the other hand, performance indicators include compliance and efficiency measures associated with established procedures and controls. Most policies and rules are likely to be program-specific, as opposed to government-wide regulations. As discussed in HRM or financial management indicators, performance measurements may be associated to more than one governance policy in program operation. Only a few governance assessment tools address social protection issues and they mainly focus on pensions. Hence, most indicators in Table 3 are not included in existing assessment tools. Nonetheless, many can be found among results matrices of safety nets projects.

Program operations are often sequenced in a systematic manner starting off with beneficiary outreach, application, selection and enrollment, passing through benefit delivery until exit and graduation.

Outreach activities inform potential clients of program benefits, eligibility criteria and application process. These activities should be aligned with a program coverage strategy based on the characteristics of the target population and the intended number of beneficiaries for specific regions in a given period. For example, the target population for a program may be female-headed households in selected rural communities. Alternatively, the target population for a larger urban program may be poor households nationwide. The latter case may warrant a mass media promotion campaign and a program coverage expansion plan spanning a few years. The former would require community outreach activities in selected rural areas and coverage targets may be met in a shorter period. Thus, a governance focus of this operational step would analyze the existence of an annual and medium term program coverage strategy and the extent to which this strategy and outreach activities are adequate to reach the target population given budget and implementation constraints.

19

Application, beneficiary selection and enrollment follow. In particular, the application process, the choice of beneficiary selection methods and their implementation matter for program targeting results, and presumably, for poverty outcomes. To what extent targeting methods are appropriate given country circumstances and are applied according to welldefined and sound administrative procedures are relevant governance issues. Moreover, it is important to consider applicant´s transaction costs in terms of number of visits, time spent and paperwork required for program application and enrollment. The more time and resources these spend doing paperwork, the higher the opportunity costs (i.e. work absence), and the less likely are to participate in program activities. Targeting indicators such as the share of transfers received by the poor are also discussed in Table 4 .

Payment of benefits should be accurate, secure, reliable, transparent, affordable and hassle-free, and most importantly, timely. Rules defining payment sites and recipient lists, calendar and frequency, process and amounts as well as payment agents and instruments are important aspects of program governance. EFC prevention recommendations include ensuring that benefit payment is as secure as possible, ideally by making payment directly to recipient accounts – the fewer hands physical money travel through, the better (Stolk and Tesiluc, 2010). Another aspect worth considering are the rules regarding the adjustment of benefit levels over time, whether it is constant or inflation-indexed to maintain its real value.

Exit and graduation considerations may set rules on benefit duration or eligibility recertification. If beneficiary selection includes proxy-means testing, recertification may be an opportunity to update the scoring system. Recertification may thus improve program targeting performance, but it also involves additional costs (Coady et. al., 2004).

Governance analysis would consider to what extent clear and transparent rules regarding duration of benefits, recertification frequency and processes exist.

Information management and control mechanisms are cross-cutting elements vital for the effective and efficient processing of benefit claims. Adequate MIS allow a systematic registry of beneficiaries and benefits payment record-keeping, both indispensable for program management and minimizing human error in the processing of benefits. Moreover, in some OECD countries data submitted by claimants is cross-checked against other government databases, as a cost-effective way to detect EFC (Stolk and Tesiluc, 2010). The rules regarding which information is collected, who is responsible for collection, how it is systematized, kept and used can influence organizational performance.

Finally, control mechanisms along the operational processes are also instrumental. These include a range of activities such as household information verification, database crosschecks, performance audits, and monitoring and evaluation. Some control tools involving independent assessments enable demand-side accountability and are included both in Tables 3 and Table 6.

20

Issue Coverage strategy and promotion

Application, selection and enrollment

Payment

Recertification

Information Management

Table 3. Governance Indicators in Program Operation and Control Mechanisms Policy dimension Performance dimension - Existence and tracking frequency of annual and medium term coverage strategy - Degree of alignment between promotion activities and - Consistency between coverage strategy and number and type of coverage strategy (including adequacy of dissemination beneficiaries (rural/urban, male/female, etc.) channels and materials) -Level of staff compliance with operational procedures and processing - Adequacy of public awareness activities on program rights and times obligations -Program application and enrollment transaction costs (e.g. number of - Existence of sound poverty diagnosis to identify poor, visits, time spent, etc.) vulnerable groups and those lacking services [CPIA] -Program recertification transaction costs - Type and adequacy (cost-effectiveness) of targeting method - Compliance with payment calendar (e.g. % of cash transfer payments choice (simple means test, proxy means test, community-based delivered according to payment calendar) targeting, categorical targeting, self-selection) - Compliance with eligibility reassessment calendar (e.g. % - Degree of objectivity and transparency of eligibility criteria recertification of program beneficiaries due to be recertified after - Degree of clarity and soundness (i.e. separation between established years of enrolment) assessment and payment functions) of operational procedures - Completeness and reliability of beneficiary registry and payment for program application, beneficiary selection and enrollment registry - Existence of a benefit payment calendar - Average time taken to update in the MIS changes beneficiary - Type of payment method (bank deposit, debit card, cash, etc.) information (e.g. household composition (newborns or deaths) or - Level of reliability of payment method and process (direct address changes) payment to beneficiary through secure channel) -Percentage of cash transfers made automatically through MIS - Evolution of program benefits (constant, indexed, - Number of progress reports produced by the MIS, according to discretionary evolution, etc.) [SSD] agreed reporting standards - Degree of systematic use of control mechanisms (household - Clarity and transparency of rules concerning duration of information verification, database cross-checks, telephone hotlines, benefits and frequency of eligibility reassessments etc.)

- Duration of benefits [SSD] - Degree of use of risk-based investigations - Frequency of eligibility reassessments - Frequency of EFC monitoring - Degree of clarity and soundness of operational procedures for - Relevance, independence and quality of program evaluation program eligibility reassessment - Frequency and scope/nature of performance/operational audits - Existence of adequate beneficiary registry and payment performed records - Administration costs of social assistance program [SPEPR] - Extent of use of IT system to register and process benefit claims, and cross-check data - Clarity of roles, responsibilities and procedures for data collection, systematization, quality control and reporting

21Â Â

Issue Control mechanisms

Table 3. Governance Indicators in Program Operation and Control Mechanisms Policy dimension Performance dimension - Type and adequacy of verification mechanisms of information submitted in program application (database crosschecks, random house visits, etc.)

- Adequacy of procedures to refer cases for investigation - Existence and type of monitoring mechanisms (spot checks, operational audits, third-party monitoring) of staff compliance with operational procedures - Adequacy of monitoring mechanisms for third party implementation (local government, payment agency, NGOs) - Existence and type of systematic activities for program M&E and management response to M&E findings - Existence of policies to ensure public access to M&E information at the national and sub-national government levels - Existence of systematic procedures to follow up on audit recommendations - Existence of policies to ensure public access to program beneficiary list

Note: CPIA-WB, Country Policy and Institutional Assessment; SSD - ILO, Social Security Database; SPEPR – ILO, Social Protection Expenditure and Performance Reviews

22

4.4 Organizational Performance of the Delivery of Cash Transfers Organizational performance in social safety nets can be measured along five dimensions: program relevance, coverage, poverty targeting, efficiency and service quality ─discussed separately in the next section. Table 4 includes summary measures of organizational performance in these areas. Most indicators belong to the output category, except for poverty targeting measures that may be considered a grey area between outputs and outcomes. Clear-cut outcomes such as increased consumption levels are not included here as the main focus is on the supply side.

One way to assess program relevance is through the adequacy of benefit levels compared to a benchmark. The conceptual benchmark considered in Social Protection Public Expenditure Reviews for social safety nets is the gap between a household’s income and the poverty line. Coverage is a key performance dimension that gauge to what extent program magnitude is appropriate given the target population size. If coverage is low, the program is ineffective in reaching a meaningful proportion of vulnerable population.

Coverage information could be presented by pertinent breakdowns such as rural-urban, regional breakdowns, gender, and so forth. It should be measured as a rate where the denominator is the total population deemed as vulnerable, for example, the total number of poor households.

Commonly used poverty targeting measures include the share of total transfers received by the population falling within the bottom 40, 20, or 10 percent of the national income distribution, the share normalized, and the inclusion and exclusion errors. However, to what extent better targeting as measured by these indicators imply a large impact on poverty or a more cost-effective intervention is an empirical question yet to be settled12.

Finally, a starting point to assess program efficiency may be administrative costs. They should be high enough to allow adequate program administration but low enough to be efficient. Cost information should be compared to international or regional benchmarks.

Another useful measure of program efficiency is the proportion of program funds that are lost due to EFC13. Some of the indicators of organizational performance are included in social protection assessment tools such as the ILO Social Security Database (SSD) or the World Bank Social Protection Public Expenditure Reviews (social protection-PER).

12

See Ravallion 2007 for a critical review of different targeting measures.

The following definitions are taken from Stolk and Tesiluc, 2010. Targeting errors may be due to program design (when social assistance programs use imperfect poverty proxies to identify poor beneficiaries) and to program implementation (when eligibility decisions and payments diverge from program rules). The later component is due to EFC. Error refers to unintentional mistakes on behalf of benefit claimants or staff in the benefit office. Fraud refers to intentional behavior on the part of the benefit claimant to defraud the benefit system. Corruption is an intentional attempt by staff to exploit the social protection system.

13

23

Relevance Coverage

Poverty Targeting14 Efficiency

Table 4. Organizational Performance Indicators - Adequacy of benefit levels with respect to poverty line [social protection-PER] - Coverage rate (number of program beneficiaries with respect to total vulnerable or target population disaggregated by rural-urban distribution, region, gender, formal/informal sector, and so on) [social protection-PER, SSD, SPEPR] ‐ Horizontal distribution of benefits (proportion of cash benefits distributed by gender, formal/informal sector, and distribution to other groups identified as the most vulnerable) [SPEPR] - Targeting Differential (difference between program participation rate of the poor and the participation rate for the non-poor) - % of transfers delivered on time and to the right recipient - Error, fraud and corruption rates (proportion of total transfers that are lost due to error, fraud or corruption) [social protection-PER] - Administration costs [social protection-PER, SPEPR]

Note: social protection-PER- WB Social Protection Public Expenditure Review; SSD- ILO, Social Security Database; SPEPR- ILO, Social Protection Expenditure and Performance Reviews.

4.5 Quality of Social Assistance Service Delivery Accessibility and quality in social services provision are generally recognized as important factors influencing social inclusion15. A number of OECD countries have developed service quality standards for organizations delivering public services. A few examples include the Charter Mark Standard in the UK, the National Standards for Disability Services in Australia or the Customer Service Standards of Rockville City Government in the US. Some of the quality dimensions emphasized in these initiatives that may be relevant for social assistance are the following. Table 5 includes alternative indicators in each dimension.  



Service quality standard-setting and performance: existence and compliance with service and performance standards.

Service accessibility: extent to which social assistance services are physically, spatially and socio-culturally accessible to all individuals in the target population, regardless of their age, gender, ethnicity, language, disability condition, location, and so on.

Privacy, dignity and confidentiality: extent to which each beneficiary is treated with the same level of privacy, dignity and confidentiality.

14

See Ravallion 2007 for a broader list and discussion on different targeting measures.

Measurement of service quality has been extensively researched in the private sector management and marketing literature. Nonetheless, the conceptualization and operationalization of service quality as well as the link between service quality, purchase intentions and business performance continue to be subject of debate (Seth et al. 2005). In the public sector, research on the impact of service quality on human development outcomes is more recent and has been focused primarily on health care and education services.

15

24

 

 

Cultural awareness: delivery of non-discriminatory services which are sensitive to social and cultural values of the individuals, their families and community.

Complaints and disputes: existence of publicized and easy-to-use complaints procedure, including a commitment to deal and solve complaints within a time limit.

Exit and reentry: extent to which individuals are assisted to plan for the exit from social safety nets and assured that reentry is available, if required.

Service integration: development of links with other social assistance service providers at local, state and national level to ensure access to complementary services.

Service quality standard setting and performance

Service accessibility

Table 5. Service Quality Indicators - Existence of precise and measurable service standards - Regular performance monitoring and benchmarking with similar services - Clients and staff participation in setting and reviewing service standards - Dissemination to clients and potential clients of service standards and performance - Assessment, recording and analysis of clients’ satisfaction levels and action taken to deal with any problems Accessibility policies - Existence and application of equal opportunities and disability policies, procedures and training for service delivery - Extent to which target population is defined, their needs regularly identified and services are planned and delivered to meet those needs Affordability - Availability and use of different means to reduce the cost of program information and application (e.g. mobile offices, toll-free phone numbers, etc.)

- Travel time and transportation costs to facilities - Application and enrollment costs - Average time required for program application and enrollment - Individual’s perception of ease of service accessibility and paperwork required Physical, spatial and functional accessibility - Convenience of location and access (% of facilities in close proximity to public transport, with clear signage, level access, that are compliant with accessibility regulations for people with physical disabilities) -Convenience of facilities (e.g. % of facilities with adequate waiting areas which accommodate children, are properly maintained, sanitary, completely operational, fully stocked and supplied ) - Extent of use of technology to provide information and access to services (if appropriate) - Posting and observance of operation hours (e.g. % of facilities with posted operation hours, % of facilities with staffed information/registration counters during business hours at random visits) - Convenience of operation hours (% of beneficiaries who consider operation hours convenient) Socio-cultural accessibility - Availability of information materials in different languages and formats -Ratio of the number of beneficiaries who have difficulties communicating in the official language and the number of trained interpreters or bilingual staff - Availability of information and tailored service access for individuals with special needs (people with visual or hearing difficulties, those who have difficulties understanding or reading the official language, and people with

25

Privacy, dignity and confidentiality

Cultural awareness

Complaints and disputes

Exit and reentry

learning difficulties) - Individual’s perception of service provision in a manner sensitive to the age, sex, ethnic, and linguistic background of each person - Social distance - Existence, implementation and monitoring of policies and procedures on protecting individuals’ privacy, dignity and confidentiality - Existence, compliance and monitoring of ‘customer care’ policies - % of staff – client interaction in which staff performed satisfactorily (i.e. listen to client’s request/question and provide complete, knowledgeable, accurate, precise information regarding the inquiry) - Individual’s perception of respectful, honest and professional treatment - Individuals’ perception of discrimination - Extent to which complaints of abusive treatment are monitored and addressed - Number of complaints of abusive treatment (trend over time) - Extent to which staff have knowledge of the social and cultural groups represented in the target population and an understanding of those social and historical factors relevant to their current circumstances - Extent to which service delivery considers the needs and unique characteristics of social and cultural groups represented in the target population and involves these groups in the planning and implementation of services - Extent to which specific cross-cultural staff training, and involvement of representatives of relevant cultural groups in service delivery planning is promoted - Extent to which social assistance employs staff or develops links with other service providers/organizations with relevant experience in the provision of services to the specific social and cultural groups represented in the target population - Extent to which issues related to social and cultural prejudice among staff are monitored and addressed - Extent to which representation of social and cultural groups amongst caseloads are monitored and reviewed - Individuals’ perception of service delivery in a manner which is sensitive to their social and cultural beliefs, values and cultural practices - Existence of staff guidance and training to handle complaints and empowerment to put things right - Extent to which the number and type of formal and informal complaints, compliments and suggestions received, and how quickly they are dealt with are recorded, analyzed and posted in the internet - % of local offices that publicized regularly information on the number and type of complaints, compliments and suggestions received, along with the improvements made as a result - Clients and staff assessment of complaints procedure - Regular review and improvement of complaints procedure, taking into account the views of clients and staff - Number and type of formal and informal complaints, compliments and suggestions received - Average time to address a complaint or dispute - Percentage of unresolved complaints or dispute - Existence of documented individual exit plan which is begun during entry to social assistance to ensure ongoing continuity of other relevant social services once the client has exited from social assistance - Extent to which the exit plan is reviewed in collaboration with the client - Extent to which clients and other service providers and agencies involved in follow-up (if applicable) are aware of how to gain entry to social assistance at a later date, if needed - Monitoring of reasons for exiting social assistance services

26

Service integration

- Percentage of clients who exit social assistance services by reason - Percentage of clients who exit social assistance services and apply for reentry within a certain period of time - Percentage of clients who exit and reentry social assistance - Existence of arrangements to provide coordinated services and formal processes to promote inter-agency collaboration - Extent to which separately funded programs work collaboratively to achieve a single integrated social assistance system - Percentage of clients who enter other social services through social assistance - Percentage of clients who benefit from coordinated services

In addition, there are two additional dimensions that are gaining significance in service quality assessment and may be relevant for some social assistance programs. First, there is the focus on individual needs that shifts away from the traditional ‘top-down’ service provision to service delivery designed to meet individual needs and personal goals. Second, there is an emphasis on decision making and choice that provides individuals a range of service alternatives and information to assist in the selection of the most appropriate option in the setting most empowering for the client.

4.6 Demand­side accountability in service delivery Demand-side accountability refers to the influence that individuals can have in their dual role as direct service clients –i.e. client power - as well as citizens that participate in political processes that define collective objectives –i.e. political voice (World Bank, 2003a). Client power may be enhanced by choice in service delivery (e.g. vouchers in education), better information and direct participation. Political voice is exercised through voting, advocacy, and representation in political bodies, often facilitated by public disclosure of key policy information.

Governance policies and performance shape the mechanisms, both formal and informal, through which client power, and to some extent, voice, are fostered and exercised.

Following Ringold, et. al. (2010), Table 6 includes demand-side accountability measures for social assistance programs in the following areas.   

Transparency: access to key policy information (budgetary, rules of operation, performance, and so on).

Participation: client involvement in program decision-making, implementation and monitoring, and capacity building activities for stakeholder participation.

Grievance redress mechanisms: existence of procedures to handle complaints and mechanisms for their follow up and resolution.

27



Independent assessments16: tools and arrangements for independent verification of access to, quality and effectiveness of services through audits, service delivery and other surveys, monitoring and evaluation.

Some of the governance policies and performance in these areas have been considered in preceding sections and can be discussed interchangeably. For example, the existence of independent assessments was considered as part of control mechanisms. Governance measures of transparency were included in the areas of financial management and operational procedures.. In addition, grievance redress mechanisms were addressed as one of the service quality dimensions. For the sake of a comprehensive discussion on demandside indicators, a number of examples mentioned previously will be repeated here. Since evidence on how to foster client power and to what extent it has an impact on development outcomes is work in progress, listed indicators should be treated as hypothesis to be tested.

16

This category combines the third party monitoring and independent assessment areas.

28

Issue Transparency

Participation

Table 6: Demand-side Service Quality Indicators in Social Assistance Programs Policy dimension Performance dimension - Existence and thoroughness of policies for public access to - Type and adequacy of activities and mechanisms for information key social assistance information (expenditure, rules of dissemination (general information, awareness and sensitization operation, beneficiary list, procurement, M&E and other campaigns, mass media, written materials, community meetings, etc.) performance information, etc.) at the national and sub-national - Share of local governments that publish quarterly reports of budget government levels execution, procurement, performance information, etc.

- Share of local offices that post key program information (budget and expenditure, service standards, performance measures, complaints procedures, etc.) in accessible places - Timely publication on the internet of key social assistance information (expenditure, rules of operation, performance assessment, etc.) at the national and sub-national government levels - Availability on internet of a summary of program budget written in clear, simple language - Public awareness of program rights and obligations (e.g. % of target population who correctly identify the eligibility criteria for social assistance benefits) - Perceptions of transparency in social assistance services (among clients, journalists, etc.)

- Existence of client participation principles and mechanisms - Level of client participation in setting and reviewing service standards Existence of staff guidance and training to promote client - Level and type of client participation in service implementation and participation monitoring (consultations, planning, service contracting, oversight, - Existence of capacity building activities for stakeholder participatory M&E) participation - Type (social audits, local committees, etc.) and functioning of client participation mechanisms (i.e. high level of involvement of a wellrepresented group of beneficiaries) - Clients awareness of participation mechanisms (% of clients who are able to mention at least one participation option) - % of clients that report an active participation in service planning, implementation or monitoring - Clients perception of the value of their participation to influence service quality

29Â Â

Issue Grievance Redress Mechanisms

Independent Assessment

Table 6: Demand-side Service Quality Indicators in Social Assistance Programs Policy dimension Performance dimension - Existence of grievance redress policies and procedures - Client awareness, potential and actual use of grievance redress (procurement, fraud, corruption, service quality, etc.) mechanisms (% of clients who are able to mention at least one complaints - Existence of grievance redress institutional arrangements mechanism, % of clients who would complain in case of poor service (appeals committee, ombudsman, etc.) quality, % of clients that considered service quality was bad and used the - Type of channels for individuals to complain and refer cases established procedures to complained about it, % of clients who have (e.g. telephone hotlines, on-line, town-hall meetings) provided feedback) - Existence of staff guidance and training to handle complaints - Number and type of formal and informal complaints, compliments and and empowerment to put things right suggestions received - Existence of procedures to receive, record and address - Average time to address a complaint or dispute complaints, compliments and suggestions - Percentage of grievance redress claims settled within agreed time - Extent to which there is public access to information on the - Clients and staff assessment of complaints procedure and its number and type of complaints, compliments and suggestions effectiveness received, along with the improvements made as a result - Percentage of unresolved complaints or disputes - Extent to which there is a regular review and improvement of - % of local offices that publicized regularly information on the number complaints procedure, taking account of the views of clients and and type of complaints, compliments and suggestions received, along with staff the improvements made as a result - % of local offices that have made during the last semester at least one change in service policies or procedures as a result of client feedback.

- Existence and adequacy of auditing policies (procurement, - Frequency and scope/nature of audits performed financial, operational and performance) - Type, relevance, independence and quality of program M&E tools (spot - Existence of systematic procedures to follow up on audit checks, facilities survey, scorecards, beneficiary assessments, impact recommendations evaluations, etc.)

- Existence and content of M&E policies (i.e. existence of - Client awareness of independent assessment activities, results and systematic activities, adequate institutional arrangements and program management response to them corresponding budget allocation for M&E) - Existence of systematic procedures for management response to M&E findings

30Â Â

4.7 Data Collection Indicators for measuring governance and service quality are diverse and require a variety of data collection instruments. For example, policy indicators tend to focus on assessing the existence and adequacy of rules, norms and regulations and may be more amenable to measurement through expert surveys. In contrast, governance performance indicators are more likely to be quantitative measures that can be obtained from administrative data for regular monitoring. This information can also be measured or independently assessed through facilities surveys. Measures of organizational performance on targeting, service quality and demand-side accountability may require household or beneficiary surveys, as well as qualitative data.

Expert surveys: This tool elicits expert assessment of key performance dimensions using a set of evaluation criteria. It is useful to assess indicators that are better measured through a combination of quantitative and/or qualitative variables. For example, measuring the existence and clarity of job descriptions and responsibilities would consider the proportion of staff positions with a documented job profile, as well as a qualitative assessment of their adequacy. PEFA Public Financial Management Performance Framework discussed earlier is an example of a widely used expert survey for assessing governance policies and performance on financial management issues.

Administrative or monitoring data: Existing government databases can be a rich data source, particularly for governance performance information on human resources, financial and resource management, operational procedures, information management and control mechanisms. For example, indicators such as staff turnover rates, budget execution, the percentage of cash transfers made directly to the recipient’s account, administration costs and the coverage rate can be measured through administrative data. The quality and objective measurement of these data may be heterogeneous, thus it is important to crosscheck with other data sources such as facility or beneficiary surveys. For instance, records may show that a certain amount of transfers have been delivered but they may not capture whether a payment agent received an amount from the recipient to disburse the cash.

Facility surveys (PETS/QSDS): This instrument collects data on the quality and quantity of public services delivered and the resources used to deliver them at the facility level.

Public Expenditure Tracking Surveys (PETS) measure the flow of funds and other resources through the system and assess whether they result in service delivery. PETS can be linked to Quantitative Service Delivery Surveys (QSDS) that focus more on the quantity and quality of services and the behavior of providers. Although most PETS have focused on health and education, they can be adapted to social assistance services.

Beneficiary or household surveys: A number of indicators are aimed at measuring providers’ performance through clients experience with service delivery. Beneficiary or 31

household surveys collect data on individuals’ behaviors, knowledge, perceptions and practices, particularly relevant for assessing service quality and demand-side indicators.

Examples of indicators measured through household and beneficiary surveys include transaction costs (in terms of money and time spent) involved in program participation, amount and frequency of social assistance transfers received, service accessibility, awareness of rights and obligations and program engagement. National household surveys are also useful for targeting measures and independent measurement of service coverage.

Qualitative data: Surveys and quantitative data sources can be complemented with qualitative data that helps understanding the gap between governance policies on paper (de jure) and their actual implementation (de facto). For example, while human resource policies may specify provisions for competitive hiring processes, in practice, positions may be sold or procedures circumvented in many ways. Moreover, there are some areas that are more amenable to qualitative assessment. For example, socio-cultural barriers to service access could be better assessed using qualitative methods since socio-cultural issues can be differently defined and understood by groups or individuals.

4. Conclusions Governance lenses have not been applied yet to social protection in a comprehensive and systematic way. This paper aimed at contributing to this process by developing a framework for measuring governance and quality of service delivery in social safety nets.

The proposed framework assesses governance policies and performance in human resources, financing and resource management, operational procedures and control mechanisms. The framework also incorporates measures of organizational performance in cash transfer delivery, emphasizing service quality, and demand-side accountability.

Finally, based on the proposed framework, a list of governance and service quality indicators for social assistance programs is identified. The selection of any particular set of indicators has underlying normative assumptions. There is some evidence from OECD countries on which governance interventions are cost-effective measures for reducing EFC. However, not all governance interventions have proved their impact on organizational performance, including service quality, or program outcomes. Therefore, the list of indicators should be treated as hypothesis to be tested rather than evidence to be followed.

Furthermore, the selection of specific governance indicators will differ by country and program context. For example, governance issues will vary between programs that are operated by a central bureaucracy and those operated by an agent, such as local governments. Similarly, different governance aspects such as controlling error, fraud or corruption may be more or less relevant for low, medium and high income countries.

32

5. References

Adema, Willem and Maxime Ladaique (2009), “How Expensive is the Welfare State?: Gross and Net Indicators in the OECD Social Expenditure Database (SOCX)”, OECD Social, Employment and Migration Working Papers, No. 92, OECD Publishing.

African Development Bank (2009), “Federal Republic of Nigeria. Country Governance Profile”, African Development Group.

Bitner, Mary Jo, et. al. (1997), “Customer Contributions and Roles in Service Delivery”, International Journal of Service Industry Management, Vol. 8. No. 3 Bjuremalm, Helena (2006), “Power Analysis – Experiences and Challenges”, Department for Democracy and Social Development.

Boulding, William, et. al. (1993), “A Dynamic Process Model of Service Quality: From Expectations to Behavioral Intentions” Journal of Marketing Research, Vol. XXX Brinkerhoff, Derick W. and Thomas J. Bossert (2008), “Health Governance: Concepts, Experience and Programming Options” Health Systems 20/20, February 2008, USAID. Clingendael Institute (2005), “Problem Driven Governance and Political Economy Analysis”, The Netherlands Ministry of Foreign Affairs.

Clingendael Institute (2008), “Framework for Strategic Governance and Corruption Analysis. Designing Strategic Responses towards Good Governance”, The Netherlands Ministry of Foreign Affairs.

Coady, David, Margart Grosh ,and John Hoddinott (2004), “Targeting of Transfers in Developing Countries: Review of Lessons and Experience”. The World Bank: Washington, DC. De Crombrugghe, Denis, et al. (2009), “Institutional Profiles Database III”, Documents de Travail de la DGTPE, Document 2009/14, Direction Générale du Trésor et de la Politique Économique.

Drivers of Change Team (2003), “What does Drivers of Change mean for DFID?” Department for International Development Policy Division, A draft approach paper. United Kingdom.

33

European Centre for Development Policy Management (2008), “Analyzing and Addressing Governance in Sector Operations”, Tools and Methods Series, Reference Document No. 4, European Commission.

European System of Integrated Social Protection Statistics (2008), “ESSPROS Manual”, Eurostat Methodology and Working Papers, European Commission.

Federal Ministry for Economic Cooperation and Development (2007), “DevelopmentOriented Transformation in Conditions of Fragile Statehood and poor Government Performance”, Federal Ministry for Economic Cooperation and Development: Germany.

Fiszbein, Ariel, Dena Ringold, and Halsey Rogers (2011). "Making Services Work: Indicators, Assessments, and Benchmarking of the Quality and Governance of Public Service Delivery in the Human Development Sectors." Policy Research Working Paper 5690. The World Bank: Washington, DC. Fritz, Verena et al. (2009), “Problem Driven Governance and Political Economy Analysis. Good Practice Framework”, World Bank.

Hagemejer, Krzysztof, “Social protection expenditure and performance reviews.

Methodological note on definitions, classifications and performance indicators”, Financial, Actuarial and Statistical Services, International Labour Office. Mimeo.

Halloran, John, and Kerstin Calderon-Vera, “Access to Quality Social Services. A Strategy Paper”. European Social Network. Mimeo. http://www.peer-review-socialinclusion.eu/key-themes/quality-and-accessibility-of-social-services-1 Inter American Development Bank, “Governance Indicators Database”, accessed on June 25, 2010. Available online at [http://www.iadb.org/datagob/index.html].

International Labour Organization, “The Social Security Database: User Guide Book”, accessed on June 26, 2010. Available online at [http://www.ilo.org/dyn/sesame/SESHELP.SSDBCodeBookIndex] International Monetary Fund (2009), “Safeguards Assessments”, International Monetary Fund. Finance Department. ________ (2007), “Questionnaire on Fiscal Institutions”, International Monetary Fund. ________ (2006), “Reference Guide to Anti – Money Laundering and Combating the Financing of Terrorism”, Second Edition and Supplement on Special Recommendation IX,

34

The International Bank for Reconstruction and Development/The World Bank/The International Monetary Fund.

Kaufmann, Daniel, et al. (2009), “Governance Matters VIII. Aggregate and Individual Governance Indicators 1996–2008”, World Bank Policy Research Working Paper 4978. Washington, DC: World Bank.

Kaufmann, Daniel and Aart Kraay (2008), “Governance Indicators: Where Are We, Where Should We Be Going?” The World Bank Research Observer, 23(1):1-30.

Kaufmann Daniel, Aart Kraay and Massimo Mastruzzi (2007), “Governance Matters VI: Aggregate and Individual Governance Indicators for 1996–2006.” World Bank Policy Research Working Paper 4280. July. Washington, DC: World Bank.

Lewis, Maureen and Gunilla Pettersson (2009), “Governance in Health Care Delivery: Raising Performance,” Washington, DC: World Bank. Mimeo.

Lewis, Maureen and Gunilla Pettersson (2009), “Governance in Education: Raising Performance,” Washington, DC: World Bank. Mimeo.

Mathauer; Inker (2004), “Institutional Analysis Toolkit for Safety Net Interventions,” Social Protection Discussion Paper Series No. 0418. Social Protection Unit, Human Development Network. Washington, DC: World Bank Mardan, Mohan (2009), “Sector Level Risk Assessment and Risk Management Plan to Assess Public Financial Management, Procurement, and Corruption Risks in the Urban Development Sector”, Asian Development Bank Consultant’s Report.

Millennium Challenge Corporation (2009), “Report on the Criteria and Methodology for Determining the Eligibility of Candidate Countries for Millennium Challenge Account Assistance in Fiscal Year 2010”, Millennium Challenge Corporation.

Mutual Information System on Social Protection (2007), “Organisation of Social Protection. Charts and Descriptions”, European Commission, Directorate-General for Employment, Social Affairs and Equal Opportunities.

Mutual Information System on Social Protection (2008), “Comparative Tables of Social Protection Systems: Correspondent’s Manual 2008”, Council of Europe.

OECD (2009), “Donor Approaches to Governance Assessment. 2009 Sourcebook,” DAC Network on Governance

35

______ (2008), “Survey of Donor Approaches to Governance Assessment,” DAC Network on Governance ______ (2006), “Methodology for Assessment of National Procurement Systems”, Version 4, OECD. OECD, “Benefits and Wages 2007”, accessed on June 29, 2010. Available online at [http://www.oecd.org/document/33/0,3343,en_2649_34637_39619553_1_1_1_1,00.html#r elated OECD, “OECD Family Database”, accessed on June 29, 2010. Available online at [http://www.oecd.org/document/4/0,3343,en_2649_34819_37836996_1_1_1_1,00.html] OECD, “Pension Database”, accessed on June 29, 2010. Available online at [http://www.oecd.org/document/16/0,3343,en_2649_34757_45558288_1_1_1_1,00.html] OECD, “The OECD Sickness, Disability and Work Project”, accessed on June 29, 2010. Available online at [http://www.oecd.org/document/20/0,3343,en_2649_34747_38887124_1_1_1_1,00.html] Office of Conflict Management and Mitigation (2005), “Conflict Assessment: A Framework for Strategy and Program Development”, United States Agency for International Development.

Operations Policy and Country Services (2009), “Country Policy and Institutional Assessments. 2009 Assessment Questionnaire”, The World Bank: Washington, DC. PEFA Secretariat (2005), “Public Financial Management. Performance Measurement Framework”, The World Bank: Washington, DC. Ravallion, Martin (2007), “How Relevant is Targeting to the Success of an Antipoverty Program”, Policy Research Working Paper 4385, November 2007, Office of the Director, Development Research Group, Washington, DC: World Bank.

Reid, Gary J. (2008), “Actionable Governance Indicators – Concepts and Measurement,” AGI Initiative. The World Bank: Washington, DC. Ringold, Dena, et. al. (2010), “The Demand-side of Governance in HD: Results of a Stocktaking of FY08 Projects” The World Bank: Washington, DC. Rogers, Edward W. and Patrick M. Wright (1998), “Measuring Organizational Performance in Strategic Human Resource Management: Problems and Prospects” Center 36

for Advanced Human Resource Studies, Working Paper 98-09, School of Industrial and Labor Relations, Cornell University, Ithaca, NY Savedoff, William D (2011), “Governance in Health Sector: A Strategy for Measuring Determinants and Performance,” Portland, ME: Social Insight. Policy Research Working Paper 5655. The World Bank: Washington, DC. Seth, Nitin, S.G. Deshmukh, and Prem Vrat (2005), “Service Quality Models: a Review” International Journal of Quality and Reliability Management. Vol. 22 No. 9 Spector, Bertram et al. (2009), “Anticorruption Assessment Handbook”, U.S. Agency for International Development.

Swiss Agency for Development and Cooperation (2007), “Governance as a Transversal Theme: an Implementation Guide”, Swiss Federal Department of Foreign Affairs.

U.K. Department for International Development (2002), “Conducting Conflict Assessments: Guidance Notes”, Conflict and Humanitarian Affairs Department.

U.K. Department for International Development (2007), “Country Governance Analysis”, How to note, Department for International Development.

United Nations Development Program UNDP (2007), “Governance Indicators: A Users’ Guide,” Second edition, Bureau for Development Policy, Democratic Governance Group, New York _______ (2009), “A Users’ Guide to Measuring Public Administration Performance,” Oslo Governance Center, Democratic Governance Group, Bureau for Development Policy, Oslo, Norway.

United Nations Office on Drugs and Crime, “Comprehensive Self-Assessment Checklist on the Implementation of the United Nations Convention Against Corruption”, accessed on June 26, 2010. Available online at [http://www.unodc.org/unodc/en/treaties/CAC/self-assessment.html].

United States Agency for International Development (2000), “Conducting a DG Assessment: a Framework for Strategy Development”, United States Agency for International Development Technical Publications Series.

World Bank (2009), “Core Guidance: Preparing PERs for Human Development”, The World Bank: Washington, DC. 37

________ (2008), “Country Procurement Assessment Report. Philippines”, Document 47602, The World Bank: Washington, DC. ________ (2007), “Control and Accountability Mechanisms in Conditional Cash Transfer Programs: A Review of Programs in Latin America and the Caribbean” Operational Innovations in Latin America and the Caribbean, Vol. 1. No. 1. ________ (2003a), “World Development Report 2004. Making Services Work for Poor People,” Washington DC: The World Bank and Oxford University Press ________ (2003b), “Country Financial Accountability Assessment Guidelines to Staff”, World Bank. Financial Management Sector Board.

World Bank, “Actionable Governance Indicators Data Portal”, accessed on June 29, 2010. Available online at [https://www.agidata.org/main/Home.ashx] World Bank, “Governance Diagnostic Surveys. Survey Instruments - Sample Questionnaires”, accessed on June 25, 2010. Available online at [http://web.worldbank.org/WBSITE/EXTERNAL/WBI/EXTWBIGOVANTCOR/0,,conten tMDK:20726143~menuPK:1740556~pagePK:64168445~piPK:64168309~theSitePK:1740 530,00.html].

World Health Organization (2008), “Toolkit on monitoring health systems strengthening: Health Systems Governance,” Geneva: World Health Organization

38

Annexes

Annex 1. Overview of governance and social protection assessment tools

Tool

Approach to data and analysis

Focus areas Financial Management

General governance assessment tools France, Set of quantitative Institutional indicators drawing Profiles (IP) on expert opinion and factual data United States, Set of quantitative MCC indicators drawing Scorecard on third-party indicators World Bank, Set of quantitative Country indicators drawing Policy and on third-party Institutional indicators Assessment (CPIA) World Bank Set of quantitative Institute, WGI indicators drawing on third-party indicators EC, Incentive No detailed Tranche information Methodology available (ITM) United Qualitative expert Kingdom, perceptions CGA drawing on secondary sources

1

2

3

4

5

Public administration

Human Resource Manage ment

·

Transparency and Accountability

·

·

Corruption

Service delivery

·

·

·

·

Other Relevant Areas

-Decentralization -Micro-lending

Gender Equality

·

·

·

·

·

·

·

· ·

·

· ·

·

· ·

·

Social Protection

-Equity of public resource use -Old-age/Pension benefit

Voice and Press Freedom

-Payment exemption for basic services for the poor -Pro-poor policy

39

Tool

Approach to data and analysis

Focus areas Financial Management

7

Switzerland, MERV

8

InterAmerican Development Bank, DGIA

9

10

11

12

World Bank Institute, Governance and AntiCorruption Country Survey (GAC) African Development Bank, CGP Germany, Criteria Catalogue (CC) ADB, Governance Risk Assessment

and on a mandatory set of third-party indicators Qualitative expert perceptions drawing on secondary sources and third-party indicators Qualitative expert perceptions drawing on secondary sources and third-party indicators Quantitative data generated through representative surveys

Qualitative expert perceptions drawing on secondary sources Qualitative expert perceptions drawing on secondary sources Qualitative expert perceptions drawing on secondary sources

Public administration

Human Resource Manage ment

Transparency and Accountability

Corruption

Service delivery

Other Relevant Areas

Social Protection

-Perceptions of policy effectiveness

· ·

· ·

·

·

· - E-government -Information technologies

·

-Reforms agreement by government staff

·

·

·

·

·

·

·

·

·

·

·

·

·

·

·

·

·

·

·

-Additional expenses due to services not received -Complaints at social security institutions

-Cooperative stance within the international community

40

Tool

Approach to data and analysis

Focus areas Financial Management

13

14

15

16

17

18

and Risk Management Plans (GRARMP) Switzerland, Key Questions (KQ) Switzerland, Governance as transversal (GT) United States, DGSAF

Netherlands, Strategic Governance and Corruption Analysis (SGACA) Sweden, Power Analysis (PA)

United Kingdom, Drivers of

Public administration

Human Resource Manage ment

Transparency and Accountability

Corruption

Service delivery

Other Relevant Areas

Social Protection

and primary research for updating Qualitative expert perceptions drawing on secondary sources Qualitative expert analysis drawing on secondary sources Qualitative expert perceptions drawing on secondary sources Qualitative expert perceptions drawing on secondary sources

Qualitative expert perceptions drawing on secondary sources and a series of closed/open workshops with key stakeholders at various levels of society Qualitative expert perceptions drawing on

· ·

·

·

· ·

·

· ·

·

· ·

·

· ·

-Beneficiaries participation in policy design

· · ·

· ·

· ·

-Donors’ interests assessment

-Historical and geographical factors -Donors’ interests assessment

-Influence of poor people and organizations of poor people in the decision – making process

· -Historical factors -Role of aid

-Promotion of pro – poor growth

41

Tool

Approach to data and analysis

Focus areas Public administration

Human Resource Manage ment

Transparency and Accountability

·

·

·

·

·

·

Financial Management

19

20

21

22

23

24

Change secondary sources (DoCh) Specific/thematic governance assessment tools IMF, Qualitative expert Assessment of perceptions AML/CFT drawing on secondary sources IMF, Central Questionnaire and Bank qualitative expert Safeguards perceptions IMF, Fiscal Qualitative expert Reports on perceptions Observance of drawing on Standards and secondary sources Codes (ROSC) PEFA Set of indicators Programme, drawing on expert opinion and PEFA factual data World Bank, Analysis of Public fiscal/budgetary Expenditure data; interviews Review (SP-PER)

World Bank, CFAA

Qualitative expert analysis drawing on primary

·

·

·

·

Corruption

Service delivery

Other Relevant Areas

Social Protection

·

·

·

·

-Budget allocation for social protection, trends and processes -Adequacy of social protection programs compared with selected benchmarks -Equity of social protection programs -Old-age/Pension benefit -Unemployment benefit -Risk management

· 42

Tool

Approach to data and analysis

Focus areas Financial Management

25

26

27

28

29

30

31

OECD/DAC JV on Procurement UNODC UN Convention against Corruption Checklist Germany, Conflict Prevention/Ea rly Warning Germany, Security Sector Reform Assessment World Bank, CPAR

World Bank, Upstream ProblemDriven Governance Diagnostic (UPGD) EC, Human Rights Fact

documents and secondary sources and interviews Set of indicators drawing on expert opinion and factual data Quantitative and qualitative information obtained from official governmental sources Qualitative expert perceptions drawing on secondary sources No detailed information available

Public administration

Human Resource Manage ment

·

Qualitative expert perceptions drawing on secondary sources Variable

· ·

·

Transparency and Accountability

·

·

·

·

· ·

Corruption

·

Service delivery

Social Protection

-Procurement systems assessment

-Procurement systems assessment

· ·

Other Relevant Areas

·

-Problem – driven framework adaptable to any program or institution

Qualitative expert perceptions

43

Tool

Approach to data and analysis

Focus areas Financial Management

Sheet 32

33

34

35

36

37

Netherlands, Stability Assessment Framework, (SAF)

United Kingdom, Strategic Conflict Assessment United States, Conflict Assessment Framework United States, AntiCorruption Assessment Framework, (ACF) EC, Sector Governance Analysis Framework World Bank, Actionable Governance Indicators (AGI)

drawing on secondary sources Expert perceptions drawing on secondary sources and a set of indicators for trend analysis Qualitative expert perceptions drawing on secondary sources Qualitative expert perceptions drawing on secondary sources Qualitative expert perceptions drawing on secondary sources

Qualitative expert perceptions drawing on secondary sources Quantitative and qualitative assessment based on secondary sources

Public administration

Human Resource Manage ment

Transparency and Accountability

·

·

·

·

·

· ·

·

Corruption

·

·

·

·

·

·

·

Service delivery

·

·

Other Relevant Areas

-Deterioration of state functions (including basic services)

Social Protection

-Pensions payment failures

- Corruption indicators related to healthcare - Corruption indicators related to education

44

Tool

Approach to data and analysis

Focus areas Financial Management

Social Protection 38 ILO, Social Security Database (SSD)

Public administration

HRM

Transparency and accountability

Corruption

Service delivery

Quantitative data collected using a questionnaire

·

Other Relevant Areas

Social Protection

Indicators available for one of each of the following branches: medical care, sickness, maternity, old-age, invalidity, survivors', family, employment injury and unemployment.

-Expenditure as percentage of the GDP -Number of programs -Number of laws and antiquity -Coverage -Employee and employer contribution rate -Period and residence condition for being eligible -Duration and evolution of the benefits It also contains branch specific indicators.

-Maternity benefit -Old-age/Pension benefit -Invalidity benefit -Survivors' benefit -Family benefit -Employment injury benefit -Unemployment benefit.

45

Tool

Approach to data and analysis

Focus areas Financial Management

39

ILO, Social Protection Expenditure and Performance Reviews (SPEPR) EC, Mutual Information System on Social Protection (MISSP)

Quantitative data collected form secondary sources

41

EC, ESSPROS

Qualitative assessment based on country specific legislation

42

CE, MISSCEO

Qualitative assessment based on country specific legislation

40

Qualitative assessment based on country specific legislation

43

OECD, Benefits and Wages

44

OECD, Social Expenditure Database

Qualitative assessment based en statistical models Quantitative assessment based on official sources

45

OECD, Family

Quantitative assessment based

Public administration

HRM

Transparency and accountability

Corruption

Service delivery

Other Relevant Areas

Social Protection

-Account of receipts and expenditure - Coverage - Distributional performance

·

-Contributions rates and ceilings - Invalidity benefit - Old-age/Pension benefit - Unemployment benefit

·

·

·

- Invalidity benefit - Old-age/Pension benefit Survivor’s benefit - Family benefit - Unemployment benefit - Housing benefit - Invalidity benefit -Maternity benefits -Old-age/Pension benefit - Employment injury benefit - Unemployment benefit - Effectiveness of social benefits in supporting family incomes.

- Aggregate data on social expenditure - Data on itemized tax rates - Public spending on family benefits

46

Tool

Approach to data and analysis

Focus areas Financial Management

Database

on secondary sources and qualitative assessment base on country specific legislation

46

OECD, Pension Database

Quantitative assessment based on statistical models and official sources

47

OECD, Sickness Disability and Work (project)

Quantitative assessment based on official sources

Public administration

HRM

Transparency and accountability

Corruption

Service delivery

Other Relevant Areas

Social Protection

- Child support (maintenance) system - Maternity benefit - Additional leave entitlements of working parents - Typology of childcare and early education services - Quality of childcare workers -Old-age/Pension benefit - Aggregate data on contribution rates - Aggregate data on pension expenditure - Value of pension funds - Disability benefit - Disability employment

Sources: OECD 2009 and 2008 (1) Assessment framework: Institutional Profiles Database III. http://www.cepii.fr/institutions/09-12_DGT_IPD%202009_cahiers-2009-14%20ENG.pdf (2) Assessment framework: Report on the Criteria and Methodology for Determining the Eligibility of Candidate Countries for Millennium Challenge Account Assistance in Fiscal Year 2010. http://www.mcc.gov/mcc/bm.doc/mcc-report-fy09-criteriaandmethodology.pdf (3) Assessment framework: Country Policy and Institutional Assessment. http://siteresources.worldbank.org/IDA/Resources/731531181752621336/CPIA09CriteriaB.pdf (4) Assessment framework. Governance Matters VIII. http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1424591 (5) Assessment tool not available.

47

(6) Assessment framework: Country Governance Analysis. http://www.reliefweb.int/rw/lib.nsf/db900sid/JBRN-737J8E/$file/dfid-humanitarianactionfeb07.pdf?openelement (7) Assessment tool not available. Indicators taken from Donor Approaches to Governance Assessment Sourcebook 2009. http://www.oecd.org/dataoecd/25/12/42472200.pdf (8) Assessment tool not available. Indicators taken from the underlying Governance Indicators database. http://www.iadb.org/datagob/ (9) Questionnaire for households: http://siteresources.worldbank.org/INTWBIGOVANTCOR/Resources/sample_user.pdf. Questionnaire for public officials: http://siteresources.worldbank.org/INTWBIGOVANTCOR/Resources/sample_publicofficial.pdf (10) Assessment tool not available. Indicators taken from country - specific reports (narrative), available when country partners agree. http://www.afdb.org/en/documents/project-operations/country-governance-profiles/ (11) Assessment framework: Development-Oriented Transformation in Conditions of Fragile Statehood and poor Government Performance. http://www.oecd.org/dataoecd/4/38/43480415.pdf (12) Assessment tool not available. Indicators taken from country – specific reports (narrative). See for example the Nepal’s report: Sector Level Risk Assessment and Risk Management Plan to Assess Public Financial Management, Procurement, and Corruption Risks in the Urban Development Sector. http://www.adb.org/Documents/Assessments/Other-Assessments/NEP/Risk-Assessment-Plan-June2009.pdf (13) Assessment tool not available. Indicators taken from Donor Approaches to Governance Assessment Sourcebook 2009. http://www.oecd.org/dataoecd/25/12/42472200.pdf (14) Assessment framework: Governance as a Transversal Theme: an Implementation Guide. www.deza.admin.ch/ressources/resource_en_156840.pdf (15) Assessment framework: Conducting a DG Assessment: a Framework for Strategy Development. http://www.usaid.gov/our_work/democracy_and_governance/publications/pdfs/pnach305.pdf (16) Assessment framework: Framework for Strategic Governance and Corruption Analysis (SGACA). http://www.minbuza.nl/dsresource?objectid=buzabeheer:43185&type=org (17) Assessment framework: Power Analysis – Experiences and Challenges. http://sidapublications.citat.se/interface/frmoptimaker3.asp?doctype=3&order=createdate%20DESC&departmentid=298&topheight=55&headerheight=23&f otheight=0&leftframewidth=300&width=820&stylesheet=sida.css&frameout=0&language=14&login=True&username=sida2&password=sida2 (18) Assessment framework: What does Drivers of Change mean for DFID? http://webarchive.nationalarchives.gov.uk/+/http://www.dfid.gov.uk/Contracts/files/ojec_5512_doc_approach.pdf (19) Assessment framework: Reference Guide to Anti – Money Laundering and Combating the Financing of Terrorism. http://siteresources.worldbank.org/EXTAML/Resources/396511-1146581427871/Reference_Guide_AMLCFT_2ndSupplement.pdf. Neither social protection indicators nor adaptable indicators for social protection contained.

(20) Assessment framework: Safeguards Assessments. http://www.imf.org/external/np/pp/eng/2009/121509.pdf. Protecting IMF Resources: Safeguards Assessments of Central Banks. http://www.imf.org/external/np/exr/facts/safe.htm. Neither social protection indicators nor adaptable indicators for social protection contained.

48

(21) Questionnaire on Fiscal Institutions. http://www.imf.org/external/np/fad/trans/question/quest.pdf (22) Assessment framework: Public Financial Management. Performance Measurement Framework. http://www.pefa.org/pfm_performance_file/the_framework_English_1193152901.pdf (23) Assessment framework: Core Guidance: Preparing PERs for Human Development. http://siteresources.worldbank.org/EXTPERGUIDE/Resources/PERComplete.pdf. Includes a complete section for social protection assessment.

(24) Assessment framework: Country Financial Accountability Assessment Guidelines to Staff. http://www1.worldbank.org/publicsector/pe/cfaaguidelines.pdf.

(25) Assessment framework: Methodology for Assessment of National Procurement Systems. http://www.oecd.org/dataoecd/1/36/37130136.pdf (26) Assessment made using a computer-based application: Comprehensive self-assessment checklist http://www.unodc.org/unodc/en/treaties/CAC/self-assessment.html (27) Assessment tool not available.

(28) Assessment tool not available.

(29) Assessment tool not available. Indicators taken from country – specific reports. See for example the report for Philippines: Country Procurement Assessment Report. http://wwwwds.worldbank.org/external/default/WDSContentServer/WDSP/IB/2009/04/21/000334955_20090421082041/Rendered/PDF/476020REVISED010WB0CP AR0081April020.pdf (30) Variable methodology: Problem Driven Governance and Political Economy Analysis. The problem – based methodology permits to assess the performance of almost every institution, process and program in the country, sector, or project levels. The framework emphasizes a focus on specific problems or vulnerabilities, as well as the need to understand political economy drivers by examining them in a systematic way. http://vle.worldbank.org/bnpp/files/Fritz%20Kaiser%20Levy%202009%20PGPE_book_12-15-09%20FINAL.pdf (31) Assessment tool not available.

(32) Assessment framework: The Stability Assessment Framework: Designing Integrated Responses for Security, Governance and Development. http://www.clingendael.nl/publications/2005/20050200_cru_paper_stability.pdf.

(33) Variable methodology: Conducting Conflict Assessments: Guidance Notes. The methodology gives particular attention to the development of strategies/options for programming interventions. http://webarchive.nationalarchives.gov.uk/+/http://www.dfid.gov.uk/pubs/files/conflict-assess-guidance.pdf (34) Variable methodology: Conducting a Conflict Assessment: A Framework for Strategy and Program Development. Designed to help missions: 1) identify and prioritize the causes and consequences of violence and instability that are most important in a given country context; 2) understand how existing development programs interact with these factors; and 3) determine where development and humanitarian assistance can most effectively support local efforts to manage conflict and build peace. http://www.usaid.gov/our_work/cross-cutting_programs/conflict/publications/docs/CMM_ConflAssessFrmwrk_May_05.pdf (35) Assessment framework: Anticorruption Assessment Handbook. http://www.usaid.gov/our_work/democracy_and_governance/technical_areas/anticorruption_handbook/index.html

49

(36) Assessment framework: Analyzing and Addressing Governance in Sector Operations. The analysis framework offers guidance on how to address governance in sector operations in a systematic and comprehensive way. It can be applied for example to health, water and sanitation, education, and transport. http://www.nilsboesen.dk/uploads/docs/Sector%20Governance2008.pdf (37) Assessment tool not available. Indicators taken from Actionable Governance Indicators Data Portal: https://www.agidata.org/main/Home.ashx (38) Assessment framework The Social security database: User Guide Book. http://www.ilo.org/dyn/sesame/SESHELP.SSDBCodeBookIndex (39) Assessment framework: Social protection expenditure and performance reviews. Methodological note on definitions, classifications and performance indicators. http://www.ilo.org/public/english/protection/secsoc/downloads/stat/spersmet.pdf (40) Assessment framework: Organisation of Social Protection. Charts and Descriptions. http://ec.europa.eu/employment_social/missoc/2007/organisation_en.pdf (41) Assessment framework: ESSPROS Manual. http://ec.europa.eu/eurostat/ramon/statmanuals/files/KS‐RA‐07‐027‐EN.pdf (42) Assessment framework: Comparative Tables of Social Protection Systems: Correspondent’s Manual 2008. http://www.coe.int/t/dg3/socialpolicies/socialsecurity/Source/corrmanual_en.pdf (43) Assessment framework: Benefits and Wages 2007. OECD Indicators. http://www.oecd.org/document/33/0,3343,en_2649_34637_39619553_1_1_1_1,00.html#related (44) Assessment framework: How Expensive is the Welfare State? Gross and Net Indicators in the OECD Social Expenditure Database (SOCX) http://www.oecdilibrary.org/docserver/download/fulltext/5ks712h5cg7l.pdf?expires=1277863783&id=0000&accname=freeContent&checksum=892F72B32 012FF1670C2697DFFA1F108 (45) Assessment tool not available. Indicators taken from OECD Family Database. http://www.oecd.org/document/4/0,3343,en_2825_497118_37836996_1_1_1_1,00.html (46) Assessment tool not available. Indicators taken from Pensions Database. http://www.oecd.org/document/16/0,3343,en_2649_34757_45558288_1_1_1_1,00.html (47) Assessment framework: The OECD Sickness, Disability and Work Project. http://www.oecd.org/document/20/0,3343,en_2649_34747_38887124_1_1_1_1,00.html3

50

Annex 2. List of indicators related to social protection issues

3

6

Tool World Bank, Country Policy and Institutional Assessment (CPIA)

United Kingdom, Country Governance Analysis (CGA)

Social Protection Issue  Equity of Public Resource Use 

Labor market



Old-age/Pension benefit



Payment exemption for basic services for the poor Pro-Poor Policy



 9

World Bank Institute, Governance and Anti-Corruption Country Survey (GAC)

  

Perceptions of the effectiveness of policy Additional expense due to services requested not received Payment and bribes Complaining at social security institutions

Indicators/Questions  Strong poverty diagnosis is in place that very clearly identifies poor, vulnerable groups, and those lacking services  There are no egregious regressive revenue sources  Public expenditures are fully aligned with poverty reduction priorities  Labor market regulations and active labor market policies promote broad access to employment in the formal sector and reflects a balance between social protection and job creation objectives in accordance with the economic circumstances and values of the country.  A diversified, well-supervised, and appropriate combination of pension and savings programs (including mandatory, voluntary, public, private, funded, pay-as-you go, contributory and noncontributory programs) provide affordable, adequate, sustainable and robust income security to most of the potentially vulnerable groups with minimal distortions in the operation of labor markets.  Extent to which poor people are required to pay for basic services  Effective of payment exemption arrangements for poor and vulnerable people  Extent to which poverty reduction is an explicit policy priority and how effectively is the policy decided, implemented and monitored in practice  Extent to which public goods and services are provided in ways that recognize, address and reduce discrimination and allow all citizens – and including women, disabled people and ethnic minorities – to benefit  How much confidence do the poor have in the ability of the government to help solve their problems and in their own ability to influence it?  Additional costs incurred due to failure of the social security institution to provide requested services  

Frequency in which public servants request, or people feel obliged to give retributions like tips/gifts/ bribes, etc. at Social Security attention.

Have you filed any complaints for bad service, delay or mishandling received at some Social Security institution?

51

17

23

Tool Sweden, Power Analysis

World Bank, Social Protection Public Expenditure Review (SP-PER)

Social Protection Issue  Influence of poor people and organizations of poor people in the decision – making process  Budget allocation for social protection, trends and processes  Adequacy of social protection programs compared with selected benchmarks

Indicators/Questions  How can networks and federations of poor people’s organizations (women and men) be heard and represented in decision-making that affects their lives at the family, local, and national levels?  How are the central agents and organizations providing a voice to the poor?



Equity of social protection programs



Old-age/Pension benefit Unemployment benefit Risk management

  

  

  



 32

Netherlands, Stability Assessment Framework (SAF)

Deterioration of state functions (including those of social protection) Old-age/Pension benefit





Expenditure as percentage of the GDP

 

 

Number of programs Number of laws and antiquity



Coverage

   

 

38

ILO, Social Security Database (SSD)



Public expenditure in social protection by program and sector “total” as a share of GDP and total public expenditures - and in absolute values.

Sustainability of social protection budget Adequacy of benefit level (benchmarks will vary by program, for example, average pensions could be compared with average wages, the social pension with the poverty line, unemployment insurance with average wages, unemployment assistance with both average wages and the poverty line, social assistance with the poverty line, wages on public works jobs with the market wage for similar work and so on For pensions, consideration of intergenerational equity and intra-generational equity among participants of different schemes or contribution cohorts.

Contrast the incidence of participation and of payments made where benefits are not uniform in social protection programs.

Procedures and timing for collecting a pension To what degree do contributions rates for pensions discourage employment in the formal sector?

Do social protection programs improve the ability of the household to manage risks by reducing the probability of a shock?

Do social protection programs improve the ability of the household to manage risks by providing an income support through insurance payments (risk mitigation) or transfers (risk coping)?

Is there a deterioration or disappearance of basic state functions (e.g. a failure to provide essential services such as health care, education, sanitation, public transportation)?

Has the state failed to pay the salaries of government employees and armed forces or to meet other financial obligations to its citizens, such as pension payments?

Total expenditure (public and private) on social protection (percentage of GDP) Mandatory private expenditure as percentage of GDP. Number of social protection programs.

Number of current social protection laws Date of the first social protection law Coverage of the main social protection program -all residents; all working population (self employed, public and private sector employees); all working population with exception; all salaried workers and

52

Tool

39

ILO, Social

Social Protection Issue

Indicators/Questions assimilated categories (both public and private sector employees); private sector employees; public sector employees.  Employee and employer contribution to social protection programs (discretionary, whole cost, flat rate amount).  Employer contribution to social protection programs (discretionary, whole cost, flat rate amount).  Government contribution for social protection programs (discretionary, whole cost, flat rate amount).  Period and condition of residence in the country to be eligible for benefits of social protection programs.  Duration of benefits in social protection programs.  Evolution of benefit in social protection programs (constant, indexed, decreasing, increasing, discretionary evolution (no fixed rules)).  Existence of parental leave.  Are fathers and mothers eligible for paid parental leave?  Existence of maternity benefits in kind.  Legal retirement age for male.  Legal retirement age for female.  Does early retirement provision exist?  Is there gender discrimination in calculation of cash benefits (other than retirement age)?



Employee and employer contribution rate

 

Period and residence condition for being eligible Duration and evolution of the benefits



Maternity benefit



Old-age / Pension benefit



Survivors' benefit

   

Calculation rate of the pension of a surviving wife.

Calculation rate of the pension of a surviving husband.

Calculation rate of the pension for the first surviving child.

In case of death, what cash benefit (percentage of earnings) does a surviving spouse receive?



Unemployment benefit.



Family benefit

 

Work injury Invalidity benefit

        



Account of receipts and



Maximum age to receive unemployment benefit.

Is accumulation of unemployment benefits possible with other earnings?

Are there special unemployment provisions for old-aged unemployed?

Age limit for payment of benefits for a child.

Age limit for payment of benefits for a student.

Is there equal benefit amount for each child?

Is there any variation in the benefit amount according to the age of the child?

Is the travel between home and work also covered by work injury programs?

Calculation method of pension (flat rate, according to incapacity level, according to previous earning, according to number of years of insurance, according to the age).

Administration costs of social protection schemes.

53

Tool Protection Expenditure and Performance Reviews (SPEPR)

40

41

42

EC, Mutual Information System on Social Protection

EC, ESSPROS

CE, MISSCEO

Social Protection Issue expenditure  Coverage

Indicators/Questions 



Distributional performance





Contributions rates and ceilings



Percentage of persons covered (by gender, age, labor market status) within the whole population or the target group.

Range of contingencies, risks and needs covered (old-age and survivors, disability, unemployment, sickness and health, unemployment, maternity, family and children, poverty.

Portion of cash benefits actually distributed to horizontal distribution groups (with a particular attention to the gender distribution, formal/informal sector distribution, and distribution to other groups identified as the most vulnerable) and access of member of these groups to social protection and other basic social services Contributions rate and ceilings for social protection



Invalidity benefit



Old-age / Pension benefit



Unemployment benefit

    

Preferential employment of handicapped persons Minimum period of affiliation for entitlement Minimum and maximum pensions Supplements for dependants: Spouse/Children Earnings taken as reference and ceiling for unemployment benefits



Invalidity benefit



Old-age / Pension benefit

 

Survivor’s benefit Family benefit



Housing benefit



Unemployment benefit



Invalidity benefit

                

Early retirement benefit due to reduced capacity to work Economic integration of the handicapped Rehabilitation Assistance in carrying out daily tasks Anticipated old age pension Assistance in carrying out daily tasks Funeral expenses Birth grant Parental leave benefit Child daycare Rent benefit Social housing Benefit to owner-occupiers Vocational training allowance Placement services and job-search assistance Are workers obliged to prove that the reason they are not at work is because they are sick?

Are there any other tests besides incapacity for work and any qualifying period that the applicant must satisfy before receiving the benefit?



54

Tool

43

44

45

Social Protection Issue  Maternity benefits



Old-age / Pension benefit



Survivor’s benefit



Employment injury benefit



Unemployment benefit



Effectiveness of social benefits in supporting family incomes.

OECD, Social Expenditure Database



Aggregate data on social expenditure Data on itemized tax rates

OECD, Family Database



Public spending on family benefits



Child support (maintenance) system



Maternity benefit

OECD, Benefits and Wages



Indicators/Questions  Are there any rules on how much time should be taken before and after the birth?  Which is the amount employers are obliged to pay (this maybe a percentage of the employee’s previous wage)?  How long are employers obliged to pay?  Special supplements not related to dependants (cold weather bonuses, payments for those who reach a very great age (80 – 90 years), Christmas or holiday bonuses).  How long does someone have to have been paying contributions, working or residing in the country before they are able to make a claim for benefits?  What happens to the survivor’s pension if the spouse remarries (or cohabits) with a new partner?

Which of the following are entitled to a survivor’s pension: divorced spouse, mixed sex cohabitants (couples living together as man and wife outside of wedlock), same sex cohabitants (homosexual couples living together), illegitimate children, foster children, adopted children.  How long, after the appearance of the incapacity does a person have to wait before the benefit is paid?  Is there a list of diseases that have been recognized as occupational in nature?  Is the recipient re-examined if circumstances change i.e. their condition improves or worsens?  Insurance is voluntary or compulsory for employees?  Initial payment rate (% of earnings base)  Permitted employment and disregards  Additions for dependent family members  Net effects of taxes and social benefits in supporting family incomes by type of family (single person, lone parent, one-earner married couple with no children, one-earner married couple with two children, two-earner married couple with no children, two-earner married couple with two children)  Net replacement rates over a five-year period  Gross and net current public social expenditure (% of GDP)  Gross and net mandatory private social expenditure (% of GDP)  Average itemized tax rates (some items are inexistent in some countries)  Tax brakes for social purposes (amounts)  Child-related cash transfers to families with children  Public spending on services for families with children  Financial support for families provided through the tax system  Involvement in the determination of child maintenance (parents, court, administrative agency)  Rules for determining the amount of payments  Do different arrangements for children of unmarried parents exist?  Age at which support ends  Maternity leave maximum duration

55

Tool

46

47

OECD, Pension Database

OECD, Sickness Disability and Work

Social Protection Issue



Additional leave entitlements of working parents



Typology of childcare and early education services



Quality of childcare workers



Old-age/Pension benefit



Aggregate data on contribution rates

 

Aggregate data on pension expenditure Value of pension founds



Disability benefit

Indicators/Questions  Eligibility criteria for payments of maternity leave  Payment during maternal leave (% of earnings)  Paternity leave maximum duration  Level of payment during parental leave  Proportion of employed parents with a child under age 1 on leave  Do leave entitlements to care for a dependent relative exist?  Do leave entitlements for personal reasons exist?  Duration of additional leave  Payments conditions during additional leave  Characteristics of the Centre-based day-care  Characteristics of the Family day care  Characteristics of the Pre-school early education programs  Main type of staff  Initial training requirements  Does continuous training exist?  Child – to – staff ratio  Gross pension replacement rates by earnings level (earnings level as multiple of the mean). The gross replacement rate is defined as gross pension entitlement divided by gross pre-retirement earnings  Net (of personal taxes and social contributions) pension replacement rates by earnings level (earnings level as multiple of the mean).  Gross pension replacement rates from public, mandatory private and voluntary private pension schemes  Gross pension wealth by earnings level. The gross pension wealth shows the size of the lump sum that would be needed to buy a flow of pension payments equivalent to that promised by the mandatory pension system in each country  Net (of personal taxes and social contributions) pension wealth by earnings level  Pension contribution rate (% of gross earnings)  Pension contribution revenues (% of the GDP, by employee and employer)  Pension contribution revenues (% of total taxes)  Expenditures on old-age and survivors’ benefits (% of GDP)  Expenditures on old-age and survivors’ benefits (% of government spending)  Assets value in private pension funds (% of GDP)  Assets value in public pension reserves (% of GDP)  Disability benefit recipients in percent of the population aged 20-64  Disability benefit recipients in percent of the population aged 20-64

56

Tool (project)

Social Protection Issue  Disability employment

Indicators/Questions  Employment rates of the working-age population with disability

57

Social Protection Discussion Paper Series Titles No.

Title

1119

Measuring Governance and Service Delivery in Safety Net Programs

by Gloria M. Rubio, September 2011 (online only)

1118

Assessing Safety Net Readiness in Response to Food Price Volatility

by Margaret Grosh, Colin Andrews, Rodrigo Quintana, Claudia Rodriguez-Alas, September 2011

1117

Social Safety Nets in Fragile States: A Community-Based School Feeding Program in Togo, August 2011 (also available in French)

1116

Strengthening Governance of Social Safety Nets in East Asia

by Sara Giannozzi and Asmeen Khan, August 2011 (online only)

1115

International Portability of Health-Cost Coverage: Concepts and Experience

by Martin Werding and Stuart McLennan, July 2011 (online only)

1114

Liberia’s Cash For Work Temporary Employment Project: Responding to Crisis in Low Income, Fragile Countries

by Colin Andrews, Prospère Backiny-Yetna, Emily Garin, Emily Weedon, Quentin Wodon and Giuseppe Zampaglione, July 2011

1113

Employability and Productivity among Older Workers: A Policy Framework and Evidence from Latin America

by Edmundo Murrugarra, July 2011 (online only)

1112

Cash Transfers, Children and the Crisis: Protecting Current and Future Investments

by Ariel Fiszbein, Dena Ringold, Santhosh Srinivasan, June 2011 (online only)

1111

Severance Pay Programs around the World: History, Rationale, Status, and Reforms

by Robert Holzmann, Yann Pouget, Milan Vodopivec and Michael Weber, May 2011 (online only)

1110

Portability of Pension, Health, and other Social Benefits: Facts, Concepts, Issues

by Robert Holzmann and Johannes Koettl, May 2011 (online only)

1109

Disability and Poverty in Developing Countries: A Snapshot from the World Health Survey

by Sophie Mitra, Aleksandra Posarac and Brandon Vick, April 2011

1108

Advancing Adult Learning in Eastern Europe and Central Asia

by Christian Bodewig and Sarojini Hirshleifer, April 2011 (online only)

1107

Results Readiness in Social Protection & Labor Operations

by Laura Rawlings, Maddalena Honorati, Gloria Rubio and Julie Van Domelen, February 2011

1106

Results Readiness in Social Protection & Labor Operations: Technical Guidance Notes for Social Service Delivery Projects

by Julie Van Domelen, February 2011

1105

Results Readiness in Social Protection & Labor Operations: Technical Guidance Notes for Social Safety Nets Task Teams

by Gloria Rubio, February 2011

1104

Results Readiness in Social Protection & Labor Operations: Technical Guidance Notes for Social Funds Task Teams

by Julie Van Domelen, February 2011

1103

Results Readiness in Social Protection & Labor Operations: Technical Guidance Notes for Labor Markets Task Teams

by Maddalena Honorati, February 2011

1102

Natural Disasters: What is the Role for Social Safety Nets?

by Larissa Pelham, Edward Clay and Tim Braunholz, February 2011

1101

North-South Knowledge Sharing on Incentive-based Conditional Cash Transfer Programs

by Lawrence Aber and Laura B. Rawlings, January 2011

1008

Social Policy, Perceptions and the Press: An Analysis of the Media’s Treatment of Conditional Cash Transfers in Brazil

by Kathy Lindert and Vanina Vincensini, December 2010 (online only)

1007

Bringing Financial Literacy and Education to Low and Middle Income Countries: The Need to Review, Adjust, and Extend Current Wisdom

by Robert Holzmann, July 2010 (online only)

1006

Key Characteristics of Employment Regulation in the Middle East and North Africa

by Diego F. Angel-Urdinola and Arvo Kuddo with support from Kimie Tanabe and May Wazzan, July 2010 (online only)

1005

Non-Public Provision of Active Labor Market Programs in Arab-Mediterranean Countries: An Inventory of Youth Programs

by Diego F. Angel-Urdinola, Amina Semlali and Stefanie Brodmann, July 2010 (online only)

1004

The Investment in Job Training: Why Are SMEs Lagging So Much Behind?

by Rita K. Almeida and Reyes Aterido, May 2010 (online only)

1003

Disability and International Cooperation and Development: A Review of Policies and Practices

by Janet Lord, Aleksandra Posarac, Marco Nicoli, Karen Peffley, Charlotte McClain-Nhlapo and Mary Keogh, May 2010

1002

Toolkit on Tackling Error, Fraud and Corruption in Social Protection Programs

by Christian van Stolk and Emil D. Tesliuc, March 2010 (online only)

1001

Labor Market Policy Research for Developing Countries: Recent Examples from the Literature - What do We Know and What should We Know?

by Maria Laura Sanchez Puerta, January 2010 (online only)

0931

The Korean Case Study: Past Experience and New Trends in Training Policies

by Young-Sun Ra and Kyung Woo Shim, December 2009 (online only)

0930

Migration Pressures and Immigration Policies: New Evidence on the Selection of Migrants

by Johanna Avato, December 2009 (online only)

0929

Ex-Ante Methods to Assess the Impact of Social Insurance Policies on Labor Supply with an Application to Brazil

by David A. Robalino, Eduardo Zylberstajn, Helio Zylberstajn and Luis Eduardo Afonso, December 2009 (online only)

0928

Rethinking Survivor Benefits

by Estelle James, December 2009 (online only)

0927

How Much Do Latin American Pension Programs Promise to Pay Back?

by Alvaro Forteza and Guzmรกn Ourens, December 2009 (online only)

0926

Work Histories and Pension Entitlements in Argentina, Chile and Uruguay

by Alvaro Forteza, Ignacio Apella, Eduardo Fajnzylber, Carlos Grushka, Ianina Rossi and Graciela Sanroman, December 2009 (online only)

0925

Indexing Pensions

by John Piggott and Renuka Sane, December 2009 (online only)

0924

Towards Comprehensive Training

by Jean Fares and Olga Susana Puerto, November 2009

0923

Pre-Employment Skills Development Strategies in the OECD

by Yoo Jeung Joy Nam, November 2009

0922

A Review of National Training Funds

by Richard Johanson, November 2009

0921

Pre-Employment Vocational Education and Training in Korea

by ChangKyun Chae and Jaeho Chung, November 2009

0920

Labor Laws in Eastern European and Central Asian Countries: Minimum Norms and Practices

by Arvo Kuddo, November 2009 (online only)

0919

Openness and Technological Innovation in East Asia: Have They Increased the Demand for Skills?

by Rita K. Almeida, October 2009 (online only)

0918

Employment Services and Active Labor Market Programs in Eastern European and Central Asian Countries

by Arvo Kuddo, October 2009 (online only)

0917

Productivity Increases in SMEs: With Special Emphasis on In-Service Training of Workers in Korea

by Kye Woo Lee, October 2009 (online only)

0916

Firing Cost and Firm Size: A Study of Sri Lanka's Severance Pay System

by Babatunde Abidoye, Peter F. Orazem and Milan Vodopivec, September 2009 (online only)

0915

Personal Opinions about the Social Security System and Informal Employment: Evidence from Bulgaria

by Valeria Perotti and Maria Laura Sรกnchez Puerta, September 2009

0914

Building a Targeting System for Bangladesh based on Proxy Means Testing

by Iffath A. Sharif, August 2009 (online only)

0913

Savings for Unemployment in Good or Bad Times: Options for Developing Countries

by David Robalino, Milan Vodopivec and Andrรกs Bodor, August 2009 (online only)

0912

Social Protection for Migrants from the Pacific Islands in Australia and New Zealand

by Geoff Woolford, May 2009 (online only)

0911

Human Trafficking, Modern Day Slavery, and Economic Exploitation

by Johannes Koettl, May 2009

0910

Unemployment Insurance Savings Accounts in Latin America: Overview and Assessment

by Ana M. Ferrer and W. Craig Riddell, June 2009 (online only)

0909

Definitions, Good Practices, and Global Estimates on the Status of Social Protection for International Migrants

by Johanna Avato, Johannes Koettl, and Rachel Sabates-Wheeler, May 2009 (online only)

0908

Regional Overview of Social Protection for Non-Citizens in the Southern African Development Community (SADC)

by Marius Olivier, May 2009 (online only)

0907

Introducing Unemployment Insurance to Developing Countries

by Milan Vodopivec, May 2009 (online only)

0906

Social Protection for Refugees and Asylum Seekers in the Southern Africa Development Community (SADC)

by Mpho Makhema, April 2009 (online only)

0905

How to Make Public Works Work: A Review of the Experiences

by Carlo del Ninno, Kalanidhi Subbarao and Annamaria Milazzo, May 2009 (online only)

0904

Slavery and Human Trafficking: International Law and the Role of the World Bank

by María Fernanda Perez Solla, April 2009 (online only)

0903

Pension Systems for the Informal Sector in Asia edited by Landis MacKellar, March 2009 (online only)

0902

Structural Educational Reform: Evidence from a Teacher’s Displacement Program in Armenia

by Arvo Kuddo, January 2009 (online only)

0901

Non-performance of the Severance Pay Program in Slovenia

by Milan Vodopivec, Lilijana Madzar, Primož Dolenc, January 2009 (online only)

To view Social Protection Discussion papers published prior to 2009, please visit www.worldbank.org/sp.

S P D I S C U S S I O N PA P E R

Abstract This paper develops a framework to assess organizational performance in the delivery of social safety nets. Specifically, it provides guidance to task teams and program managers for identifying indicators of governance and service quality in targeted cash transfer programs.

The paper identifies governance issues along the results chain of service delivery and suggests policy and performance indicators for assessing program inputs - human resources, financing and resource management; and program activities - operational procedures, MIS and control. It also suggests indicators of organizational performance and the quality of outputs, including demand-side accountability mechanisms.

HUMAN DEVELOPMENT NETWORK

About this series...

Social Protection Discussion Papers are published to communicate the results of The World Bank’s work to the development community with the least possible delay. The typescript manuscript of this paper therefore has not been prepared in accordance with the procedures appropriate to formally edited texts. The findings, interpretations, and conclusions expressed herein are those of the author(s), and do not necessarily reflect the views of the International Bank for Reconstruction and Development / The World Bank and its affiliated organizations, or those of the Executive Directors of The World Bank or the governments they represent. The World Bank does not guarantee the accuracy of the data included in this work.

For more information, please contact the Social Protection Advisory Service, The World Bank, 1818 H Street, N.W., MSN G7-703, Washington, D.C. 20433 USA. Telephone: (202) 458-5267, Fax: (202) 614-0471, E-mail: socialprotection@worldbank.org or visit us online at www.worldbank.org/sp.

Measuring Governance and Service Delivery in Safety Net Programs Gloria M. Rubio

September 2011

NO. 1119