IPPF Independent Progress Review 2012 DRAFT Report – Rahnuma-FPAP, Pakistan International Planned Parenthood Federation 12 October 2012 Seema Khan Afeef Mahmood Mohsin Saeed Khan
Acronyms AJK CBD CPR CYP CSO DFID FGD GBV GPAF HQ IEC BCC IPPF LHW LHV MA M&E MDGs MIS MISP MP PAPAC PDMA PMO QAD R-FPAP RTI SARO SRH SRHR STI TBA VfM WHO
Azad Jammu and Kashmir Community Based Distributors Contraceptive Prevalence Rate Couple Years Protection Civil Society Organisation Department for International Development Focus Group Discussion Gender-based Violence Global Poverty Action Fund Head Quarter Information, Education and Communication Behaviour Change Communication International Planned Parenthood Foundation Lady Health Worker Lady Health Visitor Member Association Monitoring and Evaluation Millennium Development Goals Management Information System Minimum Initial Service Package Member of Parliament Pakistan Alliance for Postabortion Care Provincial Disaster Management Authority Project Management Office Quality Assurance Doctor Rahnuma - Family Planning Association Pakistan Rahnuma Training Institute South Asia Regional Office Sexual and Reproductive Health Sexual and Reproductive Health Rights Sexually Transmitted Infections Traditional Birth Attendant Value for Money World Health Organisation
Social Development Direct Document Title Pg 1
Contents Executive Summary ........................................................................................................................ 3 1. Introduction ................................................................................................................................. 6 1.1 Purpose of evaluation .................................................................................................................. 6 1.2 Methodology ................................................................................................................................ 6 1.3 Context ........................................................................................................................................ 7 1.4 R-FPAP background and overall budget context .......................................................................... 8 2. Evaluation results........................................................................................................................ 9 2.1 Relevance ................................................................................................................................... 9 Identifying key focus areas ............................................................................................................ 9 Targeting the needs of the poor and marginalised ....................................................................... 10 Identifying the poorest and most marginalised groups.................................................................. 14 Addressing the needs of vulnerable groups ................................................................................. 15 2.2 Efficiency and Value for Money .................................................................................................. 17 Organisational efficiency ............................................................................................................. 17 Financial efficiency ...................................................................................................................... 19 2.3 Effectiveness ............................................................................................................................. 20 Strengthening access and quality of care..................................................................................... 20 Awareness raising and sensitisation ............................................................................................ 22 Advocacy .................................................................................................................................... 23 2.4 Results and Impact .................................................................................................................... 24 Service delivery results................................................................................................................ 25 Advocacy results ......................................................................................................................... 27 Wider impacts ............................................................................................................................. 27 Additionality and attribution.......................................................................................................... 28 2.5 Value for Money ........................................................................................................................ 29 Unit Costing ................................................................................................................................ 29 Costing Results: Clinical Level Costing ........................................................................................ 29 Costing Results: Organisational Level (Family Planning) ............................................................. 30 Cost Effectiveness....................................................................................................................... 32 2.6 Sustainability ............................................................................................................................. 33 2.7 Lesson Learning and Innovation ................................................................................................ 35 3. Conclusions and Recommendations....................................................................................... 36 3.1 Conclusions ............................................................................................................................... 36 3.2 Lessons Learned ....................................................................................................................... 37 3.3 Recommendations..................................................................................................................... 39 Relevance ................................................................................................................................... 39 Efficiency and Value for Money ................................................................................................... 39 Effectiveness............................................................................................................................... 40 Results and Impact...................................................................................................................... 40 Sustainability ............................................................................................................................... 40 Learning and Innovation .............................................................................................................. 41 ANNEX 1: Documents Reviewed ..................................................................................................... 42 ANNEX 2: List of people interviewed ............................................................................................... 44 ANNEX 3: Financial Management Assessment ................................................................................ 46 ANNEX 4: Methodology for Unit Costing and VfM analysis (FP) ....................................................... 52 ANNEX 5: Additional Tables from Organisational Level Costing ....................................................... 56
Social Development Direct Document Title Pg 2
Executive Summary This document outlines the findings from a country study of the Pakistani International Planned Parenthood Federation (IPPF) Member Association (MA) – Rahnuma-Family Planning Association of Pakistan (R-FPAP) - as a contribution to the DFID Programme Partnership Arrangement (PPA) independent mid-term review. As part of the PPA, IPPF will receive on average £8.5m per year between 2011 and 2013. Whilst IPPF is a global organisation working in more than 173 countries, this PPA gives particular priority to 45 focus countries, as well as directing funding to IPPF’s Regional and Central Offices. The purpose of this review is to assess the achievements, challenges, outcomes and impacts resulting from the DFID PPA. As the PPA provides unrestricted (or core) funding our country reviews have aimed to assess the performance of the MA overall, rather than solely focusing on the impact of the DFID funds. Pakistan has some of the worst maternal and child health indicators in South Asia, with a maternal mortality rate of 276 per 100,000 live births 1. Health service provision and coverage is uneven, and there are significant disparities in terms of access to healthcare and health indicators, between the poorest and wealthiest quintiles, urban and rural areas, as well as between and within provinces. Two-thirds of pregnant women do not access antenatal care, and only 39% of births are attended by a skilled birth attendant. The use of modern contraception is also fairly low and there is high unmet need at 25%.2 R-FPAP is one of the largest non-government providers of family planning services in Pakistan. Its service delivery network, which began in 1958, now includes nearly 5,000 service points in peri-urban and rural areas of Pakistan, comprising 118 permanent clinics (including 9 hospitals) 11 mobile units, 191 associated clinics and 2,433 community-based distributors/ services. FPAP also supports 2,143 private practitioners located in urban and rural areas to provide family planning services. R-FPAP also implements various youth services, including 6 toll-free Youth Help Lines and 36 Youth Resource Centers for males and 16 Youth Resource Centers for females. Conclusions R-FPAP have been instrumental in providing access to quality services by under-served populations. By locating clinics in those areas where no other facilities exist, and providing services at low cost, R-FPAP are addressing the key access barriers of women’s limited mobility, distance and fees. They are well-regarded for their technical expertise in health, and have invested considerable resources in strengthening their Quality of Care approach. Clients are happy with their services, and particularly appreciate the proximity of clinics, low cost of medicines, and provision of advice and information. R-FPAP are particularly skilled in communicating sensitive messages to vulnerable groups, as seen by their Women as Compensation programme, and youth peer education work. They have evolved a tailored approach which involves using healthcare as an entry point, obtaining support from religious scholars, recruiting staff from their own communities, and addressing staff’s own values and attitudes through training first. R-FPAP enjoy a high level of credibility and a strong working relationship with government. They are often invited to provide technical assistance to government departments, and have 1 2
PDHS 2006-2007 Ibid.
Social Development Direct Document Title Pg 3
been able to leverage the trust that they have established with government to good effect. RFPAP has achieved various advocacy results, most recently in terms of ensuring greater attention is paid by government to the SRH needs of young people. R-FPAP are also very focussed on building relationships with civil society stakeholders and have recently convened the MDG 5b Alliance – a group of civil society organisations working on SRH – with the aim of taking a more strategic approach to advocacy. There is also a clear recognition that maternal mortality is linked to a lack of services as well as the low status of women. Thus R-FPAP have become increasingly involved in work on gender-based violence; skills development for women; and ending early/ forced marriage. Targeted efforts have also been made to understand and address the needs of particular groups – including adolescent girls, the transgender community and sex workers. There is also an increasing focus on working with men and boys – both in terms of their role in supporting women to access SRH services, and as clients themselves. R-FPAP is delivering medium Value for Money (VfM), which means that most necessary basic controls exist and are being used to a certain extent. There is however significant room for improvement, related mainly to capacity building, improving and building upon existing systems, introducing new controls and using more efficient accounting software. Improvements are also needed on financial management reporting and costing of services. Approximately 50 per cent of the Couple Years Protection (CYPs) reported by R-FPAP are through its network of private practitioners who are supported through provision of FP commodities and trainings. However, even without taking these account, R-FPAP’s services remain highly cost-effective. They are also achieving a very high cost-benefit ratio. Lessons Learned It is important to balance R-FPAP’s holistic approach with the need for strategic focus and clarity. R-FPAP should articulate and consistently communicate a collective vision which resonates across the organisation and shapes staff’s approach to their work, innovation and learning.
R-FPAP’s increased focus on performance monitoring has resulted in a dependence on routine service data. There is a real lack of attention to assessing needs and monitoring changes at the target population level, and this has led to a neglect of outcomes and impacts. While impact assessments are costly, they are important investments to make, particularly as they generate the evidence needed to support the case for further investment in SRH.
R-FPAP’s frontline workers – LHVs, counsellors, community-based distributors and peer educators – are constantly innovating in order to address the challenges they face in their everyday work. It is important to provide relevant and appropriate forums for these workers to share their experiences and learnings, so that they may inform future programming.
R-FPAP’s network of private practitioners has proven to be highly cost-effective. It is likely that R-FPAP’s own network of facilities, while also highly cost-effective, is comparatively less so because of the additional targeting measures in place to ensure access to the poor and marginalised.
There are significant opportunities for generating greater income from services whilst at the same time increasing access to vulnerable groups. Understanding better what it costs to deliver services, and developing a more structured methodology for pricing services - based on a more analytical understanding of clients’ differential willingness to
Social Development Direct Document Title Pg 4
pay - is a key opportunity for raising income, as well as improving pro-poor targeting measures.
Core funding has proved vital for enabling R-FPAP to develop a multi-dimensional approach to addressing SRH, integrate issues of women’s and youth empowerment across R-FPAP’s work, and establish learning and information sharing systems. There is scope for R-FPAP to use this funding to take its learning activities much further, and to position itself as a more ‘thinking’ organisation on SRH issues.
Recommendations Make greater efforts to understand poverty and exclusion dynamics in target districts using rapid appraisal methodologies.
Develop a more well-rounded strategy for addressing the needs of men, which includes targeting them as a key audience for behaviour change messages about women’s access to SRH services as well as cultivating them as valued clients.
Pilot further community monitoring initiatives which enable users and non-users to provide feedback on services, with a view to integrating citizen monitoring as a regular feature of R-FPAP’s quality assurance processes.
Improve staff capacity, and financial management and accounting systems in order to achieve better Value for Money.
Improve to R-FPAP’s Management Information System (MIS) so that it goes beyond collecting service data, to providing information on programme, project and activity status at both HQ and regional levels.
Improve the involvement of staff at lower levels of the organisation in strategic level processes.
Revisit the remuneration package of service providers, and ensure it is competitive with the local market, in order to attract and retain a skilled health workforce.
Consider streamlining advocacy work and focussing on more systemic messages on the status of family planning in Pakistan, government resource allocation, and its position within the health system.
Strengthen monitoring and evaluation processes, particularly in terms of impact monitoring and collecting high quality qualitative data as part of the M&E system.
Develop a comprehensive sustainability and financing plan which incorporates a more diverse range of funding sources.
Develop a better understanding of what it costs to deliver services, and a more realistic pricing strategy which is based on a more analytical understanding of their clients’ differential willingness to pay.
Strengthen internal knowledge systems so that they enable staff from all parts of the organisation to share information, extract learnings, provide feedback, and be used as a platform to disseminate innovations and learnings.
Social Development Direct Document Title Pg 5
1. Introduction 1.1 Purpose of evaluation IPPF is currently receiving unrestricted funding through DFID’s Programme Partnership Agreement (PPA) and has been implementing activities under the PPA since April 2011. This strategic level agreement provides a grant from 2011 to 2014, up to a maximum value of £8.6 million per year (for three years). This is part of DFID’s significant funding to civil society organisations as part of its overall strategy to alleviate poverty and promote peace, stability and good governance. This report presents the findings of an independent progress review (IPR) of the PPA in Pakistan, where the IPPF Member Association (MA) is Rahnuma-Family Planning Association of Pakistan (R-FPAP). The report has four main sections. This introductory section explains the overall approach to the assignment, presents the context for sexual and reproductive health (SRH) in Pakistan and briefly introduces R-FPAP. The main findings are presented in section 3, which is sub-divided into six sub-sections focusing on key review themes: the relevance of services, efficiency and value for money, effectiveness, results and impact, sustainability of the organisation and its services, and learning and innovation. The final section of the report summarises the main conclusions and lessons learned, and presents recommendations under each of the six themes.
1.2 Methodology The purpose of the IPR is to measure achievements, challenges, outcomes and impacts (both positive and negative) resulting from IPPF’s funding through the DFID PPA. It was agreed that three in-depth country evaluations would be compiled to create an overall evaluation of the PPA funding. Pakistan, Bolivia and Ethiopia were identified as the countries where in-depth evaluations would be conducted. In accordance with the Terms of Reference (ToRs), The Pakistan country evaluation specifically assessed:
The relevance of R-FPAP services; Organisational efficiency and value for money within its operations; R-FPAP effectiveness; Results and impact resulting from R-FPAP interventions and from the DFID PPA; The sustainability of R-FPAP services and other interventions; Innovation, lesson learning and sharing within R-FPAP and with external partners; and R-FPAP’s organisational culture and the extent to which this supports the above.
A set of tools were identified and developed in close collaboration with IPPF Headquarters. A mix of face-to-face interviews, focus-group discussions (FGDs) and desk review of relevant project documents were planned for the country evaluations. In Pakistan, specific sets of stakeholders were identified with whom meetings and discussions were held, including:
Member Association staff – senior staff and staff linked directly to project management; Service providers, including clinic staff and other outreach staff; Government stakeholders; Media representatives;
Social Development Direct Document Title Pg 6
Service users (beneficiaries), including younger and older women, and GBV and Swara survivors; Non-users, men and women; Other stakeholders (Lady Health Visitors (LHVs)); and Staff in the IPPF South Asia Regional Office (SARO).
Questionnaires for SARO, R-FPAP staff, users and non-users, government stakeholders and other key informants were developed by the evaluation team using the DFID evaluation criteria and identified outputs as the basis for discussion. The review team was able to visit three clinics – the Family Health Clinic (FHC) Toru in district Mardan, the FHC Mughal in Islamabad Capital Territory, and Model Health Clinic (MHC) Chakwal city in Chakwal district. None of these clinics were very far from urban centres, and due to time and security constraints, the team was not able to travel to more remote locations. The review team met with a number of service providers, partner organisations, and users. Communication was supported by members of R-FPAP staff. This may or may not have biased some of the comments and perceptions provided.
1.3 Context Pakistan has some of the worst maternal and child health indicators in South Asia, with a maternal mortality rate of 276 per 100,000 live births 3. Health service provision and coverage is uneven, and there are significant disparities in terms of access to healthcare and health indicators, between the poorest and wealthiest quintiles, urban and rural areas, as well as between and within provinces. Two-thirds of pregnant women do not access antenatal care, and only 39% of births are attended by a skilled birth attendant. The 2006-7 Pakistan Demographic Health Survey (PDHS) finds that while the use of family planning has tripled since the 1980s, it has levelled off in recent years. The current contraceptive prevalence rate of 26% per cent is roughly the same as 2003 levels. The use of modern contraception is also fairly low and there is high unmet need at 25%. The difference in fertility rate between the lowest quintile and the highest quintile is 5.8 and 3.0. There are also geographic disparities as contraceptive prevalence ranges from 15 per cent in Balochistan to 32 per cent in Punjab.4 Contraceptive use among men is particularly low. Despite relatively high rates of early marriage and pregnancy, there are few reliable sources of information about SRHR for young girls and boys. 5 The lack of accessible family planning services, cost issues, lack of information about SRH issues and where to get services, misconceptions about the risks of contraception, and poor women’s inability to exercise control over their own bodies and reproductive activity due to patriarchal cultural norms are all key barriers to greater uptake of contraceptive methods. An estimated 197,000 women are treated each year for complications resulting from unsafe abortions. Unplanned pregnancies are the main reason why women seek induced abortions, and many poor women often resort to unsafe abortions, which carry serious risks to their health. A number of studies6 have reported high rates of morbidity amongst women admitted to hospital for complications of induced abortion. Nationally, one in seven pregnancies is terminated by abortion, indicating that induced abortion is a widely used method of preventing unwanted births. It is also estimated that 23 per cent of all Pakistani women who have an abortion are hospitalised for ensuing complications. 7 3
PDHS 2006-2007 Ibid. 5 Shaikh and Rahim (2006) 6 Hussain and Khan (2008) 7 Population Council (2004) 4
Social Development Direct Document Title Pg 7
Pakistan’s health budgets also remain extremely low compared to other countries in the region. The passage of the 18 th Amendment to the Constitution in 2010 has led to the devolution of various ministries – including the health and population welfare ministries – to the provincial level. At the provincial level, two government departments share responsibilities for Pakistan’s public-sector family planning service delivery. The Department of Population Welfare is the main implementing agency of the national family planning programme – however it is poorly resourced and has poor coverage across the country. The Department of Health (DOH) is expected to collaborate with the DOPW to integrate family planning services into its more comprehensive service delivery network. However, because of the fragmentation in planning and decision-making this rarely happens. The latest challenge is the failure of most provincial governments to allocate resources to family planning in the aftermath of devolution.
1.4 R-FPAP background and overall budget context Rahnuma-Family Planning Association of Pakistan (R-FPAP) was established in 1953 as the Family Planning Association of Pakistan. It is widely considered to have pioneered the provision of family planning services and the promotion of small family messages in Pakistan. In the 1980s R-FPAP began to adopt a more holistic, development–oriented approach and changed its name to Rahnuma R-FPAP in 2005 to reflect this. Rahnuma means ‘one who guides’ in Urdu. Working in close collaboration with the government, RFPAP is one of the largest non-government providers of family planning services in Pakistan. Its service delivery network, which began in 1958, now includes nearly 5,000 service points in peri-urban and rural areas of Pakistan, comprising 118 permanent clinics (including 9 hospitals) 11 mobile units, 191 associated clinics and 2,433 community-based distributors/ services. FPAP also supports 2,143 private practitioners located in urban and rural areas to provide family planning services. R-FPAP also implements various youth services, including six tollfree Youth Help Lines and 36 Youth Resource Centres for males and 16 Youth Resource Centres for females.
Box 1. The R-FPAP network 118 clinics 11 mobile service units 191 associated clinics 2422 Community-based distributors 2143 private physicians 6 Youth Helplines 36 male Youth Resource Centres 16 female Youth Resource Centres
R-FPAP manages its programmes through five regional offices (in Provincial and Federal Capitals) and 13 Project Management Offices (PMO), with a head office in Lahore. It is a volunteer-based organisation, and volunteers serve in its governance structures – which include the National Council, National Executive Committee, Regional Councils and Regional Coordination Committees. This structure is supplemented by R-FPAP’s National Youth Network and Advisory Committees. R-FPAP’s activities are structured around the IPPF 5 As Strategic Framework, and all activities cluster around the areas of Access, Adolescents, Abortion, HIV/AIDS and Advocacy. Key approaches include: providing quality integrated reproductive health, family planning, HIV&AIDS, and abortion related services though its network of facilities and associated providers, undertaking awareness raising and sensitisation activities at community and policy levels, and implementing programmes to tackle violence against women, and youth and women’s empowerment.
Social Development Direct Document Title Pg 8
Table 1. R-FPAP 3 –year income analysis 2009 IPPF Core IPPF Restricted Total IPPF Other Donors Local Income TOTAL
$millions 1.4 0.18 1.58 0.73 0.88 3.19
% 44% 6% 46% 23% 28% 100%
2010 $millions 1.54 0.45 1.99 1.58 1.43 5
2011 % 31% 9% 40% 32% 29% 100%
$millions 1.56 1.1 2.63 2.43 1.86 6.92
% 23% 15% 38% 35% 27% 100%
Source: Rahnuma-FPAP Audit Report (year ended December 2011)
R-FPAP’s locally generated income8 has, in absolute terms, risen steadily over the past two years. However donor funding has also increased, and the organisation remains highly dependent on IPPF. IPPF unrestricted funding accounts for 23% of overall funding and DFID funding represents only around 4.5%. However, this is highly valuable for R-FPAP because of its flexibility (for more on this please see the ‘Additionality’ section in Section 2.4 below).
2. Evaluation results This section presents the main findings of the review, structured around six themes: relevance, efficiency and value for money, effectiveness, results and impact, sustainability and learning and innovation. The seventh focal theme of organisational culture is addressed in each of the sections, where relevant.
2.1 Relevance Identifying key focus areas R-FPAP’s programming is clustered around IPPF’s 5 As9 Strategic Framework - with a wide range of activities under each “A”. Programming is based on a five-year Strategic Plan, which reflects some analysis of the social, health systems and policy context around the 5 As.10 The Strategic Plan itself is developed through a process of consultation with FPAP staff and volunteers, as well as a range of government and other stakeholders. There is a particular focus on Access, Advocacy and Adolescents, and specific activities are identified on the basis of national priorities, the socio-cultural context, local needs, as well as R-FPAP’s own capacity to deliver results. Family planning remains a key priority for RFPAP. Their focus on providing SRH services through a clinic- and outreach-focussed approach is also highly relevant to the Pakistan context. R-FPAP have undertaken some formal assessments on various thematic areas, however projectised activities demonstrate a stronger programme design process in terms of documented analysis and needs assessment than the core service delivery programmes. A particular gap seems to be around understanding the needs of very poor and excluded groups and the barriers that they face in accessing SRH services. Client exit interviews are carried out through R-FPAP’s core and project activities. However, the core programme currently includes only up to 60-70 interviews a year. R-FPAP recognise that this is too small a number to provide meaningful information and so these are 8
This includes: cost recovery of contraceptives; fees for medical and laboratory services; rental income from facilities; Income from training services; interest income on investments and bank deposits; and any other income generated from R-FPAP’s own sources 9 These are: Access, Advocacy, Adolescents, AIDS and Abortion 10 Family Planning Association of Pakistan – Rahnuma (2010) Strategic Plan 2010-2014, R-FPAP, Lahore
Social Development Direct Document Title Pg 9
being increased to 500 a year. A recent accreditation exercise also recommended that, in order to generate more meaningful information about client perspectives, exit interviews should be conducted by third parties (such as other clients) rather than R-FPAP’s health staff. Client suggestion boxes are also in place at the facilities but it seems that these are not often used. R-FPAP recognise the importance of community awareness for creating greater demand for their services. In Mardan, for example, the organisation undertook a community sensitisation campaign before setting up their four clinics, with the aim of creating awareness and buy-in, and managing risk. R-FPAP report that in some locations, community men and women are consulted on local level planning through community meetings. However there does not seem to be a standard process through which community stakeholders are involved in the planning of R-FPAP activities at the local level. Yearly lessons learned reports are produced for each project, and these are used to identify any corrective adjustments that need to be made (for more information on these please see section 2.6). R-FPAP recognise that progress on each of the As requires change to take place both at the service delivery level, and in the enabling environment. Advocacy is thus expected to cut across all of R-FPAP’s strategic priorities. Advocacy activities consist of awareness raising among various stakeholder groups, sensitisation of government, building partnerships with a wide range of civil society organisations and working with the media. A key aspect of R-FPAP’s holistic approach is its increasing work on empowering women and adolescent girls. This includes: preventing early/ forced marriage; promoting skills development; and addressing violence against women. This focus stems from the recognition that maternal mortality in Pakistan is linked to lack of services, as well as to the low status of women, their vulnerability to violence, lack of education, mobility constraints, lack of decision-making power within the household and community, and the lack of control over their reproductive choices.
Targeting the needs of the poor and marginalised Poor people’s access to services is influenced by a combination of barriers. These include the costs of reaching and paying for healthcare services, lack of information about SRH issues and where services are available, social and cultural norms around the acceptability of certain services, and a lack of the confidence and self esteem that would make people feel they are entitled to make use of services. As a result, they often require targeted interventions to help them overcome these barriers. R-FPAP’s targeting strategy includes the following key components:
Locating clinics in under-served areas. Keeping fees low and targeting those that cannot pay through differentiated pricing. Reaching those in remote or underserved areas – mobile units. Conducting community awareness and sensitisation activities.
Locating clinics in under-served areas. R-FPAP operates clinics in all provinces and territories of Pakistan, except the Federally Administered Tribal Areas (FATA). R-FPAP’s static service delivery points are located in urban, peri-urban and rural areas with up to 60 per cent of static service delivery points sited in rural areas. However, the poverty profile of R-FPAP’s target districts is mixed. For example, while district Mardan is quite poor, districts Chakwal and Haripur in Punjab province, and Kohat in Khyber Pukhtunkhwa (KPK) are relatively better off. There are also fewer R-FPAP service delivery points in Southern Punjab, or in conflict-affected Balochistan – two of the poorest and most marginalised regions of Pakistan. However, R-FPAP have been working in Muzaffargarh, one of the poorest districts in Southern Punjab over the past two years, where they have taken over the
Social Development Direct Document Title Pg 10
running of six government Basic Health Units (BHUs) with funding from the Pakistan Poverty Alleviation Fund (PPAF). However, this project is due to come to a close at the end of the year. Figure 1. R-FPAP facility illustration map11
R-FPAP’s SRH services are provided through static and mobile service delivery units, through its network of Family Health Hospitals (FHH)12, Family Health Clinics (FHC)13, and Mobile Service Units (MSUs). These service points also support a wide range of private practitioners14 and associated clinics. FHHs are operational at various levels in the cities of Lahore, Karachi, Badin, Quetta, Faisalabad, Islamabad, Chakwal, Turbat, Peshawar, Kohat and Gilgit, and work as first referral health facilities. Ambulance services linked to the Hospitals are also provided as well as a Reproductive Health Extension Programme which provides surgical operations in remote areas.
11
Please note that this map only aims to illustrate R-FPAP’s target districts and does not provide an exact overview of R-FPAP’s service delivery points. 12 FHHs are usually 20-30-bed hospitals located in the major cities of Pakistan. They receive patients referred by the FHCs. They provide curative and preventive services in family planning, reproductive health, safe motherhood, child survival and management of sexually transmitted diseases. They also act as resource and training centers for Family Health Clinics staff and Mobile Service teams. 13 A FHC is a community-based clinic which is usually managed by a Lady Health Visitor, and supported, depending on the clinic, by laboratory technicians, community mobilisers and FPAPs mobile teams. The FHC acts as the first referral base of the service delivery system and further refers clients to Family Health Hospitals. 14 Private practitioners are qualified medical professionals who provide services through their own clinical and surgical facilities. An FPAP identified representative visits them daily to replenish their stocks and to generates a daily progress report.
Social Development Direct Document Title Pg 11
The clinics and mobile service units are located in more peripheral areas. The locations of these are decided in consultation with government, and as a general rule, all R-FPAP clinics are based at least 5 km away from any government facilities. We were told that the Toru clinic was located in an area considered to be the “village of the village”, i.e. even more remote and under-developed than the village itself. The private practitioners are also located in under-served areas, and are provided with contraceptives, IEC materials, trainings and technical support by R-FPAP. A R-FPAP representative visits the private providers daily to replenish commodity stocks, and monitor progress and quality of care. The practitioners are linked to R-FPAP’s FHHs or other nearby local hospitals and clinics so that they can provide referrals for surgical contraceptive services. R-FPAP’s private provider network is widely believed to have increased access to SRH services by the poor communities. Certainly, in many rural areas, these providers may be the only form of healthcare available. It is likely that these providers are more expensive than R-FPAP clinics, and this may represent a barrier to access by the poorest people. Targeting those that cannot pay – differentiated pricing. R-FPAP keep their fees very low. The Toru clinic charges a PKR 10 consultation (‘parchi’) fee, and PKR 40 for medicines for a week. In this way, the clinic undercuts the government facilities by more than 40%. Fees are set by district Project Management Offices (PMOs) from a range that is determined at HQ level. Currently, the range for parchi fees is Rs. 5 to Rs. 50. It is not clear how the final figure is decided upon and how this is linked to an understanding of the poverty profile of the client base. Facilities also operate a ‘no refusal policy’ whereby any patient who cannot pay is exempt. The decision to waive fees is at the discretion of the facility in-charge who assesses clients’ eligibility either for a full waiver or some level of discount, from her/ his appearance (clothing, ornaments, jewellery, etc.) However, there is no transparent or methodical way of assessing the poverty profile of clients or determining whose care should be subsidised and by how much. In addition, the information is recorded in paper form (primarily for audit purposes), and while care is taken to validate the claims, this information is not centrally recorded or analysed. The use of such an unstructured methodology makes it likely that poor people are subsidising wealthier clients. In addition, at the Toru clinic, neither the pricing structure nor the no-refusal policy were clearly displayed. Although R-FPAP’s fees are very low, it is important to consider what impact charging for services may have on the ability of the poorest women to access their services. It is often argued that people prefer to pay for services as they associate this with better quality. However, very little research has been undertaken in Pakistan on the links between fees and demand. Reaching those in remote or underserved areas – mobile units. Some of R-FPAP’s static service delivery points are used as a base for regular community outreach activity to more remote areas. R-FPAP operates 11 Mobile Service Units (MSUs) through its core funds. One of these is based in Chakwal, and covers 56 locations in the surrounding area. It is staffed by two Lady Health Visitors (LHVs) and one driver. The MSU visits two locations each day, and drops one LHV off in each of these to conduct a clinic session. Communities are informed about the dates of the visit in advance by local LHWs. The LHV provides RH and FP services (including oral pills, condoms, injectables and IUCDs) along with counselling services. For tubal ligation and implants, clients are referred to the model health clinic in Chakwal city. Important lab tests (which can be provided using different kits) are also provided as part of the service. Unfortunately the evaluation team was unable to see an outreach visit in progress, and were only able to see the vehicle. We noted that this was a Suzuki Carry Van, which is a very
Social Development Direct Document Title Pg 12
basic type of vehicle not well suited for travelling in difficult terrain. FPAP have recently replaced a number of their older vehicles. However, the vehicle seen by the evaluation team was 12 years old and we were told it breaks down frequently. Conducting community awareness and sensitisation. R-FPAP does not undertake active marketing of its services as they feel that developing a high profile national brand would affect access for poor women living in remote and conservative communities. Certainly using advertising and promotion to generate demand for services can have a direct impact on income generation. Branding can also help to position and link services to hard to reach groups, such as youth. It is also a useful way of accrediting affiliated providers. However, in a context such as Mardan, where women’s mobility is constrained and conservative social norms prevail, openly advertising clinic services might put clients at risk, or drive them away. Marie Stopes International (MSI) does brand their clinics, and it would be interesting to learn from their experience. Instead, R-FPAP undertake awareness raising activities using social mobilisers, and local service providers such as LHVs and LHWs, dais (traditional birth attendants), and the public health system for referrals. They also run regular medical camps in the communities, which include awareness sessions and distribution of free medicines. R-FPAP also uses Community-Based Distributors (CBDs) who go door-to-door and provide contraceptives. Although the Toru clinic has a good client turnover, we noticed that there were some challenges with community awareness levels there. The evaluation team partnered with a prominent health organisation in Mardan to undertake the focus groups discussions. They did not know about the Toru clinic. They indicated that the local RHC also did not know about the clinic. Unfortunately we were unable to meet the social mobiliser attached to the clinic as he was on leave. R-FPAP has found that 69% of those interviewed during the exit interviews were referred to the clinic through an existing user. This indicates that word-of-mouth is an important way of people finding out about R-FPAP’s services. However given the very small size of the exit interview sample, this can only be seen as a very loose indication. Of R-FPAP’s four clinics in Mardan, two have female mobilisers, and two have male mobilisers. The Toru clinic also benefits from the work of the counsellors and LHVs attached to the Women as Compensation (WAC) project and the new donor-funded Choices project. We asked the PMO if they had a sense of whether having a male or female mobiliser affects the utilisation rate of a particular clinic, and if so, in what way. While clinic registers record where new clients have been referred from, it seems that the user statistics are not analysed to provide this sort of information. In many of the contexts in which R-FPAP works SRH and FP are sensitive topics. R-FPAP understands the issues around talking about FP and has evolved a sophisticated approach of using various entry points for its messages on SRH. These include child health, primary health, economic empowerment, youth participation and community ownership. As a result, R-FPAP provides a range of primary health services as an entry point for providing SRH services. It has also evolved an approach of using health to address other issues. In Mardan, an important project is using health as an entry point to address gender-based violence (GBV) and Swara practices against women and girls (see box 3). A key starting point for R-FPAP’s work on changing knowledge, attitudes and behaviours is their own staff. All service providers participate in value clarification exercises which, according to the Trainer’s Manual provided to us, aim to create awareness and “move participants towards support, acceptance and advocacy” for various issues, including SRH, abortion, HIV, STIs, working with adolescents, etc. They are also based on a rights-based approach, and provide guidance on the standards of care that all clients should expect to
Social Development Direct Document Title Pg 13
receive. For example, staff are taught that HIV-positive clients have a right to choose whether to disclose their HIV status or to undergo testing, as well as to receive services without discrimination; and that young girls should be guaranteed confidentiality and privacy, and staff should not ask them for unnecessary levels of information and detail. These exercises have proven very effective in some cases. For example, when R-FPAP introduced manual vacuum aspiration (MVA) to its services, all R-FPAP doctors in KPK province refused. After undertaking value clarification activities, the greatest demand for MVA now comes from Peshawar. R-FPAP also recognise that in order to change social norms and behaviours around SRH issues, it is critical to involve the community, including mothers-in-law, community elders and teachers. A range of behaviour change activities are undertaken at the community level including the dissemination of IEC materials, focus group discussions with key groups, and street theatre. Working with men and boys has become a key part of R-FPAP’s strategy, based on the recognition that increasing awareness amongst men would make them more likely to support their wives’ use of family planning methods. Additionally, men have SRH needs themselves. As mentioned already, of the four clinics in Mardan, two have female mobilisers and two have male mobilisers. A particular challenge for greater uptake of contraceptive methods is the perception in Pakistan that family planning is non-Islamic. R-FPAP has therefore consciously targeted religious scholars for their sensitisation and behaviour change work, often seeking fatwas from Pakistani and Egyptian15 scholars. R-FPAP have also developed awareness-raising and sensitisation strategies that are tailored to specific contexts (see Box 2).
Identifying the poorest and most marginalised groups R-FPAP’s key assumption is that locating a clinic in under-served areas is critical to providing access to poor and marginalised people. By and large, this assumption holds true, as large numbers of people otherwise excluded from accessing services due to distance and mobility barriers are being provided access by R-FPAP’s services. However this approach does not consider how differing levels of poverty (and indeed different identities) can affect the kinds of barriers and exclusionary processes that people are subject to, and how this in turn affects the extent to which they can benefit from R-FPAP’s services. As mentioned above, while for some projectised activities, detailed needs assessment and formative research activities have been undertaken, the evaluation team were unable to find any evidence of baseline studies that are completed before the establishment of a clinic. The feasibility studies that are undertaken are basic documents which provide limited information. Unlike many of the others, the Toru clinic feasibility study does include some limited information on the socio-economic composition of the area, but it is not clear what the methodology of the study was, and where the information has been obtained from. No information is given on the poverty profile of the area or its social composition. The project justification includes an overview of the national health situation, and not a district-specific one. We were told that these studies use data from the Pakistan Bureau of Statistics (PBS), and the Pakistan Demographic and Health Survey (PDHS) – however neither of these are referenced in the documents. The evaluation team accepts that these kinds of studies can be expensive and that any organisation needs to balance its investment decisions. However, the current feasibility studies could be greatly improved, and consideration should be given to developing a better needs assessment process. R-FPAP also shared the results of a pre-testing exercise of the ‘Entitlement Index’ which seems to have been developed by the South Asia Regional Office (SARO) in order to 15
Egypt is often considered to be the home of Islamic learning
Social Development Direct Document Title Pg 14
facilitate a poverty- and gender-focussed situation analysis of the catchment area of particular facilities. The document is not dated, and the exercise was not mentioned in the interviews, and so it is not clear what stage this initiative is at. Very encouragingly, we were also sent a ‘Social Audit report’ that was carried out in Chakwal. This included focus group discussions and interviews with women, men and adolescent girls and boys about their SRH needs. It also included a village based mapping of health facilities and service providers, a facility survey, provider interviews and client exit interviews. This is a useful document which provides a breakdown of the health issues facing the different groups surveyed, an overview of health seeking behaviour in terms of type of service provider, and an assessment of the extent to which the health facilities meet the community’s needs. Again, the document is not dated and was not mentioned in the interviews, and it is not clear how the findings have been used to inform R-FPAP’s programming. Much attention has been given to the role of men and boys in supporting women’s access to SRH services. However, there has been less thinking about how to encourage men to use the services also. Many clinics have male staff, and men are encouraged to attend. However, in the clinics we visited, the clientele was exclusively female. In fact this was identified as one of the key strengths of the clinics – that there were few men present and women felt comfortable. R-FPAP do conduct mobile camps at youth resource centers, schools, BHUs and madrassahs where men and boys can access services. However consideration should be given to how the service delivery model can be further adapted to address men’s needs.
Addressing the needs of vulnerable groups R-FPAP have identified certain groups as being particularly vulnerable to poor SRH and lack of access to quality services. These include women and girls experiencing gender-based violence, adolescents, sex workers, and the transgender community. One staff member interviewed mentioned the difficulties that religious and caste minorities face in attending clinics in Badin district in Sindh province. It does not, however seem that a social exclusion analyses is integrated across the programme, in terms of identifying and understanding the barriers for the very poor and ethnic, religious and caste minorities. Working with youth. Over the past two years, R-FPAP have sharply increased their adolescent-focussed services as part of their key long-term strategic direction. Their integrated approach to addressing youth issues includes: enhancing young people’s knowledge on sexuality and access to services; providing youth friendly services (including youth resource centres and helplines) and outreach programmes; and undertaking advocacy on comprehensive sexuality education and ending child marriage. A key achievement over the past two years is the integration of adolescent and youth issues into the mainstream RFPAP programme. R-FPAP also implements an impressive peer education programme which builds the capacities of young people on SRHR, and supports them to conduct awareness raising activities with other young people in their communities. During their training, young volunteers (aged 16-24) are taught how to use various tools, such as theatre performances, role play, IEC materials and videos to broach discussions with their peers about SRHR. It is highly encouraging that R-FPAP are able to support such work in the increasingly conservative and insecure context of Pakistan. These peer educators are even working in the highly insecure environment of Balochistan. Survivors of gender-based violence. R-FPAP perceives a strong link between genderbased violence (GBV) and SRH, and understands that reproductive health service providers have an advantage when it comes to identifying signs of violence. They have thus started to integrate work on this within their service delivery system, and all staff are trained in
Social Development Direct Document Title Pg 15
screening for GBV.16 Protocols for LHVs and doctors to conduct medical examinations for such women, as well as for counsellors to provide advice have also been developed. Staff are also trained to ensure that, in the event that a women decides to avail any support services or take any other action, this is an informed decision. R-FPAP providers can refer women to legal support providers, shelters, the police and legal aid partners. The WAC project in Mardan has developed strong referral links with partners (see box 2 below). Box 2. Women As Compensation (WAC) Programme Swara is the practice - particularly prevalent in Mardan and Swabi districts - of giving young girls in compensation for a crime through forced marriage. Most of these girls become vulnerable to sexual abuse and exploitation, early pregnancy, high risk of maternal mortality and morbidity, and transmission of STIs and HIV. Often they are unable to visit a doctor, clinic, or nurse without permission. Funded through the IPPF Innovation Fund, the initially three-year WAC project (which has been extended by a further year and will culminate at the end of the year) aims to:
provide support and services to swara girls and women, enable them to know and claim their rights, and bring about behaviour change within the community through orientation and sensitisation activities.
The project uses free medical camps and the provision of First Aid Training as entry points for talking about issues of SRH, GBV and Swara, but also as a means of accessing GBV or Swara survivors in the community. Male and female community theatre groups are also used to raise awareness. The project has established referral links with legal aid providers, shelters, other CSOs, etc. R-FPAP’s interaction with scholars, imams and community leaders, and their approach of cultivating them as advocates against the practice of Swara has proven one of the most effective aspects of the project. They identified religious scholars based in Mardan to provide commentary on these issues. One of these was Dr Farooq, a renowned scholar who supported R-FPAP’s work. In 2010, he criticised the Pakistani Taliban on television and was killed at his home shortly afterwards. This is an indication of the security threat that the project works under. The local jirga was also identified as an influential body, that could potentially champion their work. A situation analysis found that jirga members frequently participated in mushairas (poetry recitals). R-FPAP organised a mushaira for the jirga members where Pashto poems against GBV and forced marriage were read out. This opened up a discussion about the issues. Advocacy efforts with the religious leaders have clearly brought about a tangible change; local imams/clerics are now sensitised and informed to guide people on matters relating to reproductive health and family planning. Project staff also use Islamic injunctions in the interactions with community members, teachers, mothers-in-law, and survivors themselves and this has proven to be particularly effective. To date, the project has provided 54,790 SRH services to the target community. SRH and counselling srvices were provided to 656 Swara women, 1,001 GBV survivors and 870 child marriage survivors. One of the key impacts of the project is that it has managed to access extremely marginalised women. Many of the girls we spoke to had reported incidences of extreme violence and abuse. The project has supported them to return to their parental homes, and access health care and legal aid. They have also been provided First Aid training which many of them are teaching others in the community. Given the highly conservative and often insecure context of Mardan, it is remarkable that the project has achieved these changes. There are various factors which have contributed to the project’s success. These include its strategy of involving religious leaders; the use of Islamic knowledge; the 16
R-FPAP had to begin with sensitising staff around the issue of GBV, and supporting them to recognise that it is a problem. In the many contexts in Pakistan GBV is normalised, and can even have a positive connotation, i.e. women believing that their partner’s violence is a sign of love. Their experience has found that often when women become pregnant with, or give birth to, a female child, GBV can increase.
Social Development Direct Document Title Pg 16
innovative sensitisation tools; the dynamism and commitment of the staff, and their local knowledge. However there are concerns about sustainability. The original project is only able to provide a limited amount of practical support to the women that they identify. As mentioned above, the WAC project has originally planned as a three-year project, which was extended by a further year. It is due to come to an end at the end of 2012. However, given the high level of community ownership of the project and the momentum that has been generated, the FPAP Board has realised the strong need for continuing project activities. As a result, a PMO was set up in Mardan district in January 2012. Most project activities – such as meetings with influential community members, working with mothers in law, and youth theatre - are now part of its core programme. This is a strong reflection of the value of the project and its sustainability within the FPAP system. FPAP has also secured funding for the Choices Programme from the Choices Fund, a Norwegian donor, for capacity building of youth on SRHR, and provision of services to youth in a youth friendly manner.
The transgender community. R-FPAP provides technical support to Saathi Foundation - a Lahore-based NGO working to support the transgender community - to run joint clinical services for the transgender population of Lahore, provide information about HIV/AIDS and safe sexual practices, and support anyone who tests positive for HIV. R-FPAP has also been supporting Saathi in advocacy efforts to have transgenders recognised as a separate gender by the government. Capacity building support in management and resource mobilisation is also provided. R-FPAP’s partnership with Saathi is an excellent demonstration of how a marginalised group can be organised to voice their concerns, and engage in positive health seeking behaviour.
2.2 Efficiency and Value for Money Organisational efficiency R-FPAP are a well-respected, well-resourced organisation, with a wealth of experience and skills in terms of managing large programmes, working on sensitive issues, collaborating with government, and delivering quality healthcare services. Their association with IPPF lends them additional credibility and leverage, as well as access to regional and international expertise. We also noted high levels of commitment and dedication at all levels of the organisation. R-FPAP’s strategic planning, activities and even organisational structure are completely aligned with IPPF’s 5 As Strategic Framework. This reflects the resonance of the Framework with R-FPAP’s holistic approach, and certainly R-FPAP’s understanding of the 5 As is carefully aligned with the Pakistan context. However, this commitment to each of the 5 As has resulted in a wide range of activities being pursued. This approach seems to have undermined R-FPAP’s ability to develop, or at least explicitly articulate, a more consolidated focus that is targeted at the most critical issues facing SRH in Pakistan today. Few staff were able to talk to us about outcomes and impact, indicating gaps between high-level strategic planning, and the understanding of staff – at both HQ and field level - of R-FPAP’s broader strategic and project objectives. This may be, in part, because R-FPAP’s strategic priorities are too numerous and too fragmented to enable the development and communication of a strategic vision that the whole organisation consciously and consistently subscribes to. As a large organisation which manages a very wide range of activities and donors, R-FPAP has made considerable investment in putting in place various management, reporting, monitoring and communications systems, and certainly the information flow of data around the organisation works well. There has been particular emphasis on these over the past two years, and perhaps as a consequence, we found that staff were very focussed on activities,
Social Development Direct Document Title Pg 17
process and outputs in terms of increasing access, and improving quality of care. It is our impression that more work needs to be done to help R-FPAP understand how these systems can work for them and to fine-tune them accordingly. Staff’s ability to maintain a strategic overview, for example, could be improved by making the monitoring, evaluation and information systems provide more user-friendly information about results and impact to all levels of the organisation. R-FPAP’s monitoring is overseen by two teams at HQ level. Regional offices collect routine service statistics from the service delivery outlets, which are then consolidated by the Monitoring, Evaluation and Research (MER) team at HQ level. The Regional Offices and PMOs are trained to analyse this information, and feed it back to the facilities. The Programme Management Unit (PMU) at HQ level oversees programme design, inception, implementation and progress. However the coordination between the MER team and PMU could be improved. The two units produce separate monitoring reports for example – one on service utilisation and project progress respectively. R-FPAP has a well-developed quality assurance system which includes a range of mechanisms, including regular on-site monitoring visits by a Quality Assurance Doctor (QAD), medical audits, client exit interviews, focus group discussions with key community stakeholders, and clinic suggestion boxes. These are used to develop action plans for each service delivery point, which are followed up through subsequent monitoring visits. The mechanisms for collecting feedback from clients are currently weak as only a limited number of client exit interviews are conducted. However these are being increased to 500 a year. Client suggestion boxes also exist and while the clients we spoke to were aware of these, it does not seem that they were used frequently. An important human resource management issue is staff retention, particularly at service provider level. Essentially a local NGO, R-FPAP clearly cannot compete with the salaries offered by international NGOs and donors. However, R-FPAP staff salaries are below both NGO and government levels. A R-FPAP LHV is paid PKR 10,000, compared to a government LHV who can earn as much as PKR 20,000, and a UNFPA LHV who is paid PKR 40,000. R-FPAP’s female doctors are paid a salary of PKR 30,000, while government doctors on average earn PKR 60,000. This has a significant impact on staff turnover. In the past two years, three Quality Assurance Doctors have left the Mardan PMO. The Woman Medical Officer (WMO) posts were vacant in both the Toru and Mughal clinics. A recent LHV vacancy in the Toru clinic was managed by promoting the midwife (who had recently completed her LHV training). In the absence of the WMO, she is in charge of the clinic. These examples make it clear that attracting a skilled workforce, and particularly women, to work in rural areas requires an attractive package of remuneration. The review team recognises that the imperatives of retaining staff must be balanced against those of providing good value for money, and appearing competitive to donors. Indeed, one of the reasons that R-FPAP provides such good value for money is because salaries – a key cost driver – are so low. R-FPAP should balance their salary costs in order to make sure that staff turnover does not negatively impact effectiveness and efficiency. An analysis could be undertaken of the costs of hiring replacements (which can involve initial loss of productivity due to loss of institutional memory, lower skill and knowledge levels of new staff, and other recruitment related costs), against what it would cost to retain more staff, most likely by increasing salaries. It may be that R-FPAP opt to accept recruitment costs for certain categories of staff, and retention costs for others. For staff whose retention is prioritised, some element of performance may have to built in to their contracts. Either way, baseline salaries for every category of staff should be reasonable and competitive in the local market, and should make employees feel rewarded for their work.
Social Development Direct Document Title Pg 18
These should be based on an analysis of what R-FPAP’s competitors are offering, both in terms of salaries and the results achieved. R-FPAP’s unit costs, cost per CYP and cost per DALY are extremely low, so there does seem to be some room for increasing salaries. Value for money is about getting the best quality and most impact for the money, and not just about keeping the costs as low as possible. The DALY methodology only calculates health impact and does not take into account the numerous other impacts of R-FPAP’s work that would be impossible without the committed and able staff that work in the organisation. Staff at HQ level are also over-stretched. This is mainly because R-FPAP’s portfolio of projectised activities are mostly funded through restricted grants. Few of these projects include funding for personnel costs, and so an increasing number of projects have to be managed by the core staff. The advocacy portfolio, for example, seems to be managed by only one person who is responsible for implementing activities, maintaining relationships with government and liaising with partners. She also oversees resource mobilisation activities. Staff and volunteers are supposed to be provided with training on the technical and managerial aspects of the programme. However, at the Toru clinic, the recently promoted LHV who oversees a staff of seven people, had not yet received any managerial training. Delays in training are, in some part, due to resource constraints, and the practical difficulties of training large numbers of staff. Finance staff at HQ level are also not provided with specialist trainings. It is also not clear to what extent clinic staff are able to share feedback or complaints with either their supervisors, the Quality Assurance Doctor, or PMO staff. Staff are involved in self-assessment exercises through which they appraise their own services, identify gaps, and develop an action plan with a view to improving service provision to clients. However none of the frontline staff we spoke with mentioned this process as a channel through which they could make their voices heard. At the Toru clinic, daily meetings are held, but this seems to be more with a view to reconciling service statistics than sharing concerns or ideas. At the governance level, regional representation in R-FPAP’s governing bodies is ensured through the representation of local volunteers at national levels. R-FPAP also ensures that at least 50 per cent of its governing Councils are female, and 20 per cent are constituted by young people. Ten per cent of R-FPAP‘s staff are also currently under the age of 25, and 50 per cent of staff are women. In addition, a national youth network has been established to incorporate youth perspectives into the programmes. Being part of the IPPF federation affords R-FPAP access to international knowledge and expertise, and access to technical assistance and support. R-FPAP base most of their work on materials and guidance generated by IPPF. A prominent example of this is the development of their Quality of Care approach. Support from the Regional Office, and particularly the Central Office, is valued. Inputs from Central Office are particularly valued as they provide a wider perspective. However, R-FPAP have relatively little interaction with the Central Office.
Financial efficiency The concept of VfM is differently understood throughout the organisation and is mostly confined to delivering services at the lowest cost. More understanding needs to be built on linking inputs with results (linking financial inputs with non-financial outputs). As part of this evaluation, a financial management assessment was undertaken which focussed on nine core functions: i) organisational structure and human resource; ii) accounting system and reporting; iii) budgeting and planning systems; iv) purchases and payables; v) invoicing and receivables; vi) treasury functions; vii) payroll; viii) asset management system; and ix) audit. The assessment found that R-FPAP is delivering Social Development Direct Document Title Pg 19
medium VfM. This means that most of the necessary basic controls exist and are being used to a certain extent. There is however significant room for improvement, related mainly to capacity building, improving and building upon existing systems, introducing new controls and using more efficient accounting software. Improvements are also needed on financial management reporting and costing of services. R-FPAP also do not have the in-house mechanisms and expertise needed to enable them to adequately cost their services, and measure cost effectiveness. Currently, they are using a very rudimentary method for allocating costs per CYP – by dividing all costs reported under ‘Access’ by CYPs. In addition, there seems to be little understanding of the extent to which clinic capacity is used. We believe that most of the blocks identified as providing medium VfM can be upgraded by developing a financial management review and reform programme addressing the weaknesses identified in this evaluation. In particular, immediate management attention should be directed towards R-FPAP’s treasury functions - the condition of controls on the treasury function related to cash and banking operations - which we assess as currently providing low VfM. It appears that IPPF has a great deal of influence over the way in which R-FPAP’s systems and processes are structured, and improved guidelines and guidance from IPPF will further help in improving efficiency especially at FM level. The detailed assessment is provided in Annex 3. A snapshot of the assessment is provided in the table below: Table 2: Financial Management Assessment
SCORING GRID Low VfM Blocks 1: Organisation structure and human resource 2: Accounting and reporting 3: Budgeting and planning systems 4: Purchasing and payables 5: Invoicing and receivables 6: Treasury functions (cash and bank) 7: Payroll 8: Asset management system 9: Audit Total
less than 50%
Medium VfM High VfM between between between more than 61% 51% - 60% 71% - 80% 80% 70% x x x x x
x x x 1
2
3
2
x 1
2.3 Effectiveness R-FPAP are widely recognised as having pioneered the family planning movement in Pakistan, and having one of the most well-established non-governmental service delivery programmes.
Strengthening access and quality of care Of the 5 As, R-FPAP has identified Access, with a particular emphasis on quality of care, as its niche. Located in under-served areas, their clinics are providing access to people who would otherwise only be able to access them at much greater cost or not at all. R-FPAP’s
Social Development Direct Document Title Pg 20
partnership with a range of 191 associated clinics and 2,143 private health care providers has also increased access to FP and other SRH services. Over the past two years, RFPAP’s network has expanded from 82 to 118 clinics. Since 2010, R-FPAP have achieved 17% more usage, and a 5% increase in Couple Years Protection (CYP). In 2011 alone, RFPAP established relationships with just under 200 private practitioners delivering nearly 100,000 CYPs per year.17 R-FPAP believes that the quality of care – provided through trained non-judgemental staff, and a rights-based and client-centred approach - available at their clinics is the programme’s key achievement.18 They are well-regarded for their technical expertise in health, and have invested considerable resources in strengthening their service delivery programme, and in developing systems to monitor and assure quality. Indeed, we received very positive feedback about R-FPAP’s services from users. Most women identified the proximity of the clinic, the cost of medicines, and the provision of advice and information to be key factors. Other factors include: staff interpersonal and communication skills, short waiting times, a safe environment for women, and the availability of a range of services. Box 3. R-FPAP’s strengths as perceived by external stakeholders Technical expertise in health service delivery ‘Pioneer’ status Connections with government Strong leadership and management Committed staff Good reputation Well-resourced Strong management
The clinics we visited were in good shape, with waiting areas for women. They had the necessary equipment, which was found to be in good order and clean. Neither the Toru or Mughal clinics were providing deliveries, and did not offer abortion services, and so instrument sterilisation was not an issue. Infection prevention techniques were being used in both clinics. We were told that R-FPAP have a waste disposal systems in place at all clinics, and that all staff are trained on the waste disposal policy. However, staff at neither the Toru nor Mughal FHCs were able to explain this process to the review team. The clinics had adequate stocks of supplies, and were very well stocked with essential medicines, including antibiotics. Both clinics have record systems in place to monitor stocks. In one instance we found that the clinic was receiving incorrect supplies, and having to refer patients to other laboratories for certain tests.
Both the Toru and Mughal clinics had rapid testing kits for Hepatitis B and C, but not for HIV. We were told by HQ that clients testing positive for Hepatitis B or C are referred to the nearest FHH or laboratory for confirmation, and that pre- and post-counselling is provided. However, staff at the Toru and Mughal clinics were not able to tell us about these follow up systems. We also did not notice any health education material on HIV or hepatitis displayed at either clinic. The review team are also concerned that storing the Hepatitis testing kits in hot temperatures might yield false results. High temperatures can also greatly affect the functioning of diagnostic equipment, and consequently the results. The ventilation system in the Mughal laboratory was poor - it did not have a fan or any other type of cooling system. Those clinics without electricity generators also have to contend with the frequent power outages affecting the country. The Toru clinic was particularly severely affected during our visit. The evaluation team noted an additional management issue relating to health and safety. During the visit to one of the clinics, a small fire broke out as a result of an electrical fault. There were no fire extinguishers or other safeguards in place to deal with this. 17
It is argued that a principal reason for the fall in contraceptive use and rise in fertility in Pakistan is the strategic shift that has taken place over the last decade, to measuring performance on CYP rather than CPR. As a result service providers have emphasized contraceptive surgery, to the detriment of hormonal and barrier methods. This issue must be addressed now as the large adolescent cohort identified in the 1998 census has entered the reproductive age. 18 This is addressing an important gap, as quality of care provided in the public sector is widely considered to be a key deficiency, with many poor people turning to unregulated, often untrained private providers.
Social Development Direct Document Title Pg 21
Services are supposed to be provided according to IPPF standards, and R-FPAP have produced various manuals, guidelines and self-assessment tools, on issues such as family planning, gender-based violence and MVA, which are made available to all field staff. Standards are based on clients’ rights and providers’ needs, and are expected to be followed in all R-FPAP clinics and hospitals. Staff at some, but not all, of the clinics we visited were able to talk about these standards. One of the feasibility studies shared with us mentions the involvement of Community/ Village Health Committees in monitoring service provision by the clinics. These were not mentioned in the interviews and don’t seem to appear in any other documentation. Community monitoring initiatives can play an important role in enabling citizens to voice their needs, and to hold service providers accountable. They have to be carefully designed to ensure that they do not become dominated by local elites, and poor and excluded people are able to participate equally. They can also be linked to social accountability mechanisms which enable citizens to engage directly with service providers to give feedback and influence decision-making. The Chakwal Social Audit is one example of this. If R-FPAP is truly committed to a Quality of Care approach which is centred around the needs of poor and vulnerable groups, then they may consider piloting some further social accountability initiatives.
Awareness raising and sensitisation R-FPAP are considered to have made a significant contribution to increasing awareness about family planning. They are particularly skilled in dealing with sensitive issues, as discussed above. This is achieved through a sophisticated strategy which includes using healthcare as an entry point, training frontline workers to use a range of tools to introduce sensitive subjects, building family and community support and encouraging direct interaction between project staff and families, obtaining fatwas (opinions) from religious scholars about the acceptability of their messages, recruiting people to work in their own communities, and addressing staff’s own values and attitudes first. The evaluation team was greatly impressed by the dynamism and commitment of the clinic counsellors and LHVs, the youth peer educators and the theatre group that we met. The impact of this awareness raising work is not measured by R-FPAP, and so it is difficult to know which types of initiatives are the most successful (please see section 2.4 for more on monitoring and evaluation).
Building partnerships R-FPAP enjoy a high level of credibility and a strong working relationship with government. At the federal level, they are invited to all meetings in the Planning Division, and provide technical input to the Departments of Health and Population Welfare at the provincial level. In terms of service delivery, R-FPAP works in consultation with the district government to site their clinics, and to develop referral links. In the province of Azad Jammu and Kashmir, they have had a long-standing arrangement with the government to provide services and free contraceptives in 148 government clinics. Few facilities run by the Department of Health provide family planning services, and this is the first model where both are provided under the same roof. This partnership model is bring replicated in South Punjab – one of the poorest regions in Pakistan. R-FPAP have also established referral links with governmentrun health facilities in the areas where they work. R-FPAP are valued by other civil society organisations for their depth of experience, geographical spread, technical knowledge, readiness to take up new initiatives, and their understanding of how to deal with sensitive issues. R-FPAP were also responsible for convening the MDG 5b Alliance – a group of civil society organisations working on SRH –
Social Development Direct Document Title Pg 22
with the aim of taking a more strategic approach to advocacy (see the ‘Advocacy’ subsection for more on this). R-FPAP is also a member of a number of alliances such as the Pakistan Alliance for Post-Abortion Care (PAPAC), the Media Network and the Punjab AIDS Alliance. R-FPAP also partners with Khushali Bank in providing micro-finance services to women.
Advocacy R-FPAP is well regarded by government, and provides technical inputs to various policymaking bodies. Recently, they were instrumental in the passing of a 2012 Resolution in the Punjab Assembly which directs government departments to examine the issues around Reproductive Health for young people, as well as sexuality education. R-FPAP have also provided recommendations on including provision for youth in the Punjab Health Sector Strategy. They are increasingly involved in international advocacy, and have participated in various government delegations to international events over the last two years, including the 2012 Family Planning Summit in London. R-FPAP has been engaging with Parliamentarians on the issue of budget allocations to FP, in order to enhance political commitment for FP and SRH. In 2011, as part of a regional advocacy project (initiated by SARO), and supported by GPAF, selected Pakistani parliamentarians, along with MPs from other South Asian countries, attended a capacity building workshop on SRHR. Whilst at the conference, one particular woman parliamentarian pledged to ‘do something ‘ within the year. R-FPAP report that, since then she has been reponisble for the tabling of 17 questions on budget allocation to FP in Parliament. RFPAP also recently organised stakeholder consultations for the Family Planning Summit in June 2012 which was attended by a large number of parliamentarians. The idea behind the consultation was to sensitise stakeholders but also to arm parliamentarians with information to raise MISP in the Disaster questions of provincial budget allocations to family Management Protocol planning. The events were very well-received, and achieved high numbers of attendance by parliamentarians. Box 4. Advocacy Activities Advocacy for SRHR (including FP) Advocacy for greater resources for SRHR (including FP) Advocacy for MDG 5b Media networks NGO Coalitions Islam & Family Planning Comprehensive Sexuality Education Upscaling of best practices Programme issues relating to the 18th Amendment
R-FPAP was also responsible for coordinating the MDG 5b Alliance, which is an umbrella alliance made up of high profile local and international organisations including civil society organisations, donors, government representatives, and some local community based organisations. The formation of the Alliance was based on the recognised need to move away from working in pockets, and putting resources and efforts towards a collective movement. The establishment of the MDG 5b Alliance is considered by R-FPAP to have changed the way advocacy is done in Pakistan. They are able to work more strategically, and because of their collective profile, exercise more increased influence than if they were acting individually. One interviewee suggested: “It’s a collective movement. Everyone has ownership. Earlier, people were territorial”.19 The Alliance is able to pool financial and other resources for particular events, or to support each other’s activities. For example, for RFPAP’s advocacy work on including the Minimum Initial Services Package MISP in the Punjab Government’s Provincial Disaster Management Authority PDMA, other Alliance members supported them to get appointments with their contacts. Provincial chapters of the Alliance have also been established in Peshawar, Quetta, Lahore and Islamabad. Advocacy activities in Sindh will be undertaken through the Pakistan Alliance for Postabortion Care PAPAC. 19
HQ interview
Social Development Direct Document Title Pg 23
R-FPAP believe that the strength of their advocacy lies in backing up their messages with evidence, developing tailored materials and breaking down the jargon. However, they face various challenges to undertaking advocacy in the Pakistan context – which include corruption, insecurity and ad hoc policy-making and planning. Policy makers’ competing priorities, lack of time, and the frequent turnover of decision- and policy-makers is also a major challenge. Often R-FPAP will invest time and resources in cultivating relationships with particular post-holders, only to have them move on. The increasing political influence of ultra-conservative religious leaders means that R-FPAP has to be very careful about raising the profile of the organisation, communicating their work, and selecting advocacy topics. For example, they do limited advocacy on abortion for fear that if the issue were spotlighted, this might make the space even more restrictive. Another challenge is that SRH issues are not attractive for politicians as investment in reproductive health does not show visible gains for several years. R-FPAP’s advocacy portfolio includes a large range of activities. In part this is due to RFPAP’s commitment to a holistic approach, whereby change at the service delivery level should be accompanied with changes in the enabling environment. Thus every ‘A’ has a corresponding service delivery and advocacy objective. One consequence of this is that advocacy messages are fragmented and thus the effect is possibly diluted. One of R-FPAP’s great achievements is the establishment of a collaborative working relationship with government. Much of this work involves creating awareness about a wide range of issues, and in many cases R-FPAP have been able to leverage the credibility and trust that they have established with government to good effect. However, what is needed in Pakistan is concerted action on systemic messages about the status of family planning and the government’s resource allocations. Such a campaign could mobilise other civil society actors, and be undertaken in multiple spaces, such as the research, media, parliamentary lobbying, community mobilisation, etc. There is no organisation better placed to undertake such an initiative. It would be interesting for R-FPAP to explore working in both collaborative and more confrontational ways with government. They already recognise the need for monitoring government implementation of commitments made at international and national levels.
2.4 Results and Impact As mentioned above, R-FPAP theory of change hinges upon the simultaneous implementation of service delivery and enabling environment interventions, alongside efforts to empower women and young people. There is certainly anecdotal evidence that R-FPAP are providing access to quality services by under-served populations, and bringing about changes in the overall cultural and social enabling environment, and the attitudes and behaviour of young people.
Social Development Direct Document Title Pg 24
Service delivery results Table 3. Selected service statistics New Contraceptive Users to MA Clinics by method Tubal Ligation Vasectomy IUD Oral Contraceptive Pills Injectable Implants Condoms Spermicides CYP Achievement by Outlet Static Clinic Mobile Clinic Associated Clinic Private Physician CBD / CBS
2007
2008
2009
2010
2011
276,086
269,259
308,813
335,088
392,443
6,771 430 65,635 31,494 147,023 24,637 96 359,599 142,943 21,376
6,139 628 68,129 31,251 143,081 19,927 104 366,612 115,084 14,578
5,181 914 78,880 37,547 156,801 29,488 2 405,149 123,587 15,576
4,490 759 103,076 35,928 161,284 29,551 480,491 139,992 15,969
3,169 236 116,277 46,302 190,702 185 35,572 506,187 150,941 22,686
80,192 111,170 3,917
83,726 146,615 6,608
85,394 173,905 6,687
98,840 221,007 4,682
87,794 239,981 4,785
A comparison of these overall service statistics with the data received from the Toru and Chakwal clinics broadly corresponds. The variations can be attributed to the difference in services offered at the individual service delivery points. For example, the Toru clinic does not provide sterilisation services. It is also a small community clinic used mostly by women, which explains the lower condom usage. Figure 2. Percentage of new users per service in 2011 60 50 40 30 20 10 0 IUD Overall
Injectables
Femplant
Condom
Toru Community Health Clinic
Oral pills
Sterilisation
Chakwal Model Health Clinic
The data on new contraceptive users to the clinics present an upward trend. There also appears to be decrease in the utilisation of permanent methods of contraception. This can be attributed to the Government of Pakistan’s decision not to reimburse for these services. Thus R-FPAP are now promoting other FP methods. Vasectomy rates are also much lower
Social Development Direct Document Title Pg 25
than those for tubal ligation, reflecting social norms around the responsibility for family planning, as well low levels of uptake by men of R-FPAP’s services. R-FPAP’s reported progress against the PPA logframe indicators is provided below. Table 4. R-FPAP reported progress against PPA logframe indicators Indicator Unintended pregnancies averted DALYS averted 20 Unsafe abortions averted Successful policy initiatives and/or positive changes in support of SRHR to which IPPF’s advocacy efforts have contributed SRH services provided SRH services provided to young people
2009 Outcome 116,683 26,864
2010
2011
138,382 31,860
145,782 33,564
54,593 1
64,745 0
68,208 4
2,471,230 966,479
3,315,289 1,303,334
75.0% 405,149
76.0% 480,491
80.0% 506,187
97,433 9,335
186,241 10,762
239,731 22,577
450,814 Yes
491,678 Yes
504,185 Yes
No
Yes
No
5 0
5 0
7 0
No
No
No
$1.90 / $6.60
$2.67 / $9.28
$3.20 /$11.10
Outputs 1,934,968 788,201
Percentage of clients who are poor, marginalized, socially excluded or underserved Couple Years of Protection HIV/Reproductive Tract Infection services provided Comprehensive abortion services provided Long term reversible family planning methods provided R-FPAP provides at least 6 out of 8 services in an integrated package R-FPAP provides 8 out of 8 services in an integrated package Number of national, regional and/or global financial/policymaking committees in which RFPAP is active %age of IPPF funding to R-FPAP that is performance based Does R-FPAP produce and use standardized activity cost data? Change in cost per CYP/birth averted
The evaluation team would question the high percentage of clients – up to 80% - reported to be poor, marginalised, socially excluded and under-served people. Apart from basing this assessment on geographical factors, it is not clear how this figure has been estimated. The client interviews only collect information about perceptions of quality, and do not seek to determine the interviewee’s socio-economic status. The service delivery outlets do collect information about the type of house the client lives in, as a proxy indicator for poverty. However, this information is not entered into the MIS system. The service delivery outlets
20
IPPF is collaborating with the Guttmacher Institute to develop a methodology to calculate this based on the independent ‘Adding it Up’ research. This research, which was part funded by IPPF, takes an objective and comprehensive perspective on the costs and benefits of investing in a range of family planning, maternal and newborn health interventions. IPPF is in discussion with other agencies on an ongoing basis (MSI, Guttmacher and others) to achieve convergence on methodologies used to make these calculations.
Social Development Direct Document Title Pg 26
also collect some information on poverty indicators in their fee waiver form, but this information is only used for audit purposes. The evaluation team were sent a Poor, Marginalised, Socially Excluded and/ or Underserved (PMSEU) Tool, which has been adapted to the Pakistan context and was pre-tested in three clinics in Punjab province in March 2012. The tool was expected to be rolled out to clinics in July 2012. It is expected that 5000 clients will be interviewed by early December 2012, and the results will be shared with the SARO. Possibly because it is at an early stage of roll-out, the tool was not mentioned in any of our interviews.
Advocacy results In the past year, R-FPAP has achieved some notable advocacy results. They drafted a resolution which calls for the inclusion of Comprehensive Sexuality Education in the secondary school curriculum; as well as recognition of the reproductive health needs of young people, and the need for youth-friendly services. This was subsequently tabled and passed in the Punjab Assembly. The Resolution represent a commitment from the provincial government to further examine the issues raised. With funding from the DFID-funded Maternal and Newborn Health Research and Advocacy Fund, R-FPAP has also been advocating for the inclusion of a Minimum Initial Service Package (MISP) for Reproductive Health in the Standard Operating Procedures (SOPs) of the National Disaster Management Plan. This would strengthen maternal and neonatal health services in crisis and post-crisis situations through increasing awareness of the SRH needs of women in emergency situations and providing appropriate protocols for future disaster management plans. R-FPAP has already achieved some success at the provincial level, as SOPs for the Punjab Disaster Management Authority (PDMA) are being drafted to include MISP for Reproductive Health.
Wider impacts There is evidence of wider impacts of R-FPAP’s work. Our discussions with R-FPAP’s peer educators revealed that they felt that the primary impact of the programme was the change in their own attitudes and behaviours, for example in terms of their perceptions about women who they saw going to FP clinics. The young people we spoke to21 told us that attending peer education sessions works ‘like a fire in the jungle’, and there is a significant multiplier effect. The impact of the programme was plain to see in their impressive communication skills, knowledge of SRHR, and empathy with the issues facing young people. Those participants who were new to the programme were less confident and less able to express themselves in comparison to those young people who had been in the programme a number of years – demonstrating the impact that the programme has had on their capabilities. This sense of changes in their own behaviour was echoed by the theatre group in Mardan. These young girls, some as young as 12, talked about feeling greater empathy for people as a result of their involvement with the group. They also talked about how they were able to address their family’s reservations about their involvement in theatre group, and that their families are now proud of them. They also talked about feeling greater responsibility and empowerment to tackle such issues if they saw them occurring around them. However R-FPAP’s monitoring is based on mainly quantitative indicators with very little attention given to outcomes and impacts. Apart from the Swara evaluation, and the social audit report exercise that was undertaken in Chakwal, we have seen no other evidence of a qualitative approach to monitoring. We are told that further qualitative studies on the impact of R-FPAP projects are available. However, no rigorous evidence of the impact of the core 21
Focus group discussion with youth peer educators, Islamabad/ Rawalpindi
Social Development Direct Document Title Pg 27
programme was provided to the review team beyond service figures and anecdotal evidence. Preparatory work for the clinics only includes the basic feasibility studies described in section 2.1 above. These do not provide any information about health outcomes, or community attitudes towards SRH. Thus there is no baseline against which change can be assessed. In addition, R-FPAP are not able to assess what percentage of the target populations they are reaching in their catchment areas. Few staff were able to clearly identify the groups they were unable to reach and why. As mentioned above, we recognise that robust baseline and impact assessment studies are expensive to undertake, and that all organisations must carefully balance the investments they make. However, given R-FPAP’s lengthy track record, and the precarious political status of family planning in Pakistan, it seems a missed opportunity not to invest more in monitoring and research. For the sake of argument, if R-FPAP had commissioned a longitudinal study on the impact of contraceptive uptake on people’s health and socio-economic status 20 years ago, this would now be yielding valuable evidence to support R-FPAP’s advocacy on government prioritisation of family planning. Studies which aim to measure the impact of R-FPAP’s educational, awareness raising and advocacy activities on social norms, attitudes and behaviours have also not been conducted. However, R-FPAP have recently undertaken a mapping of political stakeholders, which aims to assess their position on SRH and FP issues through a questionnaire survey, and an analysis of their reputation, press statements, and stance in Parliament. It is hoped that this will provide some sort of baseline against which future change can be measured. Apart from this, we were also not able to identify any efforts to put into place more longer-term monitoring and impact evaluation measures. However, we were told that the Monitoring & Evaluation strategy is currently being developed.
Additionality and attribution R-FPAP is providing extremely valuable services, and particularly in those areas which do not have good health service coverage. Their primary target group are the poor. In the NGO sector, R-FPAP is the largest family planning service provider with a market share of 91%. Nationally R-FPAP contributes 8%22 of the national CYPs. 0.4%23 of these can be attributed to DFID PPA funding received by R-FPAP. There are other important policy level inputs that the organisation is providing at the national and provincial levels which cannot be monetised and attributed. The flexibility provided by the IPPF core funding is highly valued by R-FPAP. Recently, this has enabled them to integrate issues of youth and women’s empowerment more fully across R-FPAP’s work. This focus on integration and training is not possible through other resources, which are restricted and tied to particular projects. Core funding also supports learning processes including the Annual Planning Meetings where HQ staff, Regional Directors and project management offices meet, and where lessons from Pakistan and regionally are presented and disseminated. R-FPAP strongly believe that the ability to put core funding towards these processes has been critical to their ability to strengthen their learning. The WAC project described earlier in the report has also been funded by the IPPF Innovation Fund and is a good example of how core funding has been used strategically to support innovation and working in the most challenging circumstances. R-FPAP staff are very successful at attracting donor funding for a range of projectised activities, and are currently managing large number of these. These projects offer further opportunity for learning and innovation through cross-fertilisation of ideas. However because 22 23
Total CYPs generated by FPAP in 2011 ÷ Total CYPs generated in Pakistan (Government Report) Share of R-FPAP in national CYPs (8%) x DFID PPA funding (4.5%) for R-FPAP= 0.4%
Social Development Direct Document Title Pg 28
they are funded mainly through restricted grants, few projects include funding for personnel costs. Thus core funding is critical for providing staff salaries for core staff.
2.5 Value for Money Unit Costing A comprehensive unit costing exercise was conducted for three R-FPAP service delivery points: (i) Model Health Clinic (Chakwal); (ii) Mobile Service Unit (Chakwal); and (iii) Community Health Clinic (Mardan). Service statistics and financial data was used for 12 months starting from 1 st January to 31st December 201124. Learning from these was applied at organisational level to obtain unit costs at organisational level. For detailed methodology please refer to Annex 5.
Costing Results: Clinical Level Costing The table below summarises the difference in service provision between the three selected service delivery points. This is important to understand the cost behaviour. Table 5. Services provided by different service outlets Model Health Clinic
Mobile Service Unit
X X X X X X X
X X X X X
FP Counseling Condoms Pills Injectable IUCD Implants Sterilisation
Community Health Clinic X X X X X
The figure below summarises the cost per visit (initial and follow-up) for each FP method offered and by three service delivery points. Figure 3. Cost per visit
24
Field visits for assessing provider time and observe the treatment practices followed took place in June 2012.
Social Development Direct Document Title Pg 29
The figure below summarises the cost per CYP for each FP method offered and by the three service delivery points. Figure 4. Cost per CYP
The overall cost (method mix) per CYP ranges from GBP 3.90 (community health clinic Mardan) to GBP 0.90 (mobile service unit Chakwal). The main factors behind this variation are different level of service utilization, and the increase in overheads with increased level of facility. Analysis of the costing results show that short-term methods are costly to provide compared to long-term methods, however clinic managers noted that that there is greater acceptability for short-term methods.
Costing Results: Organisational Level (Family Planning) Using the findings of the clinic costing exercise, a further exercise was carried in order to calculate costs per visit and CYP at the organisational level. Unit costs were also analysed by service delivery channel. It was noted that approximately 50% of the CYPs generated and reported by R-FPAP are through private practitioners who are supported through provision of FP commodities and trainings. Costs were also calculated to examine cost behaviour with and without private practitioners25.Figure below presents overall costs per visit with and without private practitioners. Figure 5. Overall organisational costs per family planning visit
25
It should be noted that the costs referred to here are those incurred by R-FPAP and not by the private practitioners themselves. These include, therefore, the costs of the inputs provided by R-FPAP to the private practitioners – for example, commodities and trainings – but not those costs that are directly incurred by the private practitioner – for example, staff salaries, utility costs. Obviously the overall cost of delivering these services will be higher than this.
Social Development Direct Document Title Pg 30
Figure below presents overall costs per CYP with and without private practitioners. Figure 6. Overall organisational costs per CYP (FP only)
From the figures above it is apparent that private practitioners contribute greatly to the cost effectiveness to the programme26. This further suggests that by creating such cost effective networks R-FPAP can further reduce costs and also improve efficiency of spending. However it is important to note that, beyond being located in under-served areas, private practitioners are unlikely to specifically target the poor in the same way as the R-FPAP clinics, i.e. through reduced fees, outreach work, and an emphasis on quality of care. Notably, R-FPAP made very little reference to their private providers network – both overall, and when asked about how they ensure access to the poor. We do not have any information about the poverty profile of the clients of the private practitioners, or how charges to clients are determined (and whether R-FPAP is able to influence these). It is likely that their fees are higher than those of the R-FPAP clinics, which will affect poor people’s ability to access them. The table below presents a comparison between the direct costs of providing different FP methods by R-FPAP and in Asia. Table 6. Comparison of direct costs of providing FP methods R-FPAP27 Asia 28 Methods £ £ IUD 0.53 0.40 Injectable 2.67 4.23 Implant 3.28 5.00 Condom 11.98 2.53 Oral Pills 6.77 3.65 Sterilization 0.67 1.01
26
As explained earlier, these figures are based only the costs incurred by R-FPAP. For this reason while performing cost benefit analysis, a sensitivity test was built around the private practitioners to test the cost effectiveness of R-FPAP FP programme with and without private practitioners. 27 Source: R-FPAP, FP costing exercise 28 Source: Guttmacher Institute (2012)
Social Development Direct Document Title Pg 31
The Guttmacher study also reported that the average cost of providing a method mix of services in Asia is GBP 2.80. Similar services are provided by R-FPAP at an average cost of GBP 1.71 per user. R-FPAP has been able to provide services at a much lower cost as compared to the Asian region.
Cost Effectiveness There are a number of impacts that may have been achieved as a consequence of RFPAP’s FP interventions. Of these, only those which can be valued with reliable estimation and which are within the scope of FP services have been accounted for. The following impacts have been valued as part of the benefits valuation: Reduction in unwanted pregnancies; Reduction in infant mortality; Reduction in maternal mortality; Reduction in unsafe abortions; and Savings resulting as reduced healthcare costs, which in turn will allow more freed up resources to spend on other more productive activities. There are other gains as well which have not been accounted for in the benefits valuation as there is insufficient evidence to provide a reliable estimate: Reduction in fertility rate, reducing pressure on resources, environment, etc. Fewer orphans due to decreased maternal mortality and lower costs to society as a result; Increased per capita household expenditure on different activities due to fewer children, leading to more savings; Value to mothers of having children by choice rather than by chance; Savings resulting from reduced maternal and child mortality which, in turn allow resources freed up from household or health sector budgets to be spent on other more productive activities; and Savings associated with reduction in pregnancy related services (e.g. ante natal care and deliveries). The cost per DALY averted is only £16, which is 47 times less than the current GDP per capita of Pakistan (£750). The Commission on Macroeconomics & Health and the World Health Organisation (WHO) recommend that an intervention is considered to be very costeffective if the ratio does not exceed the average GDP per capita of the country. Using this benchmark FP services provided by R-FPAP are highly cost effective. The interventions are also producing a very high benefit to cost ratio of 68. This means that every £1 invested in the FP services gets a return of £68. A sensitivity test was also performed to check the robustness of the results by excluding the CYPs produced through private practitioners. This changes the cost per DALY averted to £28 and cost benefit ratio to 35. Even though this is a significant change, the cost per DALY averted is still 28 times less than the average GDP per capita of Pakistan, and therefore, still highly cost effective. The table below shows other health impacts that can arise as a consequence of delivering the FP services by R-FPAP.
Social Development Direct Document Title Pg 32
Table 7. Other estimated health impacts of R-FPAP’s FP service delivery29 Attribution to Other Impacts due to FP Services Numbers PPA Grant30 Maternal deaths averted 119 5 Child deaths averted 1,314 59 Unsafe abortions averted 8,621 388 Unintended pregnancies averted 79,118 3,560 Births averted 51,942 2,337 DALYs averted 55,078 2,479 Source: MSI Impact 2 using service statistics from R-FPAP
2.6 Sustainability The sustainability of R-FPAP and its services depends on a range of factors – both internal and external. These are the enabling environment, including the policy and social context, demand for services, the ability to generate adequate revenue and internal organisational factors. R-FPAP’s biggest immediate challenge comes from the changing governance context, and the de-prioritisation of family planning issues by provincial governments. IPPF is R-FPAP’s largest donor, however, they are also able to generate a significant amount of funding from other donors in addition to IPPF. They also earn a considerable income from social marketing of contraceptives. However in terms of financial sustainability, R-FPAP remains dependent on donor funding, raising only 28% of its funding through locally generated income. R-FPAP’ senior management are cognizant of sustainability issues, and actively making efforts to generate diversified income through initiatives such as the Rahnuma Training Institute - a training facility which they are marketing to donors and INGOs – and creating sustainability reserves. However they do not yet have a clear-cut contingency plan in place in case of a cut in external funding. They have faced this situation previously. In 2001, the US Government imposed the Gag Rule, which prevented overseas US-funded NGOs from providing abortion information and care to women, and from engaging in advocacy to change abortion laws and policies, even if the NGOs used their own funds. As a result, RFPAP’s US funding was withdrawn, forcing them to close a large number of facilities. At the time R-FPAP was reaching 10 per cent of the population. By 2010, this number had fallen to 3 per cent. Even though the Gag Rule was rescinded in 2009, US funding has still not been restored. It was suggested that they would probably cope with a drop in funding by closing down some of the clinics opened in the last two years - most likely those clinics in the most remote areas. This loss in funding would also affect their efforts to bring more integration into their work, particularly in terms of youth and women’s empowerment. This focus on integration, training and learning is not possible through other resources, which are restricted and targeted to particular projects. There are further opportunities for R-FPAP to generate greater income. Currently, R-FPAP does not apply a structured methodology to pricing its services – it seems that pricing is based on untested perceptions about willingness to pay, and some comparison with the local market. Greater attention must be paid to costing services. Once this information is 29
The MSI Impact 2 tool was used to calculate these estimated impacts. This uses a different methodology to the results presented in the logframe 30 Calculation (2011 data): (IPPF unrestricted funding to R-FPAP ÷ Total Funding of R-FPAP) x (DFID’s PPA Funding); (PKR134,899,440÷PKR597,826,629=23%). (23% x 20%) = 4.5%
Social Development Direct Document Title Pg 33
available, pricing should be determined on the basis of a combination of factors, including what it costs the organisation to deliver the services, willingness to pay and conducting a price elasticity exercise. This will also require implementing a sound method to identify poor people who cannot afford to pay. Better monitoring and analysis of the socio-economic status of clients’ in each facility, and the development of poverty indicators could allow clinics to charge higher fees from wealthier clients to subsidise services for the poor. For example, some participants in the mixed-income non-user FGDs group told us that they would pay up to Rs. 100 to access RFPAP services. R-FPAP could explore the feasibility of using the Benazir Income Support Programme’s (BISP) poverty scoring data which is available for all districts in Pakistan to identify poor people more systematically. However, before implementing any such changes it will be important to assess whether and how this might affect equity and access for the poor as there is no local evidence available on price elasticity of user charges. It may be, for example, that as R-FPAP expands its network into more rural areas, there will be little variation in the socio-economic status of clients with the vast majority being poor. In such a scenario, maintaining a balance between urban and rural locations will be important – with wealthier clients in urban areas subsidising the rural network. R-FPAP is already quite successful in securing funding from other international donors and donor dependency ratio has remained unchanged at 73% in 2010 and 2011. However, they are also aware of the need to increase locally generated revenue, and have developed a Resource Mobilisation strategy, which includes local fundraising. A Survival ratio calculation31 shows that a decline in reserves of 121 days in 2010, to a reserve of 109 days in 2011. This is a reasonable level of reserves but R-FPAP should develop internal standards and mechanisms through which they can monitor how this is changing over time. Given the very low priority that is being given to FP at the moment it is unlikely that R-FPAP will be able to achieve much support from the government in the short-term. It may also be difficult to undertake fundraising initiatives for the general public in a context where SRH issues are still regarded as controversial or sensitive. R-FPAP’s emphasis on combining service delivery and work on the enabling environment is critical to the sustainability of changes on SRHR, and means that the changes will be sustained over the medium to long term. However, R-FPAP need to pay much greater attention to monitoring the kinds of changes that are occurring as a result of their efforts in order to enhance the chances of sustainability. A key challenge for R-FPAP’s sustainability is the external political environment. Post-18th amendment, most Ministries have been devolved to the provincial level. However, there are concerns about whether the provincial governments have adequate resources and technical capacity to manage these departments. Provincial health and population policies and strategies are still being developed, and while FP is mentioned is many of these, questions around the political will for promoting SRH remain. Certainly R-FPAP have found reproductive health issues are not a particularly attractive agenda for politicians as investment in reproductive health does not show visible gains for a number of years. The uncertainty is already impacting R-FPAP’s service delivery. The standard model of R-FPAP clinics was that they agreed the locations with government, and then referred to government health facilities. Without a clear commitment of funds, this is impacting their ability to refer clients. Indeed one of the objectives of R-FPAP’s strategic plan is to ensure that provincial governments allocate sufficient funds for FP programmes. R-FPAP will have to monitor these developments closely, and may have to significantly reorient their programme as a result. The setting up of Provincial chapters of the 5b Alliance will also help to develop targeted advocacy messages and strategies. 31
Survival Ratio: General Reserves x 365 days / total income
Social Development Direct Document Title Pg 34
Conflict and insecurity, coupled with the militancy of ultra-conservative groups is another key challenge to sustaining demand, enabling R-FPAP to continue its service delivery work, and promoting the willingness of politicians to engage with SRH issues. The retention of skilled staff has already been mentioned above. R-FPAP should take urgent action to ensure that staff are remunerated, at least according to government rates.
2.7 Lesson Learning and Innovation R-FPAP’s knowledge management and lesson learning approach is focussed around regular progress reviews, and information-sharing at national and regional meetings. All projects are documented in Quarterly Progress reviews, and the analysis is expected to provide information on trends, constraints, lessons learned, best practices and possibilities for scaling up. R-FPAP also produce yearly lessons learned reports for their projects which aim to: document the main achievements of the year; analyse what worked well and what didn’t work so well, and the reasons for this; and outline recommendations and lessons learned for the future. Learnings are incorporated into the next quarter, with a view to tweaking or reorienting projects as necessary. HQ staff, Regional Directors and PM offices meet at Annual Planning Meetings, where lessons from Pakistan and regionally are presented and disseminated. R-FPAP strongly believe that the ability to put core funding towards these processes has been critical to their ability to strengthen their learning. There are a large number of individual lessons learned documents, and which include various operational messages and recommendations. These are not linked to a more detailed analysis, and seem quite topline and general. It would be good to see a more consolidated analysis of lessons emerging across similar projects, or in similar districts, which provides a more detailed discussion, and perhaps links to the international evidence also. Few staff were able to talk about how R-FPAP’s learning processes work, or about institutional processes and channels through which they can disseminate new ideas and innovations and promote uptake. It seems that mostly learning is limited to the generation of reports. It also appears that most of the lesson learning process occurs at the HQ level, even if findings are disseminated to the rest of the organisation. R-FPAP have been involved in imparting their learnings to other MAs. This is on an ondemand basis, rather than through a regular mechanism set up through the Regional Office. Recently, the MA in Afghanistan asked for their support in setting up a youth helpline, and mobile clinics. R-FPAP also helped the MA in India to design age-appropriate materials on comprehensive sexuality education. R-FPAP representatives also participate in regional capacity building workshops and Strategic Planning workshops which seem to be a key forum for sharing learning. Focal persons from other MAs have also visited R-FPAP. It is important to note that there are important constraints to R-FPAP actively disseminating its work. To some extent, there is a concern that if R-FPAP were to take a higher profile, this might compromise its ability to reach poor and excluded people, as well as threaten the space that currently exists for work on SRHR. This also has an impact on the extent to which they can disseminate findings from their most innovative work, i.e. the Swara project, the Saathi Foundation, abortions, etc. R-FPAP’s recognition of the critical importance of women’s empowerment to addressing their SRHR needs is highly innovative. There are few organisations in the country that use a similar integrated approach to service delivery, advocacy and empowerment. Their approach
Social Development Direct Document Title Pg 35
to communicating sensitive messages to vulnerable groups, as seen by their Women as Compensation programme, and youth peer education work, is also very innovative. This has included using healthcare as an entry point, obtaining fatwas from religious scholars about the acceptability of their messages, recruiting staff from their own communities, and addressing staff’s own values and attitudes through training first. The flexibility afforded to them through the core funding appears to be critical in this regard, as it allows them to pilot these innovative approaches. R-FPAP staff feel that there is flexibility and openness to new ideas within the organisation. However, it was difficult to see how the insights, experiences and innovations of the frontline workers at the local level are captured. Certainly, the lack of learning forums and opportunities – either through meetings or collective trainings - was cited by the youth peer educators as a key area for development.
3. Conclusions and Recommendations 3.1 Conclusions It is widely acknowledged that R-FPAP have been instrumental in providing access to quality services by under-served populations. By locating clinics in those areas where no other facilities exist, and by providing services at low cost, R-FPAP are addressing key access barriers of transport and fees. R-FPAP are well-regarded for their technical expertise in health, and have invested considerable resources in strengthening their Quality of Care approach, and in developing systems to monitor and assure quality. Clients are happy with their services, with the proximity of the clinic, the cost of medicines, and the provision of advice and information considered key factors. However, the majority of R-FPAP clinics are not based in the poorest districts, where it is likely the need is even greater. R-FPAP’s strategy for reaching the poor and marginalised is based mainly on locating clinics in under-served areas and keeping fees low. However their poverty monitoring is weak. Baselines are not undertaken, nor any kind of formative research before establishing a new clinic. Information about the socio-economic status of clients is also not centrally recorded and analysed. As a result there is little actual evidence about how effective this strategy is, and whether there are people who are not being reached. R-FPAP enjoy a high level of credibility and a strong working relationship with government. They are often invited to provide technical assistance to government departments, and have a long-standing partnership with the government of Azad Jammu and Kashmir to provide services and contraceptives to 148 clinics. In many cases R-FPAP have been able to leverage the credibility and trust that they have established with government to good effect. R-FPAP are particularly skilled in communicating sensitive messages to vulnerable groups, as seen by their Women as Compensation programme, and youth peer education work. They have evolved a tailored approach to this which involves using healthcare as an entry point, obtaining fatwas from religious scholars about the acceptability of their messages, recruiting staff from their own communities, and addressing staff’s own values and attitudes through training first. Unfortunately, the impact of awareness raising work in not measured, and so it is difficult to know which types of initiatives are most successful and why. R-FPAP has achieved various successes in terms if its advocacy, most recently in terms of ensuring greater attention is paid by government to the SRH needs of young people. RFPAP are also very focussed on building relationships with civil society stakeholders and have recently convened the MDG 5b Alliance – a group of civil society organisations working on SRH – with the aim of taking a more strategic approach to advocacy. However, their
Social Development Direct Document Title Pg 36
advocacy portfolio is large and diverse, and includes a wide range of activities. In part this is due to R-FPAP’s commitment to a holistic approach, and the recognition that sustainable change can only be achieved through better service delivery and changes in the enabling environment. One consequence of this is that advocacy messages are fragmented and their effect possibly diluted. There is a clear recognition that maternal mortality is linked to a lack of services as well as the low status of women. Thus R-FPAP have increasingly become involved in work on gender-based violence; skills development for women; and ending early/ forced marriage. Targeted efforts have also been made to understand and address the needs of particular groups – including adolescent girls, the transgender community and sex workers. However a broader focus on social exclusion is missing. There is also an increasing focus on working with men and boys – both in terms of their role in supporting women to access SRH services, and as clients themselves. R-FPAP is delivering medium VfM. This means that most of the necessary basic controls exist and are being used to a certain extent. There is however significant room for improvement, related mainly to capacity building, improving and building upon existing systems, introducing new controls and using more efficient accounting software. Improvements are also needed on financial management reporting and costing of services. Lessons learned are regularly documented with the expectation that these will be used when designing new programmes or to make any adjustments to existing ones. However it is not clear what the process of identifying and distilling those lessons is, what information this is based on, and to what extent staff from lower levels in the organisation are involved. There is anecdotal evidence that R-FPAP is having an impact on poor people’s access to services, the cultural and political enabling environment, and the empowerment of women and young people. However monitoring is based on mainly quantitative indicators with very little attention given to outcomes and impacts. There are no baselines against which change can be assessed and we were not provided any rigorous evidence on impact. Approximately 50 per cent of the CYPs generated and reported by R-FPAP are through private practitioners who are supported through provision of FP commodities and trainings. However, even without taking these account, R-FPAP’s services remain highly costeffective. They are also achieving a very high cost-benefit ratio.
3.2 Lessons Learned 1.
Addressing the SRH needs of poor women is a complex task which requires a multidimensional approach. However, it is important to balance the need for a holistic approach with the need for strategic focus and clarity. This is particularly so in a large, federated organisation such as R-FPAP where maintaining a clear, collective vision is critical to ensuring that all parts of the organisation are working towards the same goal. R-FPAP should articulate and consistently communicate a collective vision which resonates across the organisation and shapes staff’s approach to innovation, learning, and making a difference.
2.
R-FPAP has recently made considerable investment in putting in place various management, reporting, monitoring and communications systems. This is critical for ensuring greater efficiency and effectiveness. However, this increased focus on performance monitoring – at least during this initial transition phase - has resulted in a dependence on routine service data. There is a real lack of attention to assessing needs and monitoring changes at the target population level, and this has led to a neglect of outcomes and impacts. While impact assessments are costly, they are important
Social Development Direct Document Title Pg 37
investments to make, particularly as they generate the evidence to support the case for further investment in SRH. 3.
R-FPAP have made targeted efforts to understand and address the needs of particular groups most vulnerable to poor SRH – including adolescent girls, the transgender community and sex workers. However a broader focus on social exclusion – which targets the very poor, ethnic and religious minorities, caste and occupational groups, and disabled women amongst others – is missing. These groups will exist in almost every community that R-FPAP works in but require targeted measures in order to be identified and reached. Recruiting staff from the same communities that they will serve is an important way of ensuring that service provision is relevant and appropriate for the target area, and is also a good way for the organisation to gain knowledge about the context, and to develop relationships with key stakeholders. However, local staff cannot be relied upon to provide knowledge of and access to the poorest and most marginalised groups, who can often be invisible to more privileged segments of the community. This work needs to be informed by research into the barriers for excluded groups in accessing SRH services.
4.
R-FPAP’s frontline workers – LHVs, counsellors, community-based distributors and peer educators – are constantly innovating in order to deal with the challenges that they face in their everyday work. In R-FPAP’s large, federated structure there is a risk that their experience and expertise is not harnessed, as lesson-learning and decision-making often occurs at higher levels. It is important to provide relevant and appropriate forums for these workers to share their experiences and learnings, so that they can inform future programming.
5.
R-FPAP’s network of private practitioners has proven to be highly cost-effective. It is likely that R-FPAP’s own network of facilities, while also highly cost-effective, is comparatively less so because of the targeting measures in place to ensure access to the poor and marginalised. An assessment of the poverty profile of the clientele of the R-FPAP clinics and the private practitioners would provide useful information on the costs of targeting hard-to-reach groups.
6.
Perhaps because of its success in securing donor funding from a range of donors, RFPAP has not paid much attention to improving financial sustainability. However, there are significant opportunities for generating greater income from services whilst at the same time increasing access to vulnerable groups. Understanding better what it costs to deliver services, and developing a more structured methodology for pricing services, based on a more analytical understanding of clients’ differential willingness to pay is a key opportunity for raising income, as well as improving pro-poor targeting measures. Further work on promoting efficiency and value for money also has an important role to play in the organisation’s sustainability strategy and should be approached with this in mind.
7.
Core funding has proved vital for enabling R-FPAP to develop a more multi-dimensional approach to addressing SRH, to integrate issues of women’s and youth empowerment across R-FPAP’s work, and to establish learning and information sharing systems. There is scope for R-FPAP to use this funding to take its learning activities much further, and to position itself as a more ‘thinking’ organisation on SRH issues.
Social Development Direct Document Title Pg 38
3.3 Recommendations Relevance Make greater efforts to understand poverty and exclusion dynamics in target districts. This could be done by adopting a Rapid Appraisal methodology which would involve a desk review, interviews with key informants, and a limited number of focus group discussions with community stakeholders, including men, women, and adolescent girls and boys. This would provide a more context-specific analysis of the barriers, key vulnerabilities and coping mechanisms of poor people. It would also provide some baseline information about knowledge, attitudes and practices against which change could be measured. Develop a more well-rounded strategy for addressing the needs of men. R-FPAP has successfully targeted men in terms of their ability to influence household decision-making and social norms around women’s SRH needs. R-FPAP clinics are aimed at men as well as women, but do not seem to be tailored towards their needs. Certainly, in the Pakistan context, it may be inappropriate to have mixed clinics. However, if R-FPAP is going to make the shift from seeing men as a key audience for behaviour change messages about women’s access to SRH services, to cultivating them as valued clients, then a detailed needs assessment needs to be undertaken to understand the barriers to men seeking out services and approaches which would overcome them. The modalities for delivering services to them should also be reviewed. Pilot community monitoring initiatives which enable citizens to provide feedback on services. One way of strengthening R-FPAP’s Quality of Care approach would be to enable clients to provide feedback on service delivery. The global evidence shows that individual feedback mechanisms, such as complaint boxes, are rarely used by poor people because of confidence issues, and because of the risk that this represents. Collective monitoring mechanisms, such Citizen Report Cards and social audits work much better, and have been proven to bring about improvements in governance and service delivery, and to make citizens feel more empowered to voice their needs. A social audit has already been undertaken in Chakwal. Community management structures, such as Village Health Committees could also be explored, although it would be important to ensure that these are representative of poor communities. Further pilots should be tried with a view to integrating citizen monitoring as a regular feature of R-FPAP’s service delivery.
Efficiency and Value for Money IPPF to assist R-FPAP in improving systems in order to achieve better Value for Money. Investments in capacity building, improving and building upon existing systems, introducing new controls and using more efficient accounting software are required. Improvements are also needed on financial management reporting and costing of services. R-FPAP also do not have the in-house mechanisms and expertise needed to enable them to adequately cost their services, and measure cost effectiveness. Currently, they are using a very rudimentary method for allocating costs per CYP – by dividing all costs reported under ‘Access’ by CYPs. For detailed recommendations on financial management please see Annex 4. IPPF can further assist in rolling out the branch performance tool. Make improvements to R-FPAP’s Management Information System (MIS). IPPF should work with R-FPAP to develop an Management Information System which goes beyond collecting service data, to providing information programme, project and activity status at both HQ and regional levels. A further investment could be made to facilitate linkages between the different levels of the organisation and as well as two-way flow of feedback and information.
Social Development Direct Document Title Pg 39
Improve the involvement of staff at lower levels in the organisation in strategic level processes. R-FPAP’s alignment with the 5 As demonstrates that its strategic positioning is informed by international perspectives and global priorities. However, it is just as important to ensure that district and facility-level staff who are at the forefront of delivering services and engaging with beneficiaries are involved in strategic planning processes. This will ensure that R-FPAP’s priorities are informed by local priorities, as well as understanding what the organisation can realistically deliver.
Effectiveness Revisit the remuneration package of service providers in order to attract and retain a skilled health workforce. R-FPAP should undertake an analysis of the costs of hiring replacements (which can involve initial loss of productivity due to loss of institutional memory, lower skill and knowledge levels of new staff, and other recruitment related costs), against what it would cost to retain more staff, most likely through increasing salaries. It may be that R-FPAP opt to accept recruitment costs for certain categories of staff, and retention costs for others. For staff whose retention is prioritised, some element of performance may have to built in to their contracts. Either way, baseline salaries for every category of staff, should be reasonable and competitive in the local market, and should make employees feel rewarded for their work. Given that R-FPAP’s unit costs, cost per CYP and cost per DALY are extremely low, there is clearly room for increasing salaries. Othe staff entitlements, such as trainings and supportive supervision should also be ensured. R-FPAP’s HR policies should be clearly communicated, ensuring that all staff are aware of mechanisms through which to channel concerns and issues, and feel empowered to do so. Consider streamlining advocacy work and focussing on more systemic messages on the status of family planning in Pakistan, government resource allocation, and its position within the health system. This would include concerted and long-term action, a focussed agenda, with well-developed and clear messages. Such a campaign could mobilise other civil society actors, and be undertaken in multiple spaces, such as the research, media, parliamentary lobbying, community mobilisation, etc. It would be interesting for R-FPAP to explore working in both collaborative and more confrontational ways with government. This could include the use of more innovative approaches, e.g. the use of public finance specialists to conduct in-depth analysis of government allocations for FP and develop strategic recommendations for creating more fiscal space.
Results and Impact IPPF to assist R-FPAP to strengthen monitoring and evaluation processes. Currently two departments are involved in M&E. Their roles and responsibilities are not clearly defined, and there is a lack of coordination between them. Particular attention is needed for impact monitoring and evaluation – the M&E strategy that is being developed should include provision for baselines on health outcomes, service delivery trends, and knowledge and attitudes. Attention should be given to systematic collection of high quality qualitative data as part of R-FPAP’s M&E systems, and developing in-house capacity to oversee the collection and use of qualitative data. Systems for monitoring the poverty profile of clients should also be revisited, and any information collected should be centrally recorded so as to make possible accurate reporting on the percentage of poor, marginalised, socially excluded or underserved populations reached.
Sustainability R-FPAP to develop a comprehensive sustainability and financing plan. There is an urgent need for the organisation to decide on the future funding mix based on this plan. The figure below provides an overall framework for developing such a plan.
Social Development Direct Document Title Pg 40
Donor agencies Donor agencies Membership fees Local Income through fee etc Local income
NOW
Other Income
FUTURE
Develop a better understanding of what it costs to deliver services, and a more realistic pricing strategy. This should be based on a more analytical understanding of their clients’ differential willingness to pay, which is a key opportunity, and could be used to enable wealthier clients to subsidise poorer ones. Ensure that marketing and communications are appropriate for reaching different target groups. R-FPAP to track sustainability ratios. In the changing scenario on financing, R-FPAP should very closely track and report on at least two important sustainability ratios i) donor dependency and ii) survival ratio. R-FPAP’s donors to understand that changing norms, attitudes and behaviours around women’s status is a long-term process and requires sustained investment. The Women as Compensation project for example has achieved remarkable results in terms of creating awareness in its short time-frame. Beyond its current extension period, WAC will be funded through the core programme. R-FPAP is a well-resourced organisation, and is fortunately able to sustain the programme itself. Had this not been possible, there would have been important questions about the risks that of raising the confidence and expectations of highly vulnerable women and girls, and then removing any channels of assistance.
Learning and Innovation IPPF assist R-FPAP to strengthen internal knowledge systems. These should enable staff from all parts of the organisation to share information, extract learnings, provide feedback, and be used as a platform to disseminate innovations and learnings. For R-FPAP to maintain its leading position on SRH in Pakistan, it is critical that its service delivery strategies, advocacy activities and trainings are based on the latest national and international evidence. A new knowledge management system could also capture these.
Social Development Direct Document Title Pg 41
ANNEX 1: Documents Reviewed IPPF PPA Logframe Guttmacher Institute (2012) Adding it up: costs and benefits of contraceptives services estimates for 2012, Guttmacher Institute Hussain, R. and Khan, A. (2008) Women’s perceptions and experiences of sexual violence in marital relationships and its effect on reproductive health, Health Care for Women International, Volume 29, pp. 468-483 Jaffer, R. and Jaffer, R. (2012) Women as Compensation: Reaching the Survivors of Coerced Marriages and Violence in Pakistan: End Evaluation and Documentation of Best Practices, Institute of Social Sciences Orient Consulting Group (2011) Condition of Home-Based Workers Union Council 253 Faisalabad: Baseline Survey Report, Rahnuma-FPAP/ ILO/ UN Women Population Council (2004) Unwanted Pregnancy and Post-Abortion Complications in Pakistan: Findings from a National Study, Population Council, Islamabad Rahnuma-FPAP, IPPF Annual Report, 2011 --- Audit Report (year ended December 2011) --- Chakwal Social Audit report (not dated) --- CHC Feasibility Reports --- Entitlement Index Final Report (not dated) --- Gender Equity Policy (not dated) --- No Refusal Policy (not dated) --- Lessons Learned Reports - Projects --- Media Kit Women and Children’s Health and Welfare during Emergencies (not dated) --- Monitoring, Evaluation and Research Section Activity Report – Jan-Dec 2011 --- Monitoring and Evaluation Strategy 2010-2014 (draft) --- PMSEU Estimation Tool --- Quarterly Program Planning Meeting Minutes, Peshawar Region, 4th April 2012 --- Resource Allocation Information Report, 18 th May 2011 --- Risk Management Plan --- Strategic Plan 2010-2014
Social Development Direct Document Title Pg 42
R-FPAP (2008) Socio-cultural determinants and Economic consequences of Induced Abortion Shaikh, B. T. and Rahim, S. T. (2006), Assessing knowledge, exploring needs: a reproductive health survey of adolescents and young adults in Pakistan, European Journal of Contraceptive and Reproductive Healthcare, Volume 11, Number 2, pp. 132-137 Turner, K. L. and Page, K. C. (2008) Abortion Attitude Transformation: A Values Clarification Toolkit for Global Audiences, Ipas, USA
Social Development Direct Document Title Pg 43
ANNEX 2: List of people interviewed R-FPAP Headquarters Syed Kamal Ali Shah, CEO Rizwan Baig, Director, Access Amina Akhsheet, Director, Adolescents Nabila Malick, Director, Advocacy, Resource Mobilisation and Donor Liaison Asifa Khanum, Director, Monitoring, Evaluation and Research Azeem Sultan Mir, Director, Programme Planning Division Ch. Saqlain Manzoor, Director, Finance and Budget Mudessar Hussain, Director Internal Audit Tariq Malik, Director Administration Muhammad Alam, Programme Manager, Programme Management Unit Asif Kareem, Knowledge Management Manager Regional Offices Gauhar Zaman, Regional Director, Peshawar Amir Hameed, Regional Director, Islamabad Mardan PMO Sohail Kakakhail, Project Manager Nabeela Dar, Quality Assurance Doctor, Mardan Toru Community Health Clinic Rubina Aslam, LHV/ facility supervisor Saiqa Usman, Counsellor, Choices programme (also ex-WAC programme counsellor) Noreen, LHV/ Community-based Distributor Razia, LHV/ Community-based Distributor Laboratory technician Chakwal PMO Noor-ul-Basar, Project Manager Chakwal Model Health Clinic Dr Huma Fatima, Doctor Shamim Akhtar, LHV Samia Nawaz, LHV Manzoor Hussain, Laboratory Technician Mahfooz Sadiq, LHV, MSU Qamar Sultana, LHV, MSU Mughal Family Health Clinic Ms Zehra, LHV Dr Aamina, Quality Assurance Doctor, Islamabad Nasir Mughal, Social Organizer, Islamabad Government representatives Choudhry Muhammad Khurram Gulfam, Member of Punjab Provincial Assembly Shahzad Malik, Chief, Population, Planning and Development Division Dr Aurangzeb, Deputy District Health Officer, Mardan Partner Organisations
Social Development Direct Document Title Pg 44
Sadia Atta Mehmood, National Programme Officer Youth, UNFPA Syed Ali Rizwan, Senior Programme Officer, Care International Faiza Riaz, Finance Officer, Care International Wajiha Ghias, Maternal and Newborn Health Research and Advocacy Fund Syed Ali Shah, Maternal and Newborn Health Research and Advocacy Fund Ihatsham Akram, Maternal and Newborn Health Research and Advocacy Fund Robina Ashraf, Pakistan Poverty Alleviation Find (PPAF) Qadeer Baig, Rutgers WPF Laila Naz, General Secretary, Saathi Foundation Media Maira Imran, Journalist, The News R-FPAP Youth Peer Educators (aged 15-24) Maryam Zia Misri Khan Quratulain Musaddiq Hussain Afsana Lal Raja Shahbaz Rafia Yaqub Tayyib Hussain Maya Yaqub Mardan Youth theatre group (aged 12-19) Gulrukh Ishrat Saba’un Israj Selu Jugnu Beena GBV/ Swara survivors Sidra Hameeda Begum Gulnza Afshaan Bibi Jannat Bibi Community Focus Groups32 10 unmarried female users 8 married female users 8 married female non-users (4 rural/ 4 urban) 8 married male non-users (4 rural/ 4 urban)
32
Given the sensitivity of the subject matter, we did not ask the focus group participants for their names so that they would be assured that their responses would be anonymous.
Social Development Direct Document Title Pg 45
ANNEX 3: Financial Management Assessment SUMMARY Financial management (FM) was one of the ‘key’ areas assessed during the evaluation in order to determine how efficiently the system is working to contribute towards higher value inputs, activities, outputs and ultimately outcomes and impacts. This is commonly known as the management approach to measuring VfM. This approach is concerned with internal management processes, as these reflect the ways in which organisations internally use the funding to ensure that value is maximised per £ spent33. The FM assessment tool is based on nine building blocks which include: i) organisational structure and human resource, ii) accounting system and reporting, iii) budgeting and planning systems, iv) purchases and payables, v) invoicing and receivables, vi) treasury functions, vii) payroll, viii) asset management system and ix) audit. The tool was applied during the evaluation to assess the extent to which each block is contributing towards delivering VfM services. The assessment found that the current financial management system in place is producing ‘medium VfM’. This means that most of the basic controls exist and are being used to a certain extent. However, the average overall score of 62% also demonstrates that there is a there is significant room for improvement. While specific improvements are required in each block, overall these cluster around capacity building, improving and building upon existing systems, introducing some new controls, and using an improved and more efficient accounting software. Improvements are also needed on financial management reporting and costing of services. R-FPAP also need to build in-house expertise on costing of services. Immediate management attention needs to be directed towards block 6 (treasury functions) which is currently categorised as ‘low VfM’. The concept of VfM is differently understood throughout the organisation and is mostly confined to delivering services at the lowest cost excluding the quality factor. More understanding needs to be built on linking inputs with results (linking financial inputs with non-financial outputs). Seven out of nine blocks are categorised under ‘medium VfM’ in this assessment. There are variations in score within the ‘medium VfM’ category which are captured in the scoring grid below. One block was assessed as ‘low VfM’ and only one block as ‘high VfM’. Most of the blocks in ‘medium VfM’ category can be moved to ‘high VfM’ category by developing a financial management review and reform program addressing the weaknesses identified in this evaluation. Colour coded arrows in the scoring grid indicate the ultimate objective of the reform programme through short, medium and long term strategies. A brief overview of strategies that can help in delivering ‘high VfM’ are provided in the detail review for each block.
33
Evaluation Manager, PPA and GPAF, Annex 12.
Social Development Direct Document Title Pg 46
SCORING GRID Low VfM Medium VfM less than between between between 50% 51% - 60% 61% - 70% 71% - 80%
Blocks 1: Organisation structure and human resource 2: Accounting and reporting 3: Budgeting and planning systems 4: Purchasing and payables 5: Invoicing and receivables 6: Treasury functions (cash and bank) 7: Payroll 8: Asset management system 9: Audit
High VfM more than 80%
x x x x x x x x Total
1
2
3
2
x 1
DETAILED REVIEW Block 1: Organisation structure and human resource
Score: 61%
Medium VfM
This block analyses basic organisational structure and human resource management functions. It does not provide an in-depth analysis of human resource management, but looks only at the basic minimum functions. The score indicates that minimum basic controls and procedures are in place and are also being practiced, although significant room for improvement exists. This requires further investment in developing the human resource (especially management cadre) through specialised training programmes. In order to further strengthen this important function the following actions are recommended:
Provide specialised training programmes to finance staff (e.g. GAAP, best practices etc.). Develop annual staff work plans with agreed performance objectives. Develop a policy for annual increments which are based on dual performance appraisals34. Offer more competitive pay packages and linking them with staff performance.
Block 2: Accounting System and Reporting
Score: 52%
Medium VfM
This block examines the current accounting and reporting system in place. This is a key system for analysing and comparing costs with benefits, and also for ensuring transparency and accountability of R-FPAP’s accounting transactions. The assessment suggests that minimum basic controls and procedures are in place. However, the score is low and there is a 48-point difference between the best and current score. This further suggests that considerable room for improvement exists in this block. R-FPAP should invest in procuring a comprehensive accounting software which is able to support modern accounting needs and generate better reporting, as well as providing online access to its programme offices. Budgeting and actual expenditure should also be integrated through the accounting software. Budgets are important in many ways but their importance as a benchmark for comparison against actual expenditure is the key tool that management can use to control 34
A dual performance appraisal system uses two discrete appraisal ratings to measure leaders'/workers’ performance; one rating is for their business result objectives and the other is for how well they execute their role.
Social Development Direct Document Title Pg 47
expenditures. In addition, there are serious concerns about the safety of financing data. The only backup procedure followed is to store financial data on different computers, but within the same office building. The current accounting manual was developed in 2006 and no revision has taken place since then. Surprisingly, no electronic copy of the finance manual was available within the organisation. The following specific actions are recommended for this block:
Conduct an immediate review of the existing financial accounting manual and update the manual where necessary. Train all finance related staff on the use of the updated manual and make the manual accessible to all relevant staff. Deploy an improved accounting system based on modern accounting needs. Ensure that all approved budgets are recorded in the accounting software and all subsequent expenditures are recorded against budget allocations using the accounting software. Ensure that allocating expenditure against the budget takes place at the lowest cost centres - which are the service delivery points. This will allow better costing and financial information to be provided to management on which to base decisionmaking. Develop policies and procedures on data backup to ensure that the data is stored securely and safely. Pre-number all vouchers so that they can be tracked. Undertake a periodic review of the trial balance in the head office at least once every two months, if not monthly. This will enable management to identify errors and issues at an early stage. Revise financial management reports, with consideration of how they could present more meaningful information e.g. comparison with the previous period, statistics on vacant positions in clinics, survival ratio etc.)
Block 3: Budgeting and Planning Systems
Score: 57%
Medium VfM
Financial planning is both a strategic and operational process linked to the achievement of objectives. It involves building both longer-term funding strategies and shorter-term budgets and forecasts, and lies at the heart of effective financial management. Block 3 examines how effectively budgeting and planning systems are functioning in the organisation. Financial planning doesn’t start with budgets and numbers but needs comprehensively laid down operational plans. The assessment suggests that minimum basic procedures for planning and budgeting exist and are being practised but these could be significantly improved to deliver better VfM results. R-FPAP undertakes both long-term strategic planning exercises and annual operational plans. The budgeting process is closely linked to the annual operational plans. Improvements need to be made in the budgeting process by including more detail, providing more clarity about calculations, developing a more structured budget preparation process through a budget manual, and training the relevant staff. The programme and finance managers both take an active part in the annual budget process. However, allocation of overheads is a weak area, particularly in terms of having an established procedure to recover management overheads from different projects. The impression from the current system is that overheads are being under recovered. The format and contents of the monthly financial management reports need to be improved to allow better decision-making. The following specific actions are recommended:
Develop a budget manual clearly outlining the roles & responsibilities, formation of the budget committee, reporting process, variance reporting and analysis.
Social Development Direct Document Title Pg 48
Put in place a policy to ensure that any revisions/changes in the budget are approved by the same body that approved the original budget. Develop a mechanism to allocate overheads to internal departments for central services used. These overheads should then be recovered as part of the project, where possible under respective departments. This can also be done by pricing core services and charging a price to each department (or each ‘A’) on the basis of services utilised. This area needs immediate attention from R-FPAP management. Review current variance reporting mechanism in terms of the content and methodology used.
Block 4: Purchases and Payables
Score: 77%
Medium VfM
Procedures for purchasing are needed to ensure that the organisation’s financial resources are always used for the correct purpose and that purchases made are for goods and services that provide the best possible value. Managing this block efficiently increases the chance of achieving good VfM including lower prices and increased transparency in procurement. This block attained a comparatively high score as compared to other blocks and is producing ‘Medium VfM’. Basic controls exist and are being practised with some advance level controls also in place and practice. A margin of improvement still exists and this module can be easily shifted from producing ‘Medium VfM’ to ‘High VfM’. Some of the issues can be addressed by improving policy, and others by simply improving the procurement documents. More specific rules on procuring services (e.g. technical assistance) are missing – procuring services is a specialised area and needs special procedures. Controls were assessed to be weak when a change is required in the original purchase order. Only changes which impact on the monetary value of the purchase order go through the same control as the initial purchase order. The following specific actions can help in gearing this block towards ‘high VfM’.
Train staff on the use of the purchase manual. Ensure that all purchase requests are initiated on a pre-numbered requisition form. Ensure that any change in purchase order is subject to the same controls as the initial purchase order. Develop procedures to identify missing invoices from suppliers. Develop procedures to identify outstanding payments.
Block 5: Invoicing and Receivables
Score: 68%
Medium VfM
The purpose of this block is to assess basic controls over invoicing and receivables and how they are contributing in order to deliver VfM. Proper and timely invoicing and charging appropriate fees all help in ensuring continuity in operations and performing timely activities. Basic controls are functioning well in this block and the addition of a few new procedures could make this block more efficient. A major issue which needs immediate management attention is related to establishing policies and procedures to charge user fees. Currently, this involves providing a price range to the Programme Management Offices and allowing them in consultation with local councils to set the user fees within the provided range. However there is no supporting evidence on how the range is developed. No policy exists for reviewing the charging of fees in subsequent years. The following specific actions can help in gearing this block towards ‘high VfM’.
Perform periodical ageing analysis at the head office level. Implement procedures to ensure that all receivables are followed up regularly. Develop appropriate mechanisms to charge fees and review them periodically. Charging of fees should be based on costing of the services. In order to address
Social Development Direct Document Title Pg 49
equity issues and provide social protection, the fee policy should be fully backed up by a comprehensive exemption policy. Block 6: Treasury Functions (Cash and Bank)
Score: 48%
Low VfM
This block examines the condition of controls on the treasury function related to cash and banking operations. The block obtained a score of 48%, which is the lowest score in all nine blocks assessed. It is producing the lowest VfM and some minimum basic controls are missing. Significant improvements need to be made by taking short- to long-term measures. This will require investment in terms of equipment (e.g. safe for cash storage), cash insurance cover, training of staff and developing certain check procedures. There is no current policy/ practice for testing cash controls on a periodic basis. This is important as cash is generated in almost all of the service delivery points. Clearer segregation of duties is needed. In one of the clinics visited the health care worker responsible for providing services for family planning was registering clients, entering records, preparing receipts, collecting cash and providing services. It is worth mentioning that a separate receptionist was available who was registering and collecting cash for other services. For good financial management, cash reserves are essential as there will always be times when grants are delayed or unexpected expenses occur. No practice exists to prepare monthly cashflow forecasts which are an important management tool to manage liquidity in the organisation. Procedures are also not in place for cash handover in the field. The following specific actions are recommended to gear this block towards ‘medium VfM’ and ultimately ‘high VfM’. Immediate actions: Ensure segregation of duties for handling cash and operations. For example, a statement of materiality can be developed for making sure that if the cash handling is more than £ XXX segregation of duties for cash handling is compulsory. Undertake periodic testing of cash controls especially in the field. Prepare cashflow forecasts in head office, regional office and programme management offices. Actions to be taken over medium term: Train staff on cash control and cashflow management in head office and in field offices. Protect cash by using a safe in a remote location. Insure sites with cash above a certain threshold. Introduce a rotation policy for staff involved in cash collection and handling. For petty cash handling in the field, ensure only one individual is authorised to handle the fund and laying down appropriate procedures for handing over the cash in absence of the authorised individual. Block 7: Payroll
Score: 78%
Medium VfM
A significant component of monthly expenditure is likely to be for staff salaries. It is thus important that these transactions are carefully managed according to best practice procedures understood by all members of staff. The overall score of this block is 78% which suggests that basic controls exist and are being practiced with some advance level controls also in place and practice. Further improvements can be made by enhancing capacity of the staff in select payroll procedures. Currently there is no system of charging core staff costs to different projects. Addressing this will help improve staff recovery costs from different projects and burden on the core funding will be reduced. The following specific actions can help in gearing this block towards ‘high VfM’.
Social Development Direct Document Title Pg 50
-
Train staff on payroll procedures as laid down in the manual. Develop a system to charge internal resources (experts) to different projects based on days/ time spent.
Block 8: Asset Management System
Score: 63%
Medium VfM
Fixed assets held in the form of land, buildings, vehicles, machinery and office equipment can represent considerable wealth. Although often overlooked, they require special attention to ensure their value is maintained and that they are not lost through lack of vigilance. This block assesses how the asset management system is safeguarding assets in R-FPAP and in turn contributing to improving the VfM of services provided. The block scored 63% making it a ‘medium VfM’ block. However, there is significant room for improvement. Most of the minimum basic controls are in place and practised. Some areas require immediate management attention. For example, there is currently no policy and system in place to ensure that all consumables are stored in a secure area. During one of the clinic visits, a small fire broke out as a result of an electrical fault. There were no fire extinguishers or similar items in place to deal with this. This is mainly due to lack of management attention and an appropriate policy. A fixed asset register is maintained using the accounting software, and we were shown a printout as evidence from the depreciation module in the accounting software. However, some important imformation was missing e.g. who is the custodian of the asset, what is the location, manufacturer warranty if any, etc. The following specific actions can help in gearing this block towards ‘high VfM’.
Provide training to staff on asset management. Implement an organisation-wide asset safety and security policy to ensure that all consumables are safely stored in a separate and secure area following protocols for safeguarding against theft, heat, water and other losses. Improve store room management protocols. Maintain a proper fixed asset register which includes information about the date of purchase, description, original purchase price, location, custodian, tracking number, accumulated depreciation, book value, supplier, maintenance contract and revaluation if any.
Block 9: Audit
Score: 83%
High VfM
Audit is important for organisations as they demonstrate a commitment to transparency and accountability and bring credibility to the organisation itself. It is also a legal requirement in most countries to have the financial statements reviewed by an independent auditor once a year. This block assesses briefly how the external and internal audit function is carried out. R-FPAP scored 83% for this block and is categorised under ‘high VfM’ band. Most of the good practices exist and are being followed. An internal audit function exists which reports to the governing body. The external auditor is a reputed chartered accountant firm. One qualified opinion was issued in last three years which is an area of concern.
Social Development Direct Document Title Pg 51
ANNEX 4: Methodology for Unit Costing and VfM analysis (Family Planning Services) Main Steps towards Unit Costing: Clinical Level It is important to note that this costing exercise was limited to only family planning services. A full costing approach was adopted whereby joint and non-joint costs were assigned to each cost objective35. i.
Assigning non-joint costs to each type of service that is to be costed. This involved assigning commodity costs and other medical supplies. For example, for an IUD insertion visit, this includes the cost of the IUD and other medical supplies which are directly related to the provision of service.
ii. Counting the number of visits made for each type of service during the study period. Details of service statistics were obtained from the clinics visited and M&E section in the organisation. iii. Measuring how much time a health worker is spending on each type of visit. Due to limitations of time and resources, this costing methodology used interviews to determine the amount of time spent by health workers on each visit. This is an inexpensive and easy to implement methodology with results available quickly. As it relies heavily on estimates by the health workers, these may not be accurate as workers have reasons for over-estimating, these estimates are further supplemented by observation during the clinical visit. iv. Assigning cost of health workers/practitioners (joint cost) to each service unit. This involved calculating the total time a health worker has spent on each type of service. A proportion of time spent on a specific service compared to the total time spent by the health worker was calculated. For example, the proportion of total time spent by a LHV on the first visit for oral contraceptives.
Row No.
1
2
3
Visit Type (a) Oral contraceptive – Initial visit Oral Contraceptive – Resupply visit Total
Health worker Time spent (Nurse) (b)
(%) spent
time
(c) (b1) divided by (b3)
Allocated salary (d) (c1) x total salary of nurse allocable to FP services36
Total number of visits (e)
Cost per visit (f) (d1) divided by (e1)
Sum of (b)
v. For capital items depreciation was calculated and allocated as ‘other joint costs’. Management policy was be used to determine the useful life of capital items.
35
Any particular item for which we want to calculate the cost. It was assumed that 50% of the health workers’ time is used in the provision of FP services, and so this calculation uses 50% of the total salary cost. 36
Social Development Direct Document Title Pg 52
vi. Assigning other joint costs to each service unit to be costed. In this methodology we have assumed that the major driver for cost is the time spent by health workers. Other joint costs were assigned to each service based on the time spent by all health workers on providing the service. For example, if 10% of the total time spent by health workers is for providing follow-up services for oral contraceptive clients, 10% of the other joint costs will be allocated to this particular service. vii. Allocating management overheads to clinical level. To determine a full cost at clinical level, overheads from regional/head office were also allocated to each service costed. This was calculated by dividing management costs by total costs (at head office/regional office). viii. Calculating the cost per visit. Cost per visit can now be calculated by adding all the costs obtained as above. The following table illustrates how this will be combined together.
Visit type Oral contraceptive – Initial visit
Cost of commodities per visit As calculated under (i)
Cost of health care staff per visit As calculated under (v)
Other nonjoint costs per visit As calculated under (vii)
Management charges per visit As calculated under (viii)
Total cost per visit
Calculating Cost per CYP CYP is the estimated protection provided by contraceptive methods during a one-year period, based upon the volume of all contraceptives sold or distributed free of charge to clients during that period37. Converting visit specific cost into cost per CYP we will be able to analyse costs during the time period of contraceptive protection. Type of Contraceptive Providing less than one year of protection Providing more than one year of protection
Calculation [Commodity cost of one cycle x CYP conversion factor] + [allocated personal cost of all visits in a year] + [other overheads] [Cost of all visits38] / CYP conversion factor
Steps for Organisational Level Unit Costs i. Country wide detail on service data including total number of clients for each type of cost objective was obtained from R-FPAP. ii. Based on the service estimates that were obtained through clinical level costing exercise, joint costs were allocated to each type of visit at organisational level data. Non-joint costs were calculated as for clinical level costing. iii. Cost per visits and cost per CYP were calculated as for clinical level. Steps for VfM Analysis i. Estimating CYPs. Using the data on number of FP commodities distributed, by each method; CYPs produced were estimated using USAID conversion factor These CYPs were adjusted for discontinuation rates (the number of women who will stop using a method of FP either during use or at the end of the contraceptive cycle) and switching rates (the number of women who switch from one FP method to another).
37 38
Cited on: http://www.usaid.gov/our_work/global_health/pop/techareas/cyp.html E.g. for IUD insertion: Insertion visit + Follow-up visits + Removal visits
Social Development Direct Document Title Pg 53
ii. Estimating Benefits. MSI impact estimator 1.2 was used to convert the CYPs into disability-adjusted life years (DALY). DALY is a measure of overall disease burden, expressed as the number of years lost due to ill-health, disability or early death. It "extends the concept of potential years of life lost due to premature death...to include equivalent years of ‘healthy’ life lost by virtue of being in states of poor health or disability”. By doing this a DALY is able to combine mortality and morbidity into a single common unit which is comparable39. MSI Impact Estimator was also used to calculate other health impacts averted including unintended pregnancies, unintended births, maternal deaths, abortions and infant deaths. Valuing Benefits. CYPs converted into DALYs were monetised using average per-capita national income of Pakistan. Benefits arising in the future (e.g. benefits derived from IUD insertion will arise for another 3.5 years) were also accounted for and discounted using a social discount rate of 10%. Following results were presented: -
Benefit Cost Ratio (return for every £ invested) Cost per DALY averted40 (impact)
39
Death and DALY estimates for 2004 by cause for WHO Member States: Persons, all ages. World Health Organisation, 2002. 40 The Commission on Macroeconomics & Health and the World Health Organisation recommend that an intervention is considered to be very cost-effective if the ratio does not exceed the average GDP per capita of the country.
Social Development Direct Document Title Pg 54
Social Development Direct Document Title Pg 55
ANNEX 5: Additional Tables from Organisational Level Costing Overall Cost Per CYP (GBP) Costs (GBP) With Private Without Private Practitioner Practitioner 228,239 228,239 174,713 158,814 338,290 338,290 133,423 130,562 874,665 855,904 511,972 268,062 £ 1.71 £3.19
Description Cost of salaries Cost of medicines, supplies and lab Cost of operations Management Cost Total Cost CYPs Cost per CYP
Overall Cost Per Visit (GBP) Costs (GBP) With Private Without Private Practitioner Practitioner New FollowNew FollowUser up User up 194,340 33,899 194,340 33,899 143,338 31,376 132,203 26,611 288,045 50,244 288,045 50,244 112,630 20,793 110,626 19,936 738,353 136,312 725,213 130,690 392,443 276,875 225,775 187,565 £1.88 £0.49 £3.21 £0.70
Description
Cost of salaries Cost of medicines, supplies and lab Cost of operations Management Cost Total Cost Total Visits Cost per Visit
Summary: Per visit Cost by Delivery Channel (GBP) Associated Clinic New Followuser up 0.8 0.2
Private Physicians New Follo user w-up -
Static Clinic
Mobile Clinic
New user 1.0
Follo w-up 0.2
New user 0.8
Follo w-up 0.2
Cost of medicines, supplies and lab Cost of operations
0.7
0.1
0.6
0.2
0.5
0.1
0.1
0.1
1.5
0.3
1.2
0.3
1.1
0.3
-
-
Management Cost
0.6
0.1
0.5
0.1
0.4
0.1
-
-
£3.75
£0.71
£3.14
£0.73
£2.80
£0.66
£0.07
£0.05
Description Cost of salaries
Total
Summary: Cost per CYP by Delivery Channel (GBP) Description Cost of salaries Cost of medicines, supplies and lab Cost of operations Management Cost Total
Static Clinic 0.6
Mobile Clinic 1.5
Associated Clinic 1.0
Private Physicians -
0.5
1.1
0.6
0.1
1.0
2.2
1.4
-
0.4
0.9
0.5
-
£2.44
£5.77
£3.53
£0.07
Social Development Direct Document Title Pg 56
Summary: Per visit Cost by Delivery Method (GBP) Description Cost of salaries
IUD New Follo user w-up 0.6 0.2
Cost of medicines, supplies and lab Cost of operations Management Cost Total
Injectables New Follo user w-up 0.5 0.1
Fmplant New Follo user w-up 0.6 0.3
Condom New Follo user w-up 0.1 0.0
Oral Pills New Follo user w-up 0.3 0.1
Sterilization New Follow user -up 1.8 0.4
0.3
0.1
0.4
0.1
9.0
0.1
0.2
0.2
0.1
0.1
4.5
0.1
0.9
0.4
0.8
0.2
0.9
0.4
0.1
0.0
0.5
0.2
2.6
0.6
0.3 £2.08
0.1 £0.77
0.3 £2.06
0.1 £0.45
1.9 £12.41
0.1 £0.93
0.1 £0.51
0.0 £0.29
0.2 £1.03
0.1 £0.38
1.6 £10.51
0.2 £1.21
Summary: Cost per CYP by Delivery Method (GBP) Description
IUD
Injectables
Fmplant
Condom
Oral Pills
Sterilization
Cost of salaries
0.42
1.48
0.47
9.80
5.91
0.22
Cost of medicines, supplies and lab Cost of operations
0.11
1.19
2.81
2.18
0.86
0.45
0.30
2.20
0.32
0.81
5.87
0.31
Management Cost
0.15
0.88
0.65
2.30
2.28
0.18
Total Cost per CYP
£0.98
£5.75
£4.24
£15.09
£14.92
£1.16
Social Development Direct Document Title Pg 57