NLV Summative Evaluation (2016) by The Power Is In People

Summative Evaluation

No Longer Vulnerable program (July 2012 - June 2015)

www.oxfam.org.au

Abbreviations

Executive Summary

Introduction and Purpose

Background

Report Structure

Approach, Methodology, Limitations and Validity

Approach

Methodology

Limitations

Validity

Background Analysis of â&#x20AC;&#x2DC;No Longer Vulnerableâ&#x20AC;&#x2122;

Context and objectives

Theory of Change

Theory of Action to Address Vulnerability

Broader Vulnerability Context

Progress Against Objectives

Impact

Relevance

Efficiency

Sustainability

Partnership

Participation

Gender and disability

Conclusions: Some Lessons and Recommendations

References

Appendices

Annexure One: NLV Evaluation

Annexure Two: FGD Guidelines

Annexure Three: MEL Framework

Credits

AIDS

Acquired Immune Deficiency Syndrome

CATCH

Caring, Affirming, and Training for Change

CBO

Community-Based Organisation

CLA

Cluster Level Association

CREATE

CBR Education and Training for Employment

CSO

Civil Society Organisation

DRR

Disaster Risk Reduction

ECD

Early Childhood Development

FGD

Focus Group Discussion

FSG

Farmer Support Group

HIV

Human Immunodeficiency Virus

JAW

Justice and Women

JCAS

Joint Country Analysis Strategy

KRCC

KwaZulu Regional Christian Council

LGBTI

Lesbian, Gay, Bisexual, Transgender and Intersex

MEL

Monitoring, evaluation and learning

MTR

Mid-Term Review

NDP

National Development Plan

NGO

Non-Governmental Organisation

NLV

No Longer Vulnerable Program

OAU

Oxfam Australia

OPUP

Operation Upgrade

OVSA

One Voice South Africa

PACSA

Pietermaritzburg Agency for Community Social Action

Project Empower

RAPCAN

Resources Aimed at the Prevention of Child Abuse and Neglect

RSS

Refugee Social Services

SHG

Self Help Group

STI

Sexually Transmitted Infection

TAC

Treatment Action Campaign

Tuberculosis

UNICEF

United Nations Childrenâ&#x20AC;&#x2122;s Fund

WASH

Water, Sanitation and Hygiene

WFP

Women on Farms Project 2

Introduction The No Longer Vulnerable Program (NLV) was initiated in July 2012 and ran until June 2015 with 42 partners across South Africa, specifically the provinces of KwaZulu-Natal, Limpopo, Eastern Cape and Western Cape. In October 2015, a summative evaluation of the program was commissioned to ensure that careful reflection and analysis across a broad range of projects, partners and beneficiaries was conducted. This report reflects the key issues and findings that emerged from a mixed-methods approach of document review, consultation, engagement and deliberation. The No Longer Vulnerable Program Framework, July 2012 â&#x20AC;&#x201C; June 2015, has guided the work, which encapsulates the strategic choices, theory of change, and aims and objectives of the strategy.

Approach and Method A combination of methods was used to do this, including ensuring familiarity with the underlying theory and its basis, a comprehensive review of program documentation, consultation and dialogue with program staff, partner organisations and beneficiaries through key informants interviews, roundtable discussions and focus group discussions (FGDs), and an assessment of other literature pertaining to the development context in South Africa.

Background Analysis The theory of change embedded within NLV explicitly embraces the context of vulnerability in South Africa. This enabled staff and partners to work at different scales so that lasting change could occur. An explicit recognition of this was made in the theory of change, which drew on Rao and Kelleherâ&#x20AC;&#x2122;s integral framework (2005). This framework explicitly identifies where changes must occur, ranging from cultural and institutional systems to changes at an individual level in terms of peopleâ&#x20AC;&#x2122;s attitudes and beliefs, or their access to and control over resources. Shackleton et al. (2015) note that there are different pathways for vulnerable groups and individuals to emerge from vulnerability that must take cognisance of different interest groups, cultures, values and places. Decisioncentred approaches or pathways, which focus more on processes of decision-making than outcomes, enable people to deal with uncertainty and address systemic drivers of vulnerability, rather than simply reduce risk. In essence, this is what NLV achieved. The NLV program has articulated such an approach. As stated in the formative evaluation, the program explicitly set out to enable change at community level through programming which was informed by guiding principles and activities relating to accountability, the disjuncture between policy and implementation and establishing an enabling environment for active citizenship. The response was designed to be integrated and adaptable, and drew on a range of approaches in order to effect change.

Progress Against Objectives An assessment was undertaken of progress against program objectives and an evaluation of the impact, relevance, efficiency, sustainability, partnership, participation and inclusion of gender and disability in the integrated approach. NLV program indicators were collected from the output summaries provided by partners. Standing back from these figures, it is striking how the reach of the program increased over the reporting periods, culminating in a high of 606,529 total beneficiaries reached in July to December 2014. There were consistently more females than males, usually around 60% of beneficiaries. Overall these figures provide important evidence about the impact of NLV, as they depict the range and number of activities as well as people reached. The roundtable discussions with representatives of half the partner organisations essentially reached consensus that the NLV program had been successful. This success had been achieved in two ways. Firstly, the support provided to partners has enabled them to achieve their own objectives, which aligned to that of NLV. Secondly, the theory of change significantly resonated with the ways partners operated in reality: changing consciousness at different levels to build individual and collective action to take on the structural underpinnings of vulnerability. Participants agreed that it was important to remember that when trying to enact change, organisations are dealing with conditions that have been in place over decades. With this in mind, roundtable participants agreed that NLV has had an initial impact. Overall, partners shared the sentiment of needing time to enable change: expectations of a three-year program needed to be tempered by the knowledge that three years is not a significantly long amount of time. People become less vulnerable over time because of the platform created by NLV through partners and community groups.

Relevance NLV was deemed to be relevant. Unanticipated events, political change, personnel moves, physical and technological shifts, inter-program and intra-program interactions, practitioner learning, media coverage, organisational imperatives, performance management innovations and so on, make programs permeable. The broadness of the thematic areas, and the nature of how OAU works in practice, means that there is flexibility and adaptability to address this reality. It also means that OAU is able to use this fluidity as an opportunity. At the core of this work is the notion of vulnerability in contemporary South Africa. The creation of participatory spaces has allowed communities to express disagreements, contest power and outcomes, and reach decisions. OAU recognised this explicitly in the learning emanating from the MTR. If structured in a manner that expands dialogue and enables new solutions to be created, contestation could contribute substantively to building the resilience of a community. Understanding the role of contestation in planning processes leads to a better understanding of how to develop peopleâ&#x20AC;&#x2122;s capacity to build resilience through collective action.

Responding to the Mid-Term Review The MTR report identified challenges in the design, implementation, management and achievement of NLV that needed to be addressed to meet the objectives and outcomes. Oxfam management responded formally to the recommendations of the MTR, defining a set of responses that have unfolded since September 2014. However, the ambitious operational plans of 2014/15 had to be downscaled and key activities reviewed due to the time constraints with OAU’s closure, and the changes resulted in a number of activities being cancelled. In responding, however, to the review findings, management acknowledged the value in OAU sharing lessons and raising awareness of the learning emerging from NLV. This would focus on OAU staff thinking and learning internally. Through a reflection on praxis – drawing on program work/implementation experience, engagement with partners and beneficiaries, and critical discussions and reflections with other Oxfam team members – the OAU team agreed to develop a deeper understanding of the conceptual underpinnings of the program framework and to document emergent concepts and theories.

Capacity-building The design and delivery of capacity-building programs that improved skills, knowledge and technical competency of partner staff was identified as a key mechanism to enable change. As NLV started to wind down, a strategic decision was made to ensure that a significant effort was made to underpin partner sustainability. This was in addition to the sustained commitment to building partner capacity and leadership throughout the program, the impact of which was aptly summarised by a partner who argued that OAU’s capacity-building and exchange programs were empowering and enabled partners to network and mix across disciplines (OAU Partners, 20151).

Efficiency OAU has clearly invested and used resources appropriately to ensure it has consistently and successfully built the capacity of its partners to address the complexity that is vulnerability in South Africa. It has used resources to support partners to effectively implement context-specific programming that was co-designed and highly valued by partners and communities alike. It enabled partners to link and learn collectively and engaged appropriate technical expertise both inside and outside of Oxfam. NLV was managed responsively; the large investment in organisational capacity-building support was central to strengthening the organisational architecture that supported and enabled partners’ work. This investment was also directed to partners to sustain them beyond the close of NLV and, indeed, the close of Oxfam Australia in South Africa.

OAU Partners, 2015, “The Seeds are germinating” Legacy and Lessons: Oxfam Australia in South Africa, Durban, KwaZulu-Natal.

Sustainability According to the NLV framework document, sustainability was to be ensured in two main ways. The first was contingent on the program’s contribution to the capacity of people to claim their rights, respond to external trends and shocks, influence those with power and to hold duty-bearers to account. The second was through organisational support to partner organisations, particularly through development of their strategies, improvement in organisational functioning and managing change. During the roundtable discussions with partners, more nuanced perspectives emerged about sustainability and the long-term impact of NLV. An important view emerging was that while the programs were successful, there was a challenge with the sustainability of the impact. This referred to the time required to change individual mindsets and to deepen consciousness about change, elicit action and build collective action. Without adequate material support in their lived realities, people sometimes returned to their communities and shifted back into a ‘survival mode’.

Partnership The focus of OAU on building partnerships and strengthening organisations it worked with was recognised in the roundtable discussions with partners as being key to the success of NLV. Apart from the transfer of resources, the emphasis on partners becoming self-reflective was identified as the source of the lasting impacts of NLV. This process was emulated in the communities in which partners worked to develop a level of self-reflexivity, “and in this way, allow them to capacitate themselves” (Durban FGD). This emphasis on individual and group consciousness was explicit in the Integral Framework, which was core to the NLV theory of change.

Participation Although focusing explicitly on participation, this remained a significant challenge, whether in a communal area under traditional leadership or because of one’s sexual orientation, gender or health status.

Gender and Disability NLV has a strong gender analysis. Work has been done internally with staff and externally with partners and other groups to develop the skills to recognise and respond to the need to place women more centrally as both actors in development and as primary beneficiaries of OAU’s work. This reflects the abiding concern of the program: unless the structural roots of vulnerability are addressed, any change may be temporary and illusionary. Gender work has been largely embedded throughout OAU’s work. The important learning identified was that women’s empowerment is often the key to addressing vulnerability, an argument that seemed to be increasingly accepted and recognised across programming from WASH, food and nutrition security, and smallholder farming, to disaster risk reduction and accessing services. The overarching strategy to empower women was through their participation in groups that build community solidarity to address social issues. Group interaction included sharing information, building trust and establishing alliances. The issue of disability has been a key component of NLV since its inception. The first report at the end of 2012 highlighted the decision to engage organisations to provide tailor-made capacity-building support on disability inclusion, with an early result that some partners developed programs that were both inclusive of people with disabilities and tailor-made to meet the local contexts. Work on inclusion of people with disabilities continued to increase across partner organisations through the duration of NLV.

Conclusions The strong and respectful partnership established between OAU and partners, and between partners, was evident throughout this evaluation. These relationships supported the efficient delivery of NLV and supported a focus on program delivery and collaboration for practical program improvement. It is also clear that the leveraging of partner knowledge, skills and resources has been extremely high, leading to innovation within NLV (including DRR, the use of the Integral Framework, and integration) with several partners forming collaborations outside of NLV. The capacity-building inputs have clearly cascaded to benefit communities, with partners bringing about a range of outcomes for communities, including changes in knowledge, practice and increased access to services. The indications that aspects of NLV will continue to be integral to partner organisations is high. The program design supports a high level of sustainability by using an integrated approach, which aimed to ensure partners continue to deliver services and support after the partnership with OAU has ended. Overall, the program performed well in supporting partners to ensure programs effectively targeted and reached vulnerable groups. This included disability inclusion and child protection. Comprehensive data provided clear evidence of the many benefits experienced by women, children and people living with disabilities across the NLV duration. Indeed, there is clear evidence to indicate that partners deeply valued NLV and have contextualised it to their programming contexts and communities. This is significant given the wide range of partner organisations and the willingness to embrace a new way of programming built upon integration. This also reflects how NLV was designed and the appropriateness of the approach in South Africa.

Background The Oxfam Australia (OAU) Country Office in South Africa, located in Durban, KwaZulu-Natal, has promoted and supported an integrated programming approach called the ‘No Longer Vulnerable’ Integrated Program (NLV). This has been pioneered as a new and effective way to enable positive change for the people with whom the organisation works. The approach has deliberately prioritised people’s access to a broad range of services, and the support that they need, at the shortest possible distance to where they live and work. Civil society organisations, supported by OAU, have delivered programs that improve health outcomes relating to HIV and AIDS, tuberculosis (TB) and water-related infections and diseases, increase and sustain food security and livelihoods options available to households, and increase and uphold access to social protection and socio-economic rights. The No Longer Vulnerable program was initiated in July 2012 until June 2015 with 42 partners across South Africa, specifically the provinces of KwaZulu-Natal, Limpopo, Eastern Cape and Western Cape. In October 2015, a summative evaluation was commissioned to ensure that careful reflection and analysis across a broad range of projects, partners and beneficiaries was conducted. This was to help OAU evaluate the overall impact of the program. This included identifying strategies and approaches that have worked, and which have not, and to make recommendations as to how the overall functioning of the program could have been better achieved. This report reflects the key issues and findings that emerged from a mixed methods approach of document review, consultation, engagement and deliberation. The No Longer Vulnerable program framework, July 2012 – June 2015, has guided the work, which encapsulates the strategic choices, theory of change, and aims and objectives of the strategy. The objectives of the evaluation were: 1. 2.

To assess progress against the program objectives (at OAU, partner and community levels), the relevance of the theory of change, and the partnership approach and model as supporting the advancement of these. To provide an assessment of the relevance of the OAU program, and assess progress against effectiveness, efficiency and sustainability – as it relates to the outcomes – and to the project’s outcomes in terms of sustainability and lessons learned.

Report Structure The report is structured as follows. • • • • •

The first section introduces the study, outlines the guiding questions and provides an overview of the research methodology. The second section outlines the approach of the summative evaluation, including the conceptual approach, methodology, limitations and validity. The third section provides a background analysis of the No Longer Vulnerable program by focusing on its key objectives, the theory of change and the theory of action, which were designed to respond to pertinent contextual challenges. These provide the basis on which to reflect on the impact. The fourth section considers the progress against the program objectives, and provides a reflection on the key questions of the evaluation pertaining to relevance, effectiveness, efficiency and sustainability of the integrated approach. In particular, it identifies the major learning emerging from the work. The final section draws the evaluation together through an elaboration on key learning and recommendations for the programs that might follow under other similar organisations.

The following section outlines the approach of the summative evaluation, including the conceptual approach, methodology, limitations and validity of the review. Approach A summative evaluation can be defined as “making explicit judgement about the worth of all or part of a program by collecting evidence to determine if acceptable standards have been met” (Case et al., 19882). In other words, a summative evaluation should be designed to present conclusions about the merit or worth of a program and, simply stated, to determine what the program achieved. As No Longer Vulnerable is reaching closure, the summative evaluation draws out the outcomes and lessons learned for future programs of a similar kind. The evaluation took into consideration the findings of the formative evaluation3 undertaken at the inception of the program, the mid-term review conducted in 2013, and an analysis of more recent program documentation emanating from programming in the past eighteen months. These documents were used to develop an analytical approach. This approach has been to test the theory of change that OAU defined at the outset of the ‘No Longer Vulnerable’ framework. This theory of change outlined how the program expected change to happen within a given context. In evaluating ‘realistically’, the evaluator has returned to the core theories about how the program was supposed to work and then interrogated them against the evidence of impact or change. A combination of methods was used to do this; these included: ensuring familiarity with the underlying theory and its basis; a comprehensive review of program documentation; consultation and dialogue with program staff, partner organisations and beneficiaries through key informants interviews; roundtable discussions and focus group discussions (FGDs); and an assessment of other literature pertaining to the development context in South Africa. These methods are elaborated on below.

Methodology Information for this evaluation was collected using a combination of methods, including appraisal of secondary material and a range of consultations through key informant interviews, focus group discussions (FGDs) and roundtable discussions with staff members, partner representatives and beneficiaries. Locally-based facilitators, using a common set of guidelines, ran the FGDs with the beneficiaries. This approach was designed to promote mutual learning and reflection as the evaluator and OAU staff, partners and beneficiaries jointly interrogated the framework. The evaluator took detailed notes during the meetings, which were supplemented by records of the discussion captured by a rapporteur. This approach was repeated with facilitation-run FGDs, whereby a rapporteur supported the facilitators conducting the discussions. These records provided the primary information on the evaluation. All of the material was collated to a produce a response to the objectives of the evaluation.

2 3

Case R., Andrews, M., and Werner, W. 1988. How can we do it? An evaluation training package for development educators. Research and Development in Global Studies, British Columbia. Drimie, S.2012. The Oxfam Australia Program Framework for South Africa Formative Evaluation, Oxfam Australia (internal report)

The scope of the work was broken down into three phases. Phase one was a review of the formative evaluation, the mid-term review (MTR) report and recommendations. This was done to identify key issues that required consideration in the last phase of the program implementation. In particular, emphasis was placed on how OAU responded to the key challenges identified in the MTR. Phase two consisted of a meta-analysis. A desk review was conducted of the program narrative reports and various monitoring, evaluation and learning (MEL) initiatives that illustrated, and gave evidence of the progress of the program during the three-year phase. These included OAU-produced case studies, ‘link-and-learn’ event reports, and program reflection meetings with staff and partners, with particular emphasis on the period January 2014 to September 2015. In addition, the learning papers and other documentation produced in response to the MTR recommendations were reviewed. These included a reflection on the ‘integral framework’, the program’s theory of change, OAU’s partnership model and a number of staff learning papers on integration, gender and contestation. Of particular importance were the data from ‘partner satisfaction surveys’, which were conducted from inception, to enable OAU to monitor its relationship with partners over time. Care was taken to draw out both qualitative and quantitative analysis from all of this existing material. A separate exercise was a detailed analysis of MEL data, as presented across the lifespan of NLV, in order to gauge changes over time. This quantitative analysis was undertaken in close conversation with the MEL officer based in Durban. Phase three consisted of primary research, primarily focused on roundtable discussions and FGDs with OAU staff, other OAU affiliate representatives, partners, beneficiaries, local community representatives and other key stakeholders. A selection of these informants was made, in collaboration with the evaluation core team, using a sampling framework and a schedule for the FGDs was agreed. Guidelines were developed for the FGDs, which were reviewed with OAU staff, and the facilitators selected to lead the beneficiary groups across several provinces. An informed consent form was also drafted for use with the beneficiaries. As mentioned above, the evaluator facilitated the two partner FGDs with support provided by an expert rapporteur to ensure the deliberations were recorded, as well as two FGDs with the OAU management team and program team respectively. In terms of selection, 22 partners were chosen out of a total of 42 that worked with OAU over the duration of the program to participate in the partner roundtable discussions. Of these representatives, eighteen organisations finally attended the FGDs. They were selected through the use of criteria applied across a matrix to ensure a wide coverage of organisations (see Annex One). In order to ensure a proportional spread across participants, the criteria included location, priority areas or themes, and whether partners were involved in the relatively recent MTR, as those which had not, were prioritised. Partners’ current workloads, in terms of OAU activities, were taken into account; partners with over five extra Oxfam activities (e.g. capacity-building activities) were not included. An in-depth approach was chosen for the beneficiary FGD work, rather than an attempt to achieve representation across thousands of individuals. The perspectives gleaned were complemented by a quantitative assessment using MEL data, as well as other more recent evaluator data, which drew on a series of surveys, intended to support the exit of OAU. This was to ensure validation across different sources of information and to triangulate these as best as possible.

Eleven beneficiary FGDs were conducted with small groups ranging from four to fifteen in each group. The 79 individuals engaged in the discussions with the FGD facilitators that lasted for about two hours. These groups were convened with the support of partner organisations, taking into account several selection criteria including geographical spread, whether the partner engaged was participating in another discussion, ‘level of access’, which referred to people over 18 years of age (many partners work with children which was not appropriate for this process), as well as distance from the partner office. Externally based facilitators, able to communicate in local languages, facilitated the beneficiary FGDs. As described, the facilitators were provided with guidelines and consent forms, which were jointly reviewed in a facilitated discussion. Where required, conversations were held with facilitators to ensure understanding of the objectives of the evaluation and the approach. This was to ensure that the method was universally applied in a way that allowed participants to share openly and honestly.

Limitations The analysis and recommendations are subject to potential bias emerging from the author’s own personal viewpoints, gaps in the literature that may have emerged in the initial consolidation of material and in what was pursued in terms of secondary material, and errors in the translation of data from the primary research to the summarisation in the review. A key limitation raised by OAU staff was the MEL system, which was systematically mined to identify the key outputs of NLV. Some senior staff expressed concern that the MEL system was not robust enough to collect the volume of data that was needed across all the outcome areas. Similarly, partners interpreted data collection differently and were not always consistent in reporting. An example of this is around ‘additional’ numbers versus actual numbers of people, receiving services in the six month periods. Many of these discrepancies were addressed through the programming period but inconsistencies likely remained. In addition, OAU missed an opportunity to collect data on some of the ‘management’ objectives and process indicators that would have provided real ‘evidence’ as to whether OAU’s approach to development was working or not. These indicators related to stronger civil society organisations, more sustainable community interventions, and a sustainable environment. In terms of the beneficiary FGDs, two key limitations emerged. Three facilitators conducted the discussions across very different locations in the country. Despite a common framework (see Annex Two) and training, inconsistencies may have emerged in how questions were interpreted and the manner in which the discussions were facilitated. Secondly, it was explicitly recognised that this was not a representative sample and therefore what emerged from beneficiaries could not be extrapolated beyond raising and illustrating key points and demonstrating views from particular places at a particular moment in time. Nonetheless the findings and subsequent recommendations were verified through a continuous process of validation through the key informant interviews, roundtable discussions and informal discussions with senior staff at OAU. The methodology was designed to provide a sequence of activities that allowed for identification of themes and issues and to corroborate emerging findings in a consultative manner. As discussed below, care was taken to ensure that the findings were validated.

Validity Apart from validating the findings through the phased approach and ongoing interactions with staff and partners, questions were asked throughout the evaluation about whether the basic plan emerging from the theory of change was sound, plausible, durable, practical and, above all, valid. In so doing, the underlying theory of change of the framework was verified and tested. The validity of findings and recommendations was also safeguarded through careful sampling across the evaluation. As mentioned, the meta-analysis drew on a review of all program documentation produced by OAU and partners over the last eighteen-month period, as well as a critical reflection on the MTR, which had drawn on earlier material. As such, it is argued that a comprehensive sample of documentation provided the material for this review. In terms of the two facilitated discussions conducted with OAU staff, most program coordinators and senior management attended to ensure that all major themes were discussed. Efforts were made to engage all participants in these discussions and to draw out opinions and impressions about key issues. Individual interviews and discussions were held with some staff members around specific questions. Similarly, two roundtable discussions were undertaken with partner organisations, including representatives of senior staff who were involved in decisions around the implementation of No Longer Vulnerable. The discussions were conducted in such a way as to enable participants to engage with each other and to collectively pursue particular themes or issues based on guiding questions. The opinions of the participants can therefore be seen as a credible reflection of the work and views of partner organisations. The partners invited to these discussions were involved in work that spanned all the major themes and focus areas that the program was designed to address. Relating to this, a wide spread of beneficiary groups were selected for the FGDs according to the methodology discussed.

This section provides a background analysis of the ‘No Longer Vulnerable’ Integrated Program by focusing on its key objectives as they relate to the context of South Africa and the strategy, or ‘theory of action’, that was articulated to give practical application to the program’s theory of change. In other words, the analysis sets out to understand how OAU understood vulnerability in South Africa, how change might happen in theory, and the strategy to execute this in practice. This analysis provides the basis on which to reflect on the impact of the program in the subsequent section. Context and objectives The overarching vision of the program was “less vulnerable people living and working in South Africa” particularly within the communities in which OAU partners worked, as a result of its effective implementation. In order to achieve this, three main objectives were identified for the program. Essentially, civil society organisations would be supported by OAU to deliver programs that will contribute to: 1. Improved health outcomes relating to HIV and AIDS, TB and water-related infections and diseases; 2. Increased and sustained food security and livelihoods options available to households; and 3. Increased and upheld access to social protection and rights. Two crosscutting issues were identified that would be crucial for attaining these objectives. Firstly, the programs undertaken would: employ sound power analyses that promoted women’s leadership; worked towards a broad awareness and promotion of women’s rights; and addressed the gendered impact of HIV, food security and climate change on women. Secondly, active citizenship would be promoted, in which the broader population of the country would be enabled to hold duty-bearers to account for the delivery of quality social services. Rather than framing these as advocacy and campaigns activities, the program would reflect its impact under the broad category of program work, known as influencing.

Theory of Change The MTR provides a clear statement and analysis about the theory of change embedded within NLV (20144). The complex context of vulnerability in South Africa and the stated objectives of the program required working in multiple spheres in order for lasting change to happen. An explicit recognition of this was made in the theory of change that lies at the heart of NLV, which drew on Rao and Kelleher’s integral framework (20055).

Women and men’s consciousness

Women and men’s access to resources

Norms and exclusionary practices

Formal institutions, laws & policies

Formal

inFormal

vidual Level Indi

Systemic Level

4 5

Drimie, S. 2014. Mid-Term Review (July 2012 - December 2013) No Longer Vulnerable Program (July 2012 - July 2015), www.oxfam.org.au Rao, A. and Kelleher, D. 2005. “Is There Life After Gender Mainstreaming?”, Gender at Work, www.oxfam.org.au

This framework explicitly identifies where changes must occur, ranging from cultural and institutional systems to changes at an individual level in terms of people’s attitudes and beliefs, or their access to and control over resources. Each of these is presented as a quadrant of a whole that should be considered holistically. The framework document provided an example of applying the theory of change using the quadrants: individuals within communities (rights holders) holding local government (duty-bearers) to account; improved access to services; effectiveness of governance through strengthened connection between rights holders and duty-bearers; and reduced inequalities through access of better services. According to OAU staff, interviewed throughout the evaluation, the implementation of the program was based on the organisation’s ‘theory of action’ that has emerged over time. Distilled into its basic form, this was to strengthen civil society organisations, through their work empowering communities, and through them change society. In thinking critically about how this could be achieved, OAU staff identified variants of the mechanisms inherent in specific programs as being key to the change process. This entailed leveraging different resources such as money, time, skills, networks, partnerships, and issue knowledge, to enable partners to work with and within the communities they served and to which they belonged. Concepts of learning, innovation and self-organisation were at the core of how OAU understood change to happen. The allocation of resources enabled partners to empower communities to lead the change, supported by learning, reflection, innovation and organisation. As such, OAU has endeavoured to be consistent in communicating with organisations and in the way it behaves towards them. Training, methodology, dissemination of information and dialogue are important features of OAU’s contribution to the growth and development of partners. OAU’s tools and processes lead to an understanding of issues that change norms and values.

Theory of Action to Address Vulnerability NLV posits that in order for development to occur and to be sustained, civil society organisations have to be supported to develop community development approaches that are high quality, strategic, adaptive and responsive to immediate community needs, while all of the time remaining focused on long-term gains. As articulated within the MTR, five main mechanisms were identified in which support would be provided to partners to reach the vision of “less vulnerable people living and working in South Africa”. These mechanisms were to: 1. 2. 3. 4. 5.

Provide program funding to support the partner organisations’ ongoing work; Design and deliver capacity-building programs that improve skills, knowledge and technical competency of partner staff; Commission and disseminate contextually relevant research; Create opportunities for networking and linkages with other like-minded organisations, nationally and internationally; and Document and disseminate partner and beneficiary experiences and learning.

Through these mechanisms, OAU worked with and strengthened the capacity of civil society to respond to development issues through their own empowerment, through the implementation of development initiatives, and through holding government accountable. The MTR highlighted a number of practical steps that characterised this approach: • • •

Step One – listen to partner needs and identify how to respond. Step Two – leverage funding, partnerships, networks and technical skills to implement an appropriate response. Step Three – secure entry points into communities using existing partnerships, where a sense of trust and durability in relationships had been built. 18

• • • •

Step Four – allocate required resources (skills, money, networks, partnerships, and issue knowledge) to partners based on an analysis of partner strengths and weaknesses. Step Five – adopt an integrated approach to partner and community needs, leveraging a range of resources to deal holistically with challenges. Part of the strategy was to acknowledge challenges and to turn these into opportunities (for example, building capacity to provide added value in partner organisations). Step Six – strengthen symbiotic relationship between partners including OAU to allow peer learning and review as well as the leveraging of partnership strengths. Step Seven – adopt an ongoing, innovative learning approach to allow for trial and error in meeting challenges.

Each of these steps reflects the plausible mechanisms through which the program framework operated. Thus, at the core of NLV lay an integrated programming approach informed by the theory of change and guided by the theory of action. In essence, the program has not prescribed a particular model for the delivery of these integrated programs, but rather promoted a holistic understanding of the interrelatedness of the stressors impacting on communities.

Broader Vulnerability Context Some key reflections of the broader South African development context are important to provide a backdrop to the evaluation and to consider the plausibility of the stated theory of change and theory of action. Although these reflections can in no way be seen to be comprehensive, some important recent perspectives are advanced to deepen the analysis of the issues that NLV has had to engage. In considering what change might be needed for a socially just future in South Africa, Shackleton et al. interrogate the entrenched nature of poverty and vulnerability and what is required to enable people to change their lives in the context of multiple stressors, in particular climate change (20156). Through a review of empirical case studies, the authors propose certain approaches that might be needed for poor and vulnerable people to escape the structural traps they find themselves in. Many people are at risk of being trapped as a result of deeply entrenched structural, discursive and social-institutional barriers that undermine specific and generic adaptive capacity (Adato et al, 20067). To give examples, various structural (e.g. legacy of apartheid), cognitive (e.g. forms of denial), normative and cultural (e.g. conformity, traditions, place), and behavioural barriers can keep people, or certain groups of people, in a trap. These can block people’s own ability to respond and undermine efforts around planned adaptation. Change is required to alter this state. This is essentially the fundamental alteration of the nature of a system, once the current ecological, social or economic conditions become untenable. It implies the conversion to a better, or more beneficial, state that can occur through changes in policies and governance; institutions; farming systems; locations or identity; value systems; regulatory, legislative or bureaucratic regimes; financial institutions; and technological or biological systems. For sustainable, long-term transformation, change may need to occur on multiple levels. Several options emerge as to how this change can be achieved. Key is an explicit focus on intersecting dimensions of inequalities, including gender, class, ethnicity, race, and age, to help identify what prevents certain people adapting, and why some people can. As argued, deep structural lock-ins hold poverty and vulnerability in place. Systemic change involving shifts in political discourses, the distribution of resources,

6 7

Shackleton, S; Ziervogel, G; Sallu, S; Gill, T and Tschakert, P. 2015. Why is socially-just climate change adaptation in sub-Saharan Africa so challenging? A review of barriers identified from empirical cases, WIREs Clim Change 2015. doi: 10.1002/wcc.335. Adato M, Carter MR, May J. 2006. Exploring poverty traps and social exclusion in South Africa using qualitative and quantitative data. Journal of Development Studies, 42:226–247.

largely those of government, and governance structures and a more pro-poor agenda are required. Citizen action, social movements and strong leadership, and political will are often needed for such reform. Complementing this, forms of social protection may be needed for the marginalised, where multiple barriers result in traps. Policy changes should be accompanied by support for local action and change. The approach should build on ‘seeds of change’ or innovations. Social learning and agency building is key to this. Shackleton et al. (2015) note that the complexity of adaptation itself, which involves multiple linkages and steps at different scales, leaves it subject to multiple hindrances. These require different pathways, catering for different interest groups, cultures, values and places. Decision-centred approaches or pathways, which focus more on processes of decision-making than outcomes, enable people to deal with uncertainty and address systemic drivers of vulnerability, rather than simply reduce risk. This all requires proactive adaptation that cycles between incremental and transformative action. The NLV program has articulated such an approach. As stated in the formative evaluation, the program explicitly set out to enable change at community level through programming which was informed by guiding principles and activities relating to accountability, the disjuncture between policy and implementation, and establishing an enabling environment for active citizenship (OAU, 2012a8). The response was designed to be integrated and adaptable, and drew on a range of approaches in order to effect change. Further, recognising that social change is inherently complex, as clearly articulated in the preceding analysis, change must happen in multiple spheres, and at multiple levels (OAU, 2012b9). These changes pertain to those defined in the Integral Framework. Put simply, supporting change required that programs were responsive to the broad range of issues that impact on people’s lives – both their material needs in the short-term and structural issues in the medium to long-term. This was contingent on its contribution to the capacity of people to claim their rights, respond to external trends and shocks, influence those with power, and to hold duty-bearers to account. For this reason, the program approach was that of being rights-based, participatory and building capacity. The theory of change and theory of action were largely targeted at strengthening local organisations. As identified in the NLV formative evaluation, efforts by external agents to boost grassroots capacity were unlikely to succeed if they sought to create new organisations or to impose resources and new approaches on existing associations (Friedman and Hudson, 201110). A core tenant of NLV was that in order to address these challenges successfully, civil society organisations would be compelled to be collaborative and act in a mutually reinforcing manner (Moolman, 201511). Thus, support was intended to build capacity of partner organisations to participate in change processes and concentrate on supporting opportunities to engage with change, rather than dispensing money. OAU’s position was that it could act as an ‘honest broker’ and command sufficient resources in the form of capacities as well as finances, to be able to offer adequate support. NLV was clearly well-conceptualised and well-positioned to enable vulnerable groups to respond to their situation and escape the traps that undermined efforts to change their lived reality. This pivoted on partnership with other civil society organisations operating at various levels, and an explicit strategy to elicit change through various spheres from the individual or personal, to the structural. The following sections draw out how this happened in practice, through NLV.

8 9 10 11

Oxfam Australia. 2012a. The Oxfam Australia Program Framework for South Africa Formative Evaluation, June 2012, Durban, KwaZulu-Natal. Oxfam Australia. 2012b. No Longer Vulnerable: The Oxfam Australia Program Framework for South Africa, July 2012 to June 2015, Durban, KwaZulu-Natal. Friedman, S and Hudson, J. 2011. “Filling the Gaps? The Future of CARE in South Africa”, Report on CARE’s Role in South Africa in the Light of Current Development Needs and Stakeholder Responses Commissioned by CARE South Africa. Moolman, A. 2015. An ecological approach to partnership, draft reflection document, November, OAU, Durban, South Africa

This section provides an assessment of progress against program objectives and an evaluation of the impact, relevance, efficiency, sustainability, partnership, participation and inclusion of gender and people living with disabilities in the integrated approach. Each of these evaluation domains has been elaborated on using a number of guiding questions to frame the enquiry. For each domain, the questions are posed and then interrogated using the evidence collected through the evaluation process. As discussed, this reflects a combination of different types of data, including quantitative data from MEL processes and qualitative data emerging from the roundtable discussions, FGDs and key informant interviews. Other information has been gleaned through the meta-analysis of program documentation, learning notes and similar material.

Impact The key questions guiding this section include: • • • • •

To what extent has the program achieved its objectives? How do the results achieved compare to the expected results? Do beneficiaries, including men and women, girls and boys and marginalised groups, including people living with disabilities, believe the project has improved their wellbeing? How do the results contribute to each of the outcomes of the NLV objectives? What was learnt about the implementation of the NLV program that might inform other program designs (or policy)?

Although these questions are addressed in sequence, there is some overlap in terms of the analysis.

Impact Against MEL Indicators NLV program indicators were collected from the output summaries provided by partners. It should be noted that these indicators were primarily related to outputs, as a direct consequence of the level of indicators that were set within the projects. The output indicators were merged into a single analytical framework to enable a detailed evaluation over the three years of implementation. This provided an opportunity to gauge how the program had fared against the indicator matrix proposed at the outset of the program. This indicator matrix is presented in Annex Three. It is important to note that these indicators were not set in stone but were intentionally flexible to respond to a changing context and partners’ needs. This reveals the learning ethos and commitment to partnership that permeated the program. The analysis therefore draws on the MEL quantitative data to examine changes occurring at an individual, organisational, community and societal level and, where possible, the figures and analysis date back to the start of the program. The analysis begins with a more detailed evaluation of the final reporting periods, which reflects the apex of most of the programming. This is followed by an analysis across the entire program to draw out trends over time.

As outlined in the initial program framework, the results correspond to the themes of health, water, sanitation and hygiene (WASH), social protection and rights, food security and disaster risk reduction (DRR), all of which were treated as being interconnected in the highly integrated approach of NLV. Another important theme was that of partners and their engagement with OAU and each other. Health Outcomes As to be expected of NLV, considering its genesis and emphasis on improving health outcomes, health continued to be a key focus of the majority of partners, with 68% reporting on this theme in June 2015, which was similar to the previous period (OAU, 2015b12). Overall these health indicators continued to decline from the previous period, apart from a slight increase in female condom distribution, with 2,000 more than the previous period and a marked continued increase in the number of HIV prevention programs and participants. The total number of home-based care visits conducted continued to decrease, as it has done since 2013. The prioritisation of interventions for those living with, and those affected by HIV and AIDS was a key foundation for decreasing household and community vulnerability. These interventions included the delivery of health services, as well as processes to enhance stakeholder engagement and active citizenry on health issues. An important contextual detail was that the majority of partners reporting on these indicators (seven partners) conducted both home-based care and voluntary counselling and testing (VCT) or HIV counselling and testing (HCT) sessions. Other partners reported conducting these on their own. This reinforces the deep understanding that distribution or access to female and male condoms was only one part of the prevention strategy. In terms of access to treatment, 76% of beneficiaries initiating antiretroviral treatment (ART) with support of partner organisations continued to be female (OAU, 2015c). The initiation of TB treatment resulted in slightly more males (52%) than females (48%). When comparing TB and ART treatment initiation over the last two years of the program (2013 – 2015), it can be noted that there were roughly 2000 more beneficiaries who initiated ART treatment with the support of partner organisations. The support of individuals to access and continue treatment became a priority of partners in the final reporting period given the “nationwide stock-outs of ART and other medication” (OAU, 2015c). As an example of this, Bela Bela (HIV and AIDS Prevention Group) emphasised treatment access, together with home-based care visits, which, in 2014, resulted in no TB patients defaulting on their medication and a “minimal loss” of HIV patients following up on their treatment (OAU, 2015c). The number of HIV prevention interventions “increased substantially in the last six months with 208 more being reported” (OAU, 2015c). As expected, the number of beneficiaries increased with almost 80,000 beneficiaries being involved – the majority of whom were female – from January to June 2015. This was led by Bela Bela, which reported 200 more HIV prevention interventions, reaching over 12,000 more beneficiaries in the 2015 period. WASH Outcomes The WASH Program (AACES) explicitly set out to integrate WASH across the NLV program, with specific attention on improving the understanding of, and programming around, various aspects of this work. Of the 38 partners who reported from January to June 2015, twelve provided results on the WASH indicators, reflecting over a quarter of all partners. A striking result captured in the final output report was that access to WASH services continued to increase overall in the period January to June 2015 (OAU, 2015c). Of this improvement, the increase in access to appropriate sanitation was most notable with 8,922 additional beneficiaries gaining access in 2015, doubling the total accumulation by the end of the period.

12 OAU. 2015b, No Longer Vulnerable Report for period January 2015 to June 2015, Durban, KwaZulu-Natal.

This increase can largely be attributed to One Voice and the Denis Hurley Centre, as they reported over 6,000 and over 2,000 additional people over this period respectively. The report notes that both these partners did not report on this figure in the previous two periods, meaning that it was unclear whether this was an accumulation over the last 18 months or last six months; the increase was likely related to the drive of One Voice to enrol schools in the program, with many learners making up the numbers. It should, however, be noted that enrolment does not necessarily result in access to appropriate sanitation so some caution is necessary with these figures. During the last six months One Voice continued to build on the dual issues of eliciting meaningful participation and engaging in influencing activities, especially in relation to WASH and Menstruation Management. In 2014, One Voice engaged with stakeholders during a menstruation management workshop so that they could better support female learners on this issue.Â In 2015, they continued conducting further technical trainings with school staff on WASH-related issues, such as the WASH basic principles, WASH infrastructure, and developing organically-grown school gardens and maintaining them. Similarly, the increased figure reported by the Denis Hurley Centre related to homeless people that they assist. Approximately 300 people attend the meal sessions and during these, some have access to bathrooms where they can wash. These food and shower sessions happen four days a week. Some people attend regularly but with migration into Durban during winter, this number increases towards the end of the January to June reporting period. For the first time in three years, the increase in access to appropriate sanitation was higher than gaining access to safe water and gaining access to hand washing facilities. Indeed, an additional 1,785 beneficiaries gained access to appropriate hand washing facilities with a further 750 additional people gaining access to safe water. Appropriate sanitation was clearly taking off as a key pillar of the WASH theme. Building on the previous periods, females continued to be the main beneficiaries of the increased access to water, sanitation and hand washing facilities. OAU have explicitly recognised that WASH-based programs that focus on infrastructure and access alone were unlikely to change the long-term situation facing vulnerable people (Roundtable Discussion, Program Coordinators13). As such, the NLV program ensured that decisionmaking, knowledge and awareness were also prioritised to meaningfully decrease vulnerability. The final output report noted that the number of additional women and vulnerable group members participating in decisionmaking relating to WASH was 481 during the 2015 period (OAU, 2015b). This built on 395 such beneficiaries in the previous reporting period, which, together with the remaining periods, culminated in 1,249 additional women and vulnerable group members participating in decision-making. Another important success was the number of beneficiaries with increased awareness of their rights (OAU, 2015c). Over 8,300 more people became aware of their rights, compared to the almost 1500 increase in the July to December 2014 period. This was over one-third of the total number of beneficiaries aware of their rights cumulatively over NLV, which was an important increase. One Voice reported over 6,000 people, and Sophakama reported just over 650. Local community-based organisations or communities actively participating in local government supported awareness of rights. Numbers of such entities actively participating in local government monitoring of these services increased after June 2014, when an additional twelve were added to a total of 68 over the duration of NLV. Indeed, the emphasis on this aspect of programming was clearly in the period July 2013 to June 2014, when 51 organisations were included. Looking back across the WASH aspects, emphasis on people living with disabilities, reported in a subsequent section, awareness and access to sanitation had become the main focus of this part of NLV.

13 Roundtable Discussion, OAU Partner Coordinators, Durban, 27 October 2015.

Food Security Outcomes At the outset of NLV, the food security focus was intended to be largely exploratory in order to build interventions in collaboration with partners and beneficiaries. This was, however, informed by previous work that had aligned food production, through household and community gardens, with expanding water access, sanitation and a linkage with the health work, particularly HIV and AIDS. After the health indicators, food security is the second most active component of NLV, with 59% of partners reporting on this in the final period (OAU, 2015c). From this reporting, and the details provided in partner reports, the emphasis on community and household gardens was clear. From January to June 2015, the number of community and household gardens increased at the same rate as the two previous 2014 periods (OAU, 2015c). Table 1 depicts the steady increase in numbers of household gardens from the July to December 2013 reporting period, to January to June 2015, with the accumulation calculated in the last column.

Table 1: Numbers of household gardens July to December 2013

1,249

household gardens

January to June 2014

2,788

household gardens

July to December 2014

2,008

household gardens

January to June 2015

2,096

household gardens

July 2013 to June 2015

8,141

household gardens

The large number of household gardens in early 2014 corresponds to the work of the Trust for Community Outreach and Education (TCOE) and Siyavuna, reporting 1,500 and 172 additional household gardens respectively. At this time, Biowatch reported another 658 household gardens. It is interesting to note that community gardens, although at a much lower rate of increase, also swelled in this period with 235 additional community gardens, compared to an average of 54 community gardens per remaining reporting period. OAU emphasised two productivity indicators, namely projects producing and selling surplus produce, and the number of households harvesting more than once a year, to signify the impact of these gardens. They were used throughout the OAU reporting framework. These are important indicators in that they give a sense of how participating households moved away from own consumption to selling surpluses, a significant challenge in South Africa considering the agrarian history of the country. The partners that reported surpluses tended to have similar number of projects throughout the reporting periods, including KwaMakhutha Community Resources Centre (KMCRC), Biowatch and Women on Farms (OAU, 2015c). This reflects the ability to establish surplusproducing gardens within projects as a norm and is testimony to long-term nature of sustained partnerships. A total of 7,588 households were harvesting more than once a year with a steady increase per reporting period, to culminate in an additional 2,435 in January to June 2015. Palabora drove this increase with 440 more households reported and TCOE adding in another 150 households. It is important to recognise that similar partners that had an emphasis on food security performed well across the indicators, including Biowatch, Isibani, KMCRC, Siyavuna, and Women on Farms. This related to technical training workshops and support for participants. A total of 63,247 people participated in technical training with a large addition in the first period of 2014 when 45,606 participants were trained. The last period reflected the highest results over the last two years with over 130 workshops and almost 2,000 participants. Female participants remained in the majority at 80% of participants in this period, which has been the case for the entire NLV program.

Social Protection and Rights Outcomes Reiterating the importance of social protection and rights within the NLV program, twenty partners reported on related indicators during the final period. These centred on accessing grants, legal documentation, and community engagement. Over the past 17 years, South Africa’s social grant program has evolved into one of the most comprehensive social protection systems in the developing world. Extensive research has been done on the impact of the social grant system, which has consistently shown that grants, particularly the child support grant, are well targeted at very poor households, and that they “have been central to poverty alleviation over the postapartheid years”, although they have had little effect on the overall inequality in the country (Ferreira, 201514). OAU’s strategic focus on social protection pivots on an understanding about the role of grants in diverse livelihood strategies in a context of high formal unemployment and increasing inequality. Securing social grants for people, or enabling them to do so through accessing legal documentation such as identity documents and birth certificates, remained key throughout NLV. Over the three-year period reported, a total of 18,999 beneficiaries received grants, as depicted in table 2. In a similar vein, birth certificates received and identity documents issued also decreased from highs in the first period: 5,629 out of a total of 7,209 birth certificates and 10,395 out of a total of 15,559 identity documents. The female/male ratio for these interventions has remained relatively constant with female beneficiaries in the majority: 10,865 female beneficiaries receiving grants (out of 18,999), 3,899 female beneficiaries receiving birth certificates (out of 7,209) and 6,694 female beneficiaries receiving identity documents.

Table 2: Number of new beneficiaries receiving grants per reporting period July to December 2013

January to June 2014

11,000

beneficiaries

3,388

beneficiaries

July to December 2014

2,588

beneficiaries

January to June 2015

1,849

beneficiaries

total period

18,999

beneficiaries

Disaster Risk Reduction Most of the various DRR indicators, such as fieldworkers trained and community members surveyed, continued to decline in the last reporting period, most likely due to program closure (OAU, 2015c). Of particular importance was the fact that eleven partners reported on the 27 DRR indicators, demonstrating how this program has become integrated into many of the partners’ ongoing work. It is also significant that of the many indicators reported on were relatively consistent in terms of numbers reached per period, demonstrating a sustained emphasis throughout the program. Interestingly, numbers of community meetings surged by 2,535 in January to June 2015 to add to a total of 3,026 meetings over the NLV program. This reflected the period of xenophobic violence in South Africa, and in particular Durban, where Refugee Social Services conducted sessions with community members taking refuge in shelters and camps. It is worth noting that community action plans created by government reached a total of 51 plans, with 35 being developed in the period July to December 2014. As these are likely to have a lasting impact along with the awareness-raising developed during community engagement, the strategic intent of the timing can be identified.

14 https://africacheck.org/factsheets/separating-myth-from-reality-a-guide-to-social-grants-in-south-africa/#sthash.6bJyrUih.dpuf

Stakeholder Engagement Stakeholder engagement has been a core mechanism for implementation of NLV in terms of building trust, understanding, partnerships and developing ongoing learning and reflection. This engagement has been measured through the number and types of dialogues, forums, meetings and other interactions that demonstrated important patterns across the entire NLV program. As stated in the final NLV Program Report, â&#x20AC;&#x153;it is these engagements that are the bedrock of partnership, government accountability, active citizenship, and community ownershipâ&#x20AC;? (OAU, 2015c). In addition, many of these indicators can be used to track progress from the inception of the program in July 2012. However, one cannot directly compare the number of meetings held to overall participants, as many partners would report on one of these indicators and not on others (OAU, 2015c). In terms of the health focus, education and awareness-raising meetings were consistently the type of meeting most held from July 2012 to June 2015. In total, 9,471 education and awareness-raising meetings were held over the duration of the program with 287,693 people attending. Of these, 195,404 were female. A total of 889 public community meetings occurred with over 53 such meetings taking place in the January to June 2015 period. A total of 815 meetings with government officials happened, with around 110 of these occurring per period, indicating a consistency throughout the program. Over the course of the program, 348 meetings of various types took place on WASH issues with 168 network meetings, 125 meetings with government and 55 meetings with Traditional Authorities. There is no data for numbers of people attending these meetings. No discernable variation occurred across the six periods making up the three years of NLV. For food security, education and awareness-raising meetings were the highest within each period, culminating in 63,247 people attending over the duration of the program, of which 41,860 were female. Meetings with government and communities remained relatively consistent over the entire period culminating in 235 and 250 meetings respectively. Meetings with Traditional Authorities peaked in early 2014 with 69 meetings but decreased after that to culminate in 158 such meetings over that period. In terms of social protection and access to rights, the number of education and awareness-raising meetings remained consistent over each reporting period, culminating in a total of 2,601 meetings. The number of community, government, and network meetings was also consistent, totalling 124 community meetings, 170 government meetings and 373 network meetings over the program. Over the last two years, network meetings remained the highest after education and awareness-raising meetings, with about 90 occurring every six months. Meetings with government peaked in early 2014 and have decreased slightly since then. Community meetings were highest at the end of 2013. Meetings with Traditional Authorities reached a peak of 26 in January to June 2014, and then dropped off to reach a total of 65 meetings. In terms of DRR, the number of education and awareness-raising meetings continued to decline in the final reporting period largely due to the particular duration of this aspect of the program. DRR was supported for two years between 2013 and 2014 with a reduced focus on two of the organisations that received funding in 2015, that of Project Empower (PE) and Refugee Social Services (RSS). A total of 1,125 education and awareness-raising meetings occurred in total. As mentioned above, the number of community dialogues skyrocketed in the last reporting period to 2535, driven by RSS in their response to the xenophobic attacks in Durban, which started in April 2015. Many of these dialogues took place over the last three months in the temporary shelters for foreign nationals that were set up, as well as in communities. In total, 7,983 community dialogues occurred over the duration of the program.

Overall Impact In terms of an overall picture of direct and indirect beneficiary numbers, table 3 provides an appraisal of each of the six month reports and gives an indication of the reach of NLV.

Table 3: Overall appraisal of impact (key indicators)

Period

January to June 2013

July to Dec 2013

January to June 2014

July to Dec 2014

January to June 2015

Total beneficiaries (men, women, children)

245,604

597,828

482,419

606,529

597,443

Direct beneficiaries

98,899

98,522

91,628

123,883

146,119

58,211

60,000

59,000

80,000

Young people (19-35 years age)

23,896

30,449

People living with disabilities â&#x20AC;&#x201C; direct beneficiaries

1,717

2,528

793

1,073

Indirect beneficiaries

498,929

383,897

514,901

473,560

Female beneficiaries

280,000

300,000

Young people (13-35 years age)

656

People living with disabilities â&#x20AC;&#x201C; indirect beneficiaries

1,075

1168

Female beneficiaries

OAU, 2015b; 2014b15; OAU, 2014a16; OAU, 2013a17; OAU, 2013a18 In January to June 2013, a total of 245,604 beneficiaries were reached, of which 146,119 were female (OAU, 2013a19). This report did not provide a breakdown between direct and indirect beneficiaries nor stipulate how many people living with disabilities were reached. The January to June 2013 figures were similar for the first report, reflecting on the last period in 2012, which stated that a total of 131,080 beneficiaries were reached, of which 74,450 were female (OAU, 2012c20). Standing back from these figures, it is striking how the reach of the program increased over the reporting periods, culminating in a high of 606,529 total beneficiaries reached in July to December 2014. There were consistently more females than males, usually around 60% of beneficiaries. Overall, these figures provide important evidence about the impact of NLV, as they depict the range and number of activities, as well as people reached. To gain deeper insight into the implications of these, FGDs were facilitated with partner organisations in order to draw out the nuances behind the numbers. These are discussed in the following section.

15 16 17 18 19 20

OAU, 2014b, No Longer Vulnerable Report for period July 2014 to December 2014, Durban, KwaZulu-Natal. OAU, 2014a, No Longer Vulnerable Report for period January 2014 to June 2014, Durban, KwaZulu-Natal. OAU, 2013b, No Longer Vulnerable Report for period July 2013 to December 2013, Durban, KwaZulu-Natal. OAU, 2013a, No Longer Vulnerable Report for period January 2013 to June 2013, Durban, KwaZulu-Natal. OAU, 2013a, No Longer Vulnerable Report for period January 2013 to June 2013, Durban, KwaZulu-Natal. OAU, 2012c, No Longer Vulnerable Report for period July 2012 to December 2012, Durban, KwaZulu-Natal.

Partners’ Views on Impact Interrogating the impact of NLV requires evaluating the process of change experienced by partner organisations and beneficiaries reflecting back to the theory of change. This is because strengthened CSOs deliver various programs that achieve health, livelihood and social protection outcomes, which empower communities, and through them, change society. In other words, the evaluation must consider how partner organisations have experienced this process in order to achieve the ultimate objectives. The roundtable discussions with representatives of half the partner organisations essentially reached consensus that the NLV program had been successful. This success had been achieved in two ways. Firstly, the support provided to partners has enabled them to achieve their own objectives, which aligned with that of NLV. This support related to the various mechanisms embedded in the NLV theory of action: program funding, capacity strengthening, relevant research, networking, linkages and partnerships, and facilitation of learning. In the words of a partner organisation representative, Thusanang, the organisation “has received wonderful support from Oxfam where the relationship has visibly supported the organisation’s own objectives. The support from Oxfam has been more than anyone could have asked for” (Roundtable Participant, Cape Town21). Another from Treatment Action Campaign (TAC) agreed, acknowledging that “our existing objectives were supported by Oxfam and there were quite a number of success stories” (Roundtable Participant, Cape Town). Secondly, the theory of change significantly resonated with the ways partners operated in reality: changing consciousness at different levels to build individual and collective action to take on the structural underpinnings of vulnerability. These aspects of OAU’s work combined in important ways: “Every area that Oxfam has being involved in has left a legacy” (Roundtable Participant, Durban22). This statement referred to both physical infrastructure such as “water tanks and gutters” and to “a certain level of ‘conscientising’ that has taken place” (Roundtable Participant, Durban). The reflections within the roundtable discussions provided important nuance to how impact has been achieved. An overriding argument was that the development challenges facing South Africa were so complex that a threeyear program could only begin a process. A representative of TCOE stated, “It cannot say absolutely that it has reached its goals through NLV. Development is long term. Transformation of lives happens over time and in the same way the contribution of Oxfam will be seen in time. The work will continue. Transformation takes time” (Roundtable Participant, Cape Town). In particular, this reflects the challenges around change associated with the seemingly intractable land (and food) question in South Africa. An important reflection on change emerged in the Durban discussion whereby a participant explained that “people cannot be switched on immediately” (FGD, Durban, KwaZulu-Natal) to understand and drive their own change. Often people’s immediate individual needs had to be met before moving on to discuss the larger community concerns or larger ideologies. Indeed, focusing on the political side of development was not possible or welcomed by some organisations and communities. As one respondent put it, “In the sector there is a service delivery ethos that is at odds with the development process and agenda required to address vulnerability in South Africa. Changing consciousness is different to alleviating circumstances right now, which is the immediate demand. Balancing larger ideology with the idea of specific service delivery is difficult and there is an uncomfortable tension between the two” (Roundtable Participant, Durban). An important view was that OAU had not adequately created the space to hold the tension of developing consciousness within partners and communities that lead to political interrogation to reveal the structural underpinnings of vulnerability. This view was modified with the recognition that three years was not enough time to reach that stage.

21 Roundtable Discussion, OAU Partner Representatives, Cape Town, 17 November 2015. 22 Roundtable Discussion, OAU Partner Representatives, Durban, 18 November 2015.

In discussing this issue, participants agreed that it was important to remember that when trying to enact change, organisations are dealing with conditions that have been in place for many decades. With this in mind, participants agreed in both roundtables that NLV has had an initial impact. Overall, partners shared the sentiment of needing time to enable change: expectations of a three-year program needed to be tempered with the knowledge that three years is not a significantly long amount of time. People become less vulnerable over time because of the platform created by NLV through partners and community groups. A representative of RAPCAN provided an important perspective about change when reflecting on their program on human rights and children’s rights: “It takes five to ten years for behaviour change in a system” (Roundtable Discussion, Cape Town). The platform created by NLV provided the beginning of a “long conversation” that needed to take place, such that people can understand and respond to their own situation (Roundtable Participant, Durban).

Beneficiaries’ Views on Impact The FGDs conducted with beneficiaries across the program, representing a variety of partner initiatives across the four provinces, reveal a great deal about what NLV has achieved in terms of impact. A selection of these views is presented below in a way that allows a range of beneficiaries’ voices to provide perspective about outputs. Beneficiary statements are presented verbatim with reference to the place where the discussion took place, to demonstrate the breadth of views. They are roughly clustered to reflect the four quadrants of the Integral Framework. Individual mindsets: “When I started with...[The partner] I was an introvert and now I am more of an extrovert. My personality has adjusted. The body map activity has brought self-awareness to me” (FGD, Pietermaritzburg, KwaZulu-Natal).

“The health care professionals’ mindset has changed” (about LGBTI) (FGD, Ceres, Western Cape).

“From the education they have received, people in the area now understand about being gay and they are accepting gay people” (FGD, Pietermaritzburg, KwaZulu-Natal).

Individual actions: “I had no house before but now I have a home for my children; my children also have a full uniform for school” (FGD, Eshowe, KwaZulu-Natal).

“My child told me that in the youth groups they learn about self-awareness, goals and future plans” (FGD, Eshowe, KwaZulu-Natal).

“The partner gives people an opportunity to learn to read and write. Some people never thought that they would ever learn to read and write again. People gained a lot of confidence. The writing skills enable them to sign documents, read letters, do home works and to use the ATM [automated teller machine]” (FGD, Indwedwe, KwaZulu-Natal).

Shared value and vision: “The program has touched and changed many lives, we now have focus and vision and know what we want to achieve in life” (FGD, Vhutshilo, Limpopo).

“From savings groups people build houses, buy furniture, educate their children, some get married from the money and do traditional ceremonies” (FGD, Melmoth, KwaZulu-Natal).

“Before the farmers used to plant in seasons but now they plant throughout the year and this gives them food all the time” (FGD, Umgababa, KwaZulu-Natal).

“The garden project is giving people food and sense of purpose in life, before we worked with... [The partner] we did not take gardening seriously” (FGD, Indwedwe, KwaZulu-Natal).

Shared actions and structures: “The making of water tanks is sustainable as the tanks are made of cement [and do not break], people cannot steal the tanks, there are no insects inside, the water is cold like fridge water and the tanks are made by women” (FGD, Eshowe, KwaZulu-Natal).

“There has been some noticeable change around the work the women and men have been involved in the past few years [including] through the savings programme women are now able to educate their children up to tertiary level, through the making of the water tanks children are now drinking clean water at school, in the garden people use compost instead of fertilizer which is the more healthier way to go and is called cheaper because you do not use money to buy and transport fertilizer, places that did not have water now have water, and women, men, children and youth now have stronger relationship” (FGD, Eshowe, KwaZulu-Natal).

“Since [the partner’s] involvement in the community, we have noticed that different community stakeholders are showing interest in our work, we started with one home one garden and then proceeded to use the identified land as a group. The land was given to the community to use for farming few years back by the municipality but we only started using it as a group of farmers in the beginning of this year” (FGD, Umgababa, KwaZulu-Natal).

“[The partner] responded to the water crisis in Nkanini [and helped form] a committee for the water crisis, the committee looked at research work to be done, advocacy work took place and engineers were also involved. Now there is water and boreholes in Nkanini” (FGD, Eshowe, KwaZulu-Natal).

“Relationships with government departments have been strengthened; there are regular meetings with different government departments in the community (Imbizo) organised by [The partner]” (FGD, KwaMakhutha, KwaZulu-Natal)

A scan across these quotations reveals that in the views of the FGD participants, change has happened and not just as depicted in MEL data, in terms of the ebb and flow of indicators. The lived realities of beneficiaries have also changed, albeit in different ways and concentrations. Some of the quotations reveal how incomplete the change may be, others how small changes in mindsets have lifted community activism to engage the structural underpinnings of vulnerability.

Relevance The questions guiding the analysis in this section include: • • • • •

How did the program adapt in response to changes in the local social and political context and the experience of partners? What were the success factors and challenges in ensuring advancement towards the program aim? How have the recommendations from the NLV mid-term review been implemented towards realisation of program aims and objectives? Was the integral framework theory of change relevant? Did it provide a framework to articulate how change happens within the South African context? Was OAU’s capacity-building program and approach relevant and did it bring the expected results at partner and program level?

Program Adaptation and Success Factors In a recent gathering attended by experienced practitioners, academics and rural strategists to advise on the design of a new rural development program in the Eastern Cape, a number of rural development practices, activities, strategies, and interventions were reviewed to draw out the successes and failures over the past 30 years (Bunce and Cousins, 201523). A key argument emanating from this meeting was that in many cases, interventions to revive agriculture in the former homelands have failed because they are not rooted in the local context and fail to understand the underlying complexities and challenges faced by communities, in all their diversity. For programs to succeed, they require participatory approaches characterised by social learning, community ownership and a grassroots approach that build on people’s capabilities and passions. This resonates with the NLV approach and the explicit intent to build from the grassroots in an adaptive and appropriate manner. A key observation in the MTR is worth repeating here. Unanticipated events, political change, personnel moves, physical and technological shifts, inter-program and intra-program interactions, practitioner learning, media coverage, organisational imperatives, performance management innovations and so on make programs permeable (MTR, 2014). The broadness of the thematic areas, and the nature of how OAU works in practice, means that there is flexibility and adaptability to address this reality. It also means that OAU is able to use this fluidity as an opportunity. At the core of this work is the notion of vulnerability in contemporary South Africa. As emphasised in the MTR, a number of examples of where integration has happened across OAU’s work have emerged from the grassroots, through careful community facilitation, social dialogue, diagnosis and learning (2014). These mostly arose from an iterative approach as projects unfolded, informed by the practice of partners and communities themselves. The rationale for this was that the multiple, overlapping issues facing communities cannot be addressed through silos. As a learning organisation, OAU has established institutional arrangements at local levels that stimulate experimentation, adaptation and learning. This reflects a core ethos of OAU in South Africa: to provide space for reflection and adaptation to enable programming to be responsive and for partners and communities to adapt to a changing context. The creation of participatory spaces has allowed communities to express disagreements, contest power and outcomes, and reach decisions. OAU recognised this explicitly in the learning emanating from the MTR. If structured in a manner that expands dialogue and enables new solutions to be created, contestation could 23 Bunce, B and Cousins, B., 2015. Communal Area Agriculture and Rural Development: A Consultation Workshop Held at the Donald Woods Foundation, Hobeni, September 15th - 17th, 2015, Institute for Poverty, Land and Agrarian Studies, University of the Western Cape

contribute substantively to building the resilience of a community. Understanding the role of contestation in planning processes, leads to a better understanding of how to develop people’s capacity to build resilience through collective action. As argued in the MTR, development is bound to be contentious and contested, particularly when a large section of the population is weakened by a daily preoccupation with the struggle for survival, in the most unequal country in the world that is crippled by high levels of poverty and unemployment. Despite this reality, however, ‘cookie cutter’ interventions are often meted out when dealing with service delivery and development across these communities. A general fault of planning is its tendency to romanticise the coherence of communities and, in doing so, underestimate their complexity. A key aspect of OAU’s work through NLV has been around building social cohesion and active citizenship. Beneficiaries themselves powerfully articulate these issues. “We meet every month to deal with personal issues, support and encourage each other. We even buy each other some things like dishes. When there is an issue in the community like water, we all unite and approach the Councillor to resolve the issue” (FGD, Melmoth, KwaZulu-Natal).

“We involve people from every background, we bring closer those that are isolated by visiting them in their homes. Sometimes people are very shy to talk and express themselves because of their background, we work at the level of everyone and try to reach to the most vulnerable. By being kind, loving and listening people start to open up and share their issues” (FGD, Indwedwe, KwaZulu-Natal).

The partner has had “conversations with the traditional leaders and community members around rape and murder cases, discussing some of the community beliefs that put the lives of the gay and lesbians at risk, there are certain things that make gay and lesbian not acceptable in the community like they are not allowed to get married or do some of the cultural traditions” (FGD, Pietermaritzburg, KwaZulu-Natal).

“The HIV/AIDS Support groups provide safe space to talk. People find some of the activities very beneficial to them, like checking-in, where people share about the time they were away from the group. In this way you can see the progress, you are able to follow up at home when there is a need. We have built strong relationships, people have talk partners, we talk comfortably with each other, and we even appreciate each other when we are together by giving each other hugs” (FGD, Melmoth, KwaZulu-Natal).

“Through self-help groups, relationships are strengthened and there is forgiveness among neighbours” (FGD, Eshowe, KwaZulu-Natal).

“The more I talked about my HIV status, the more I felt free. I felt I belonged. In the group there is one voice; we speak one language. Before in our clinics we used to see people being admitted in wheel chairs, others very sick in bed, but today because of ARVs people walk themselves in and out of the clinic” (FGD, Melmoth, KwaZulu-Natal).

The partner “brings together different organisations in the community and teaches them how to get funding, this is unifying the community” (FGD, Eshowe, KwaZulu-Natal).

“Young people are always busy with activities; they are now visible for positive things in the community. Schools appreciate the work that the youth are involved in. Youth find the drama activities useful to them as they find a way to tell their own stories through drama” (FGD, KwaMakhutha, KwaZulu-Natal).

“Young people are involved in life skills activities, counselling and motivation, they do home visits, art and drama. Young people have been mobilised to fight crime and drug abuse in houses that have been used for this kind of crime. There are no more hiding places for crime and drug abuse in the community” Arts and drama help to decrease teenage pregnancy” (FGD, KwaMakhutha, KwaZulu-Natal).

These quotations allude to examples of how active citizenship has facilitated change through social dialogue to building accountability within communities. This aspect of NLV reveals the more overt political work, whereby individual consciousness through collective action can be used to engage structural issues. Although not always explicit and well understood by partners, having a multi-pronged approach that can be targeted at national level, or within a particular community, has proven effective. In particular, given the urgency to simply survive in a very challenging environment, NLV has provided the opportunity to support different ways of creating dialogue within communities to build civil society. Active citizenship has a clear linkage to securing the rights of people. As revealed in the MTR and articulated again in the roundtable discussions, partners implemented increasingly complex strategies for securing rights, advocating for justice and advancing the voices of minorities (MTR, 2014). These transcended different levels from national level processes to attending ward committee meetings. Partners’ engagement of local politicians such as ward councillors often serves as an initial link between communities and local government. Communities that have been mobilised can better hold local officials to account for the quality of service delivery. Many aspects of this work are once again best articulated by the beneficiaries themselves. “Knowing your rights is becoming a reality as people know where to go to report when their rights are violated” (FGD, Eshowe, KwaZulu-Natal).

“The resource centre has opened an opportunity for most of us. We can now work together in harmony with different government departments in the community. We know the procedures for complaints. We have been empowered to know our rights, how to use them and we have gained confidence to speak” (FGD, KwaMakhutha, KwaZulu-Natal).

“We can also bring up issues when we are not happy about the services provided by the police, clinic or any other government department. We know the procedures to report poor service delivery. We know the procedures to report at the clinic or at the police station. Some of the cases we report involve standing for a long time in a queue just to collect medication, we know that it is not acceptable to stand in a long queue at the clinic just to collect medication, we report to the clinic manager and we get attended to. There is a clinic committee that we report to and together approach the clinic manager. The community policing forum also account to the people in the community” (FGD, KwaMakhutha, KwaZulu-Natal).

“People [now] know the consequences of committing gender base violence” (FGD, Eshowe, KwaZulu-Natal).

“In our community we do not have a voice around our services, our Councillor does not have an office in the area and is not responding to our issues. One day we had to take our issue to the Mayor after several attempts with the Councillor. We were looking for some information regarding bursaries and internships that the municipality was offering. The Mayor was very helpful to us and provided us with some assistance, we applied for the bursaries and internships” (FGD, Pietermaritzburg, KwaZulu-Natal).

The general impression gained from these quotations is that the work of NLV to link active citizenry with access to rights has yielded important results. Clearly the notion that citizens are not passive users of public services but active holders of fundamental rights can be realised in practice.

MTR Recommendations and Responses The MTR report indicated that the integrated approach had pioneered new and effective ways that were resulting in positive change for the people with whom OAU works (2014). The report indicated that integration was happening across the work through partners and communities with a number of examples and experiences detailed. The approach had allowed civil society partners to identify the links between factors underpinning social vulnerability and to prioritise these according to what beneficiaries articulate. This empowered partners and communities to begin engaging with the structural underpinning of vulnerability, such as holding local government to account for non-delivery and poor service delivery. The MTR report identified challenges in the design, implementation, management and achievement of NLV that needed to be addressed to meet the objectives and outcomes (OAU, 2014c24). Oxfam management responded formally to the recommendations of the MTR, defining a set of responses that unfolded from September 2014. This section appraises each of the recommendations, the stated response, and the progress. The analysis is drawn from a combination of the roundtable discussions with Oxfam staff and management in October 2015 and the report documenting progress against the management response. An important point is that due to the 2020 changes within the larger Oxfam body and the establishment of Oxfam South Africa, several opportunities for consolidating NLV had to be cancelled. These changes included the “resultant redundancy of the Country Director” and the “decision to close the OAU office and programs in March 2016” (OAU, 2015e25). Essentially this meant that the ambitious operational plans of 2014/15 were downscaled and key activities reviewed due to the time constraints and the changes which resulted in a number of activities being cancelled. In responding, however, to the review findings, management acknowledged the value in OAU sharing lessons and raising awareness of the learning emerging from NLV. This would focus on OAU staff thinking and learning internally. Through a reflection on praxis – drawing on program work/implementation experience, engagement with partners and beneficiaries, and critical discussions and reflections with other Oxfam team members – the OAU team agreed to develop a deeper understanding of the conceptual underpinnings of the program framework and to document emergent concepts and theories.

24 OAU. 2014c. Management Response to the South Africa Midterm Review of ‘No Longer Vulnerable’ Strategic Plan 2013 -2105, September, Durban, KwaZulu-Natal. 25 OAU. 2015e. No Longer Vulnerable Operational Plan 01 July 2015 to 31 March 2016, Durban, KwaZulu-Natal.

Five learning papers were subsequently developed by OAU staff with the broad themes of: • • • • •

The “Integral framework as a theory of change”, by Allan Moolman; “Integration as a way of working”, by Pumla Mabizela and Inger Harber; “Responsive HIV and AIDS programming”, by Francesca Alice; “Gender and vulnerability”, by Glenise Levendal and Renee Jeftha; and “Trust building and constructive contestation”, by Nicholas Molver and Wendell Westley.

These papers were intended to be broadly academic in nature in terms of drawing on theory, providing a clear argument and drawing on evidence to make a case around each issue. In turn, these would be edited to produce a set of plain language documents and media to support the dissemination of the learning. Integration as a way of working The key recommendation was to promote opportunities for conscious reflection amongst Oxfam staff to interrogate and reflect on the grassroots experiences of integration, more critically to derive a set of principles that might inform future work. OAU management acknowledged the complexity of the NLV program and the importance of more reflection amongst staff to develop thinking and learning internally. Management proposed to coordinate face-to-face events and to develop learning papers on broad concepts of integration, vulnerability, gender, and to look at contestation as a means of embracing this, with gender as a cross cutting theme. Learning papers were commissioned amongst OAU staff to conduct critical research and reflection whilst ensuring that the learning was accessible to encourage engagement with a wide range of practitioners. Progress has been made on the NLV learning pieces. In April 2015, the group of Oxfam writers participated in a facilitated writing retreat to finalise the structure and content of their individual writing pieces and think through a process of getting reviews on their work. Since the retreat, through the support of a consultant, the writers have been guided in various ways to work towards the completion of the papers. Many of these are near completion. Active citizenship The key recommendation was to articulate a multi-pronged approach targeted at national level, or within a particular community, to link organisations across a particular issue. A secondary recommendation was to strengthen the national civil society platform Awethu! to build solidarity in confronting development challenges in South Africa, and contribute to addressing the fragmented nature of many civil society attempts to hold the state and private sector to account. As a result of the NLV program coming to an end in March 2016, some initiatives had to either be downscaled, postponed or be cancelled. A linking and learning event intended to bring national perspectives together during a large, three-day event, focused on several areas, was re-designed as one-day provincial events. Twenty short stories or vignettes have been written and have been edited to be shared with partners, and also be included in the final publication celebrating OAUs work in South Africa including NLV. Similarly, the contribution towards AWETHU! was terminated, largely because of the closure of the program and the need for the team to shift focus, rather than engaging with a new area of work.

Gender The key recommendation was to continue to place women more centrally as both actors in development and as primary beneficiaries of Oxfam’s work. Part of this was to define, very clearly, how specificity and location influence, and contribute to, different forms of discrimination in the urban and rural contexts. OAU management proposed to host a series of ‘Gender Conversations’ internally and as part of a broader, ongoing learning strategy. Due to the resignation of the gender coordinator who was assigned this task, as a result of the program closure, this internal learning had to be scrapped from the plan. Embracing contestation The key recommendation was to create participatory spaces to enable communities to express disagreements, contest power and outcomes, and reach decisions. This implied ensuring that these spaces were structured to expand dialogue and enable the emergence of new solutions to contribute to building the strength of a community. This was to engage the role of contestation in planning processes for better understandings of how to develop people’s capacity to build resilience through collective action. The response from OAU management was to create spaces that allowed communities, partners and Oxfam to talk about contestation issues. The intention was to use these conversations to appreciate how to engage in these spaces and to assist partners to take up this challenge. The creation of ‘spaces’ for dialogues was, however, not done due to the closure of the program and an emphasis on ensuring the sustainability of existing processes. Nonetheless, the issue of contestation was both a subject of the vignettes and one of the major writing pieces, and as such, key lessons have been distilled. Embracing community complexity The key recommendation was to ensure that the experiences of OAU and partners embracing of complexity within communities was an explicit learning gained from the process, and was documented as such across the organisation. This was to include examples of both underestimating complexity and when it had been factored in. The holistic interventions modelled by the DRR/Urban work exemplified the approach of embracing complexity. As a result of OAU management, this became a subject of the vignettes. Health The key recommendation was to design a two-pronged strategy to establish and demonstrate an effective health system that simultaneously mobilised people whilst partnerships were built with government to demand an adequate and functioning service at district and local level. OAU management recognised the enormity of both this question and making it explicit in programming. As such, this became a key focus of one of the extended learning pieces. WASH The key recommendation was to explicitly integrate WASH training with a range of issues, as this had proven to be an effective mechanism to address vulnerability in a holistic manner. As this conformed to ongoing programming within NLV, this recommendation was carried through in full. This included the program evaluation and final documentation of the approaches from the program over three years.

Food security and livelihoods One of the key recommendations was to review and potentially challenge assumptions within the South African National Development Plan (NDP) around agricultural development and food security. However, due to the time constraints mentioned above, and the changes resulting in OAU closing NLV, engaging with the NDP was called off. Broader questions about livelihood work to explore such as ‘why is our situation like this?’ and ‘what is our capacity to create change?’ have been the subject of the vignettes. The emphasis has been on creating more holistic strategies for addressing vulnerability. DRR The key recommendation was to consolidate and build new opportunities to push the boundaries of disaster preparedness and risk reduction. This experience has informed the design of the new Urban DRR program, which has been integrated into the Oxfam humanitarian strategy for the Southern African region. Part of this work has been to consolidate the decentralisation approach of the National Disaster Management Centre under a formal memorandum of understanding between the Department of Cooperative Governance and Traditional Affairs (COGTA), and Oxfam Great Britain (OGB). Conceptual Framework to Guide Integration The key recommendation was to use the Integral Framework as a learning piece. As such, a paper has been prepared for the end of program, focused on what Oxfam has learnt about using the framework as the core of its theory of change. Vulnerability Similar to the issue above, the key recommendation was to use the experience of addressing vulnerability as a learning piece. A paper has been prepared for presentation at the end of the program which focuses on what Oxfam has learnt about the notion of vulnerability as a key learning, as well as an important resource generally.

Partners’ Use of the Integral Framework When asked whether they had come across the Integral Framework in working with OAU, a number of partners expressed a similar view to that identified in the MTR. Although staff and partners intuitively recognised the framework, it was mostly not an explicit part of programming. For example the representative of TCOE argued, “no reference has been made to the framework” (Roundtable Participant, Cape Town). Others, however, acknowledged that although not explicit it was “how we work and function in any case” and that OAU “developed the Integral Framework from what was seen and heard” and that “it has never been imposed” (Roundtable Participants, Cape Town). Similarly to the findings of the MTR, many partners believed a more conscious use of the framework would have strengthened their strategy, particularly in identifying what was missing to elicit change. Partners also acknowledged that OAU’s practice of linkages across partners meant that in many instances, all four quadrants were being realised across their work. Essentially, the framework helped partners view the whole system in which people are embedded and reinforced the recognition that by working only in one sphere of this system, change will not be achieved. When considering the practical application of the integral framework, respondents from partner organisations in both the Cape Town and Durban roundtables agreed that the emphasis on shifting mind-sets was reflected throughout their work and therefore underscored the relevance of the theory of change. When people

(community members or beneficiaries) first come to the program, they are often in a survivalist mode. As they start to become economically sustainable and their livelihoods strengthen, they start to develop aspirations that shift them to focus on leadership training, skills around engagement and participation, and business insight. This process was seen as a continuum; “Not everyone gets there, and [therefore] there was the need to have a model that was flexible enough to engage the full spectrum of people” understanding that “not everyone gets to the same level” but “become [less] or ‘no longer vulnerable’” having participated in the program (Roundtable Participant, Durban). A good example of this is demonstrated by Woza Moya beneficiaries describing the process and broader impact of their program: https://www.youtube.com/watch?v=DpxtpwUilEo An interesting analogy used to describe this process was that of scaffolding. Many partners expressed the importance of building scaffolding such that people could move up through different levels, each with distinct programmes and goals in mind. Several organisations, including Operation Upgrade of South Africa and Siyavuna, shared broad characterisations of this process (Roundtable Discussion, Durban). When people are first introduced to the organisation, they are often isolated and extremely vulnerable. The first step is to establish and involve them in community groups to create collective agency and strength. The second step is to develop a livelihoods impetus in these groups such as through a savings process or small-scale production, which helps them to be less economically vulnerable. The social component is of key importance as an individual’s sense and experience of what can be achieved together. Along with the focus on economic vulnerability, crosscutting issues like HIV and violence are also mitigated by group interaction. The experience of the Mazisize Self Help Group describes this experience as it unfolds in practice: https://www.youtube.com/watch?v=1zFM6n74KD8 A representative of Operation Upgrade argued “sometimes when engaging with communities, you can see an immediate change in the mind-sets of the people involved. Having seminars and talks and creating dialogues with participants which address topics like human rights, and child abuse offer immediate responses and shifts in mind-sets. In these seminars, it is important to explain and personalise larger topics like ‘children’s rights’ to people so these terms are not immediately dismissed. The immediate positive feedback after such engagement shows the conscientising processes” (Roundtable Discussion, Durban). This resonated with RSS which claimed that NLV had a “lasting impact especially in home-based child care”, as awareness and training had “shown dramatic results” in that children were no longer left alone, or allowed “to be victimised” (Roundtable Discussion, Durban). That being said, the group of partner representatives argued that the “conscientising process was a continuous one”, particularly as there was sometimes a high turnover of participants. There needed to be constant evaluation as an important part of the process. This reflective practice was built through both the training of beneficiary groups and their practice, with an emphasis on how they interacted being an important aspect. This interaction pivoted on being conscious about sharing ideas and experiences, learning from each other, reflecting on how things change and talking about burdens. This has been captured by Neil Prose, a paediatrician and dermatologist at Duke University in the United States of America, and Thirusha Naidu, a psychologist at University of KwaZulu-Natal, South Africa, who identified two methods that sustain the work of small groups. This is captured in the video clip below in a deliberation on circles of people using mindfulness (concentrating attention and energy on what is happening right now) and reflective practice (thinking carefully about feelings, words, actions and how these impact on others): https://www.youtube.com/watch?v=opC5dQehTpY&feature=youtu.be

Capacity-Building As mentioned under the section on OAU’s theory of action informing NLV, the design and delivery of capacitybuilding programs that improved skills, knowledge and technical competency of partner staff was identified as a key mechanism to enable change. As NLV started to wind down, a strategic decision was made to ensure that significant effort was made to underpin partner sustainability. This was in addition to the sustained commitment to building partner capacity and leadership throughout the program, the impact of which was aptly summarised by a partner who argued that OAU’s capacity-building and exchange programs were empowering and enabled partners to network and mix across disciplines (OAU Partners, 201526). Over 2014, several critical initiatives supporting partners and their work unfolded, consolidating the previous eighteen months of work. This work was often difficult due to the complexities of learning at individual and organisational levels. Capacity-building happens in a number of ways within the Oxfam partnership. Arguably, this has been one of the greatest strengths of this program: the adaptability and targeting of the support given to community-based organisations (OAU, 2014b). It was evident that capacity-building efforts shifted more towards organisational and leadership development processes during 2014 with a number of partners. These were key risk and sustainability issues for partners generally, but most especially as the closure of NLV became a reality. One example reflecting this work is that of Sophakama in the Eastern Cape, which was able to complete a revised strategy and draft fundraising strategy (OAU, 2014b). Other partners were less successful in following through such capacity-building opportunities, as management did not fully adopt and drive the various recommendations and action plans (OAU, 2014b). The most important aspect of capacity-building in the final phase was fundraising and resource mobilisation. As part of OAU’s exit and sustainability plan, Oxfam contracted a consultant to work with partners on resource mobilisation (OAU, 2015e). This entailed assessments and assistance with developing donor maps and capacitybuilding webinars focused on ‘looking after donors’ and ‘building a donor base’, as well as how to access Corporate Social Investments funding. Another webinar focused on consortiums, their strengths, and challenges and how they work, marketing tools and looking at risks associated with non-profit organisations. Oxfam also committed to hosting a series of webinars for partners in order to build their capacity in using social media and digital platforms to influence stakeholders, local communities, the general public, other civil society organisations, and government (OAU, 2015e). The webinars were designed to provide space for thinking through the uses and challenges emerging from using digital technology for influencing, which gave participants the opportunity to practice using these tools, and development of strategies for organisations to actively engage in digital influencing going forward. In strengthening partners from an organisational development perspective, some responded well to the opportunity provided by OAU in the final phase to build broad alliances that could replicate something of the NLV partnership model. Partners including Save the Children South Africa, Woza Moya, KwaZulu Regional Christian Council (KRCC), Justice and Women (JAW) and a few others discussed how they could continue working together post March 2016 (OAU, 2015b). As such, they began to develop joint funding proposals in order to ensure sustainability of their respective OAU funded programs and identified capacity-building requirements until 2016 to reinforce and ensure program continuity.

26 OAU Partners, 2015, “The Seeds are germinating” Legacy and Lessons: Oxfam Australia in South Africa, Durban, KwaZulu-Natal.

Capacity-building was also targeted at beneficiaries through the strengthened partner organisations. As an example, the Farmer Support Group provided capacity-building opportunities to women and girls to participate in various meetings and forums (OAU, 2015b). Through capacity development, communities were able to organise and elect a task team that represented the community if the need arose. It was reported that farmers now receive more information regarding government services and legislation on various environmental aspects (OAU, 2015b). Another important example of successful capacity-building already reported was that of the number of additional people living with disabilities accessing WASH services (OAU, 2015b). This was recognised as a positive result strengthened by the capacity-building work around disability inclusion done with the WASH partners.

Efficiency The questions guiding the analysis in this section include: • Did the program make efficient use of investment in OAU and the partner’s use of time and resources to achieve investment outcomes? • What actions did the program take to improve efficiency over time? Questions around efficient use of investments and investment outcomes are notoriously difficult to answer, as discussed during a roundtable discussion with senior management (Roundtable Discussion, Oxfam Management). A particular challenge is that measuring the social value of a program such as NLV conflates different roles: accounting to external stakeholders including various funders and the various partner organisations in South Africa, managing internal operations, and assessing societal impact. The latter is an elusive measurement, as it requires estimating how much good an action will bring in the future, relative to how much it will cost to implement now. When engaged on this issue, it became clear that OAU has not explicitly defined what is meant by ‘value for money’ within NLV. This was partially due to the complexity of the program, which represents a different programming approach that focuses on collaboration and learning, embedding cross-cutting issues and is implemented in different contexts, with a diversity of partners. Nonetheless it is important to reflect on whether the NLV approach was ‘value for money’ in the broadest sense of the term, particularly by considering issues such as the theory of change, the partnership approach, capacity-building, participation, and the design and implementation of contextually appropriate interventions. Put differently, ‘value for money’ can be defined as the ‘best use of resources to contribute to positive significant change in the lives of most vulnerable people’, as per the broader aims of NLV. In terms of offering ‘value for money’, NLV achieved several important indicators of value: Theory of Change: OAU develop a Theory of Change and a Theory of Action (as discussed) based on previous experience, extensive consultation, engagement with partner organisations and review of theoretical frameworks. It has reviewed and tested these approaches, assessing their validity. NLV staff and partners have been engaged in this process to ensure they have fully understood the approach and application. These measures have strengthened and revised the practical application of the theory of change through the theory of action to ensure they were appropriate and the resources used for implementation were relevant and necessary.

The Partnership Approach: OAU worked to coordinate and support NLV in ways that were effective by drawing on its organisational expertise such as facilitating partnerships, linking partners to each other and technical support, and bringing about a complementary effort, with diverse organisations and skill sets. This meant that existing skills and capacities enhanced the value of the work and created efficiencies as appropriate capacity was utilised. Creating linkages and facilitating learning also helped avoid duplication and created collaborative working relationships. Indeed, the strong and respectful partnership established between OAU and partners, and between partners, has supported the efficient delivery of NLV and supported a focus on program delivery and collaboration for practical program improvement. Capacity-building: OAU worked through local structures and institutions to support communities to undertake advocacy, and encourage wider community representation and participation, particularly of marginalised people. This supported the ability of local communities to manage services locally and enabled community structures to be more inclusive of vulnerable people or groups. MEL activities also built skills, expertise and capacity within communities about how to maintain, deliver and monitor services which can support the sustainability of results â&#x20AC;&#x201C; maximising the effectiveness of the NLV. In terms of the partnership approach, by bringing partners together to learn and share, their capacity was mutually built and duplication avoided. As partner capacity has been built in ways that are relevant and tailored to individual organisations, the work under NLV is likely to continue and be sustainable. Participation: OAU has systematically ensured that partners and communities have meaningfully engaged in every aspect of NLV. This includes engaging partners to reflect on the appropriateness of inputs provided under NLV with advice on how these should be effectively allocated. This drew on a major strength of partners, which had extensive experience in delivering projects at grassroots and thus brought practical knowledge about how NLV could achieve change. The resultant feedback was incorporated into the design, implementation and revision of the program. These participatory processes provided a way to enable priorities to be set, based on local needs, ensuring resources were invested in ways that were relevant and valuable to the ultimate beneficiaries and other people impacted by NLV. Monitoring, Evaluation, Learning and Research: Systems were put in place (MEL, ongoing and appropriate research, review and reflection) within OAU and partners to capture information to enable staff and partners to learn from and improve programming. The MEL processes enabled partners and communities to analyse and assess issues pertaining to NLV, determine their value according to their own perspective and identify ways in which NLV could be of benefit to them. This strengthened the approach by directly informing ongoing implementation. Selective research, although potentially costly, provided valuable information about what strategies were effective and what resources were required to bring about change.

Contextually appropriate interventions: OAU sought to understand and respond to different contexts across South Africa, to engage stakeholders appropriately in a targeted way, particularly in the design and implementation of activities. These activities were monitored in context, and adapted as necessary. Tailoring and adapting interventions in this way enhanced the likelihood of effective programming. Monitoring of context and being responsive to changes in context also helped avoid inefficiencies, particularly as programmes could be adapted. Working with partners that brought contextual knowledge and expertise in community engagement ensured relevance and effectiveness. Financial review: From reviewing the financial acquittal reports at each reporting period, and the financial reports of partner organisations across the period of NLV, it can be argued, without undertaking a formal financial audit, that OAU has used the NLV funding in an effective, transparent and responsible way. OAU ensured that processes were in place during the design and implementation of NLV to assess whether resources and inputs allocated to implement project activities were economical, fit for purpose, and feasible on an ongoing basis. This was through partner proposal appraisal, narrative and financial reporting, as well as annual planning. From these, and weighting Oxfam Australiaâ&#x20AC;&#x2122;s own auditing mechanisms, it can be argued that OAU demonstrated its fiduciary responsibility and overall stewardship of the funds to use the investment effectively. As the broader ratios of funding allocation was clearly evident across the reporting periods of NLV, every partner organisation would have been able to gauge the overall balance and raise questions with the majority funder, Oxfam Australia, if necessary. An immediate way to review the financial investment over time is to scan across the ratios of budget allocation to management and support, partner grants, capacity-building and administration. Of these, partner grants always reflected the highest ration, between 50 to 57%. This was followed by management and support, which ranged between 29 and 35%. Administration ranged between 4 and 14% and capacity-building between 5 and 11%. Clearly the emphasis was on programming through partners, a priority sustained through the duration of NLV. Using four categories for reflection, something can be said about investment outcomes. The funds clearly fitted strategically with what NLV set out to do. In other words, funds were allocated across the various themes cascading downwards to partner organisations to implement the evolving program that was monitored both financially and in terms of MEL outcome data. Secondly, the potential outcomes of NLV were measured through a range of indicators that covered issues such as capacity-building, inclusion of marginalised groups and individuals, and the delivery of development assets. Through the use of a broad mix of indicators that would underpin long-term impact, it could be assumed that the likely impact on peopleâ&#x20AC;&#x2122;s lives would be ensured. Thirdly, the financial reporting and reviews ensured some element of cost-savings and economic effects. Partners were supported in making sound financial decisions through the relationship with OAU, formalised through the reporting procedure. It can be assumed that elements of cost-saving, where appropriate, would have been ensured. Finally, risks associated with implementation were gauged throughout the reporting periods, both in terms of how individual partners were faring in a particular place, as well as a broader contextual analysis of the South African political economy. This analysis was clearly articulated across reports over time.

An attempt to gauge outcome efficiency relative to how much OAU spent over time cannot be answered with ease. A cost-effectiveness measurement that evaluates a project’s efficiency over its lifetime would have to argue that human value, such as the number of lives saved by providing clean water, has a monetary measurement. Clearly a large number of lives have changed for the better through NLV and a range of CSOs are better positioned to continue this work. It can thus be argued that OAU’s diligence in selecting the best partners to undertake this work, to align with the overarching program objectives, to ensure connections to consolidate impact, and to continually ensure learning and capacity-building, ultimately created a framework for effective use of funds.

Sustainability The questions guiding the analysis in this section include: • What is the sustainability strategy for the program and partners, and what has been the progress against this? • What actions did OAU take to ensure that the program outcomes are sustainable? • How sustainable will the program results be? In the final output summary for the period January to June 2015, it was apparent that overall, indicators decreased from the previous reporting period, July to December 2014 (OAU, 2015c). As the momentum shifted towards closure, it became evident that programmatic work was not the same as in previous reporting periods. However, a few areas did demonstrate an increase, including the number of HIV prevention programs, access to appropriate sanitation and certain food security trends (OAU, 2015c). In particular, stakeholder engagement, measured through the number and types of meetings, demonstrated important patterns across the entire program, as discussed previously. Clearly the January announcement of the program closure led to a phase of consolidation by partners, as they sought to strengthen and sustain impact. Sustaining impact and ensuring the legacy of NLV was more than a consolidation of existing programming in the final reporting period, however. According to the NLV framework document, sustainability was to be ensured in two main ways. The first was contingent on the program’s contribution to the capacity of people to claim their rights, respond to external trends and shocks, influence those with power and to hold duty-bearers to account. The second was through organisational support to partner organisations, particularly through development of their strategies, improvement in organisational functioning and managing change. The roundtable discussions in Cape Town and Durban focused on the sustainability of NLV and, to an extent, identified these aspects without prompting. A representative of RAPCAN asked the rhetorical question of the Cape Town group, has NLV “provided an empowering experience” for partners and “will they exist after Oxfam’s involvement?” (Roundtable Participant, Cape Town). The group generally felt that engagement with OAU through NLV had been empowering, but cautioned that it was not possible to say whether a partner would persist after the closure of the program as so many factors were at play. It raised an important issue around the internal vulnerability of partner organisations and whether OAU had tried to address these. Many partners shared their experiences about how OAU had strengthened their own internal systems, which had contributed to their own sustainability. It was felt that “having someone like Oxfam to explain ‘why?’ you need certain things as an organisation” was very helpful (Roundtable Discussion, Cape Town). Similarly another participant felt that “systems have been looked at all the time by Oxfam which strengthened the organisation… Oxfam modernised how we work with people” (Roundtable Discussion, Cape Town). This partly reflects OAU’s ethos of capacitating leadership as an important element of sustainability. For example, OPUP stated that when their director died, “Oxfam brought in someone to do capacity-building with the organisation to ensure the sustainability...as it seemed unlikely they would continue at that point” (Roundtable Discussion, Durban). This 44

also related to the issue of succession planning, which entailed preparing people to take over from current leaders and focusing on how organisations would function in the long term. Partners expressed how important this had been in the discussions. Although the support of partners was strongly commended, it was also recognised that the onus was still on partners to sustain their work. The representative of TCOE stated strongly “it was not the responsibility of Oxfam to look after strategies to be incorporated into a program” and that the “vulnerability of organisations has to do with funding” (Roundtable Discussion, Cape Town). This raises the question of the exit strategy and how the question of sourcing other funding and sustainability was handled. Once again, a strong argument was made about partners’ own responsibility for their sustainability: “an organisation cannot be reliant on one major funder only” (Roundtable Discussion, Cape Town). OAU’s exit strategy was accompanied by careful monitoring of the process through a mood survey to gauge how partners were managing the closure of NLV. Using the monitoring visits in early 2015, responses received from 33 partners confirmed, “overall the partnership shows a high level of resilience in the face of this change” (OAU, 2015d27). As expected, a number of partners voiced their concerns, uncertainty, and frustrations with key issues, including spend for next financial year, funding, staff (both within their organisations and Oxfam staff), the direction and approach of Oxfam South Africa (OZA), and their own internal organisational changes. However, the majority talked about their appreciation of the Oxfam partnership and feeling a sense of loss that it will be ending. Importantly for the sustainability question, the majority of partners were confident that they will adapt to the closure and many had already indicated that they have taken concrete steps forward (OAU, 2015d). These included the difficult decisions of “potential retrenchments, project closures, and in some cases even office closures” (OAU, 2015d). The confidence about the future emanated from having experienced donor exits before: changes in funding are a part of the civil society working environment resulting in a high level of dynamism and flexibility inherent in many individuals and structures (OAU, 2015c). During the roundtable discussions with partners, more nuanced perspectives emerged about sustainability and the longer term impact of NLV. An important view emerging from Durban was that while the programs were successful, there was a challenge with the sustainability of the impact (Roundtable Discussion, Durban). This referred to the time required to change individual mindsets and to deepen consciousness about change, elicit action and build collective action. Without adequate material support in their lived realities, people sometimes returned to their communities and shifted back into a ‘survival mode’. Essentially, people who attend capacitybuilding or stakeholder engagement processes require continued, sustained support from partner organisations. In the words of one participant, “it was not about the numbers of people passing through [training] but [about] people who have passed through the organisation needing to have continued support” (Roundtable Participant, Durban). Put differently, continued relationships between beneficiaries and partner organisations need to be maintained if impact was to be sustained. This reiterates the importance of strengthening livelihoods, whether directly through new skills, or indirectly through social protection or rights, such that once the immediate interaction with the partner has ended, people are able to sustain their lives. However, participants felt far more was required than just supporting livelihoods. Often people cannot live off the skills that they have developed without some fundamental shift in the structural underpinnings of their vulnerability. This relates back to the concept of ‘traps’ discussed earlier as a major characteristic of the South African development context. It also relates back to the theory of change embedded within NLV: that change must occur in all quadrants depicted in the Integral Framework; it cannot be assumed that change in one area, such as the formal arena of policy reform at the national level, necessarily results in improvements in the ability of individuals to access infrastructure, services and knowledge. 27 OAU Australia (South Africa), 2015d, OAU Program Closure in SA: Where are we at? June, Durban, KwaZulu-Natal.

Another view on this issue was the importance of recognising the different roles of partners and, indeed, their own material situation. This highlighted “people’s own busy lives and own survival strategies” within partner organisations, which means some issues get pushed aside for more immediate concerns (Roundtable Participant, Durban). Some partners were able to engage material deprivation without having enough capacity to adequately engage the other quadrants, requiring partnerships to take this on. The argument acknowledged the need to create platforms for people and government to engage and the importance of mobilising people to fight for their rights; there was also emphasis placed on the fact that partners were set up differently and had different strengths and goals. This underscored the importance of building partnerships across organisations to ensure that a holistic approach (across the quadrants) was being facilitated. The Cape Town group agreed that “when Oxfam withdraws it will be leaving a legacy” but when it does, the strength of the partnerships will inform how sustainable this impact is. Thinking retrospectively, partners in the Durban roundtable considered whether building specific sets of partnership at the outset or early in NLV would have been better for sustainability – in other words, groups of partners coming together early in the process to negotiate joint program objectives and indicators and in so doing establish a team. In this thinking, early collaboration would have led to a better understanding of the final goal of NLV, as partners would have been able to recognise the strengths across a range of organisations. As one partner representative put it, “awareness in the beginning of this process would have been extremely significant in terms of having an even longer, sustainable impact” (Roundtable Discussion, Durban). This does not, however, take account of the adaptive or incremental nature of the NLV program, and how partnerships were built over time as needs arose. It would have been difficult to set the agenda at the outset. A final point on the issue of sustainability from the partner perspective was that of new funding streams and new relationships with donors. The approach of OAU to work with partners’ own agendas was regarded as exceptional by many partners, as donors were often seen to try to influence organisations to adopt the donor’s agendas, causing the organisations to lose sight of their own goals. Organisations should engage with funders when this is the case to make sure that the goals of the organisation are met. This was a clear message conveyed by OAU during the exit process, particularly during resource mobilisation training. Partners acknowledged the training or engagement with up-skilling in terms of fundraising that had been provided by OAU. This was extended to building websites to allow exposure to the international community and seeking a wide network to enable different opportunities for funding. A challenge with the training had been the retention of the people who had participated, which was a result of the lack of job security, linking back to the issue of sustainability. Nonetheless, the training had been beneficial and, as stated in the mood survey, the majority of partners were confident that they would adapt to the closure (OAU, 2015d). As the question of sustainability has such profound implications for the legacy of NLV, it is appropriate to draw out some key insights that emerged from beneficiaries themselves on this issue. “We want awareness and education of the work that has been done to continue and increase. The people that have got information and have been trained need to go to the community stakeholders and do more workshops. We need follow up workshops in what has been done, frequent meetings and allocation of coordinating leaders from those that have been trained to make sure that the work continues. We need to form community committees so that our voices can be heard” (FGD, Eshowe, KwaZulu-Natal).

“We see that our work has shaken the community, it has great results and the community supports us in buying our products, the community give us a good price for our products and they appreciate the products” (FGD, Umgababa, KwaZulu-Natal).

“As a minority group we would like the changes to continue, but unfortunately without any assistance from outside nothing can continue, because mainly due to funding and support by service providers” (FGD, Ceres, Western Cape).

“And due to the fact that [the partner] is also currently struggling to obtain funding there is no funds or commitment regarding sustainability” (FGD, Ceres, Western Cape).

There are clearly mixed impressions, some relating to ongoing funding to sustain change, others confident that impact would continue because of how NLV had ‘shaken’ up structures.

Partnership In exploring the partnership model and how this played out in programming, two key questions emerged: • •

How have the program’s partnership model and approach supported progress towards the program aim and objectives? Did partnerships (between OAU, partner organisations) progress in line with expectations and good partnership practice? What worked well, or not so well?

The OAU partnership approach intended to: 1. 2. 3.

Strengthen community development approaches with a focus on supporting civil society organisations (both formal and informal groups of people); Support the sustainable delivery of, and increased community participation in, integrated development programs; and Create and sustain enabling environments with a focus on communities of vulnerable people.

The focus of OAU, on building partnerships and strengthening organisations it worked with, was recognised in the roundtable discussions with partners as being key to the success of NLV (Cape Town and Durban Roundtable Discussions). Apart from the transfer of resources, the emphasis on partners becoming self-reflective was identified as the source of the lasting impacts of NLV. This process was emulated in the communities in which partners worked to develop a level of self-reflexivity, “and in this way, allow them to capacitate themselves” (Durban FGD). As will be discussed, this emphasis on individual and group consciousness was explicit in the Integral Framework, which was core to the NLV theory of change. OAU administered partner satisfaction surveys in 2013 and 2015, which provided partners with a confidential mechanism to report back on how they perceived OAU was doing. According to partners, OAU “consistently excelled in its partnership approach” throughout the program (OAU, 2015a28). Across the two surveys conducted in 2013 and 2015, over 80% of respondents on average chose the highest category for these indicators.

28 OAU, 2015a, No Longer Vulnerable Partnership Satisfaction Learning Note, Durban, July.

Furthermore, 86% of partners described OAU as a “supportive partner” with an improvement in the quality of support, with 20% more partners rating that they were “very satisfied” in 2015 compared to 2013 (OAU, 2015a). The increases in quality and nature of OAU’s support to partners reflect openness, a willingness to address weaknesses, deepening working relationships and a commitment to relationships with partners. In April 2015, following the announcement that OAU was closing its office in South Africa, 30 partners met to reflect on the value of the partnership model and document lessons learnt from the approach (OAU Partners, 2015). The meeting took place without OAU staff being present and was designed to be an open space to consider the relationship, as well as develop strategies to support partners in the future. A review of the attendance register reveals that the people there represented a broad range of NGOs from all over South Africa that delivered services to vulnerable and marginalised people, and cumulatively they had worked with OAU for more than 160 years. Apart from an internal dialogue, the meeting also intended to feed into the closeout strategy and supplement the documentation of the partnership approach. The facilitated discussion revealed that there was unequivocal appreciation for the following: • The respect displayed by OAU for partners and communities; • • • • • • •

The acknowledgment of OAU of partners’ expertise; The fact that OAU did not impose their own agenda; The welcoming of creative approaches and flexibility in programming; The humanity and strong interpersonal relations of OAU, which enabled partners to feel like family; The capacity-building and exchange programmes were empowering and enabled partners to network and mix across disciplines; The ability of OAU to balance boundaries; and The fact that OAU was a long term funder and in ‘for the long haul’.

There were, however, some concerns raised in the partners’ FGD. Two are important to raise here. The first was that “not all programme managers were reliable”, which was less a criticism of the program as a whole and more that some partners had different experiences to the ‘positive norm’ due to particular personalities and the way they conducted themselves. The second was the “need to pay tranches on time”, an issue that was raised in different ways. This reflects the various interfaces at play within NLV: between OAU staff in South Africa and partners, between OAU staff and the South African administration of the program, and between the South African administration and funding sources across the Oxfam affiliates. In anticipation of OAU’s exit from NLV, the partner satisfaction surveys were used to understand the partnership model better. This was to inform the exit process to ensure effective communication occurred throughout. Three principles were selected that demonstrated a strong partnership: mutual respect, trust, and collaboration. Results from the survey showed that, from a partner’s perspective, these principles were being practiced across the OAU partnership by the different organisations with the clear majority rating in the “good” category and above (OAU, 2015a). The survey explored different aspects of OAU as a donor and as a partner. Figure 1 is a testimony to how 23 partners viewed OAU in terms of approachability, support and trust in particular.

Figure 1: Oxfam as a partner and donor Approachable Arrogant Demanding Flexible Guarded Inflexible Innovative Responsive Slow to respond Supportive Transparent Trusting Other (please specify) 0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Source: OAU, 2015c

The partnership model also emphasised collaboration between partners to find synergies as part of an integrated approach and to build alliances in anticipation of the exit, and beyond. A challenging area, as reflected by partners in the surveys, was the principle of collaboration among partners: only 19% of partners rated the practice of this principle among themselves as ‘excellent’, and 24% rated it poorly (OAU, 2015a). The majority of partners (57%) rated one another as either “very good/good”, which reflected a relatively strong foundation of partnership. Some quotes from the 2015 partner survey are useful to illustrate how a cross-section of partners regarded the partnership approach: “Oxfam Australia treats its partners in development with respect (for what they are doing) and dignity. They are truly our partners in development. I particularly value the fact that it publishes key data for public consumption bringing important issues to the fore – for example food security issues. Their questions are empowering and they are good listeners too demonstrating they have a real interest in development and building organisational capacity. I believe many other organisations in SA can learn from this approach”. “Oxfam has a unique approach to development work that allows partners both to learn and to contribute to learning. Overall this is an empowering approach. Also OAU takes care to develop sound well-researched strategies, which have a measurable impact for the partners”. “Oxfam are true partners. They do not impose their own agenda but are open to the partners needs”. “Oxfam provides a flexible and transparent partner relationship. Oxfam’s partner interaction is superb”.

29 OAU, 2015c, Oxfam Partner Satisfaction Survey 2015, Results Presentation, Durban, KwaZulu-Natal.

“The Oxfam partnership grew from strength to strength. Very innovative and creative. It was always good and joyful to meet with other partners during link and learn activities because it was very open and relaxed. A culture for learning and growth was always guaranteed. We will miss this opportunity. Working with Oxfam has challenged us and made us to grow and engage/discover new areas we didn’t know we could work with”. The roundtable discussion conducted with a selection of partners in November 2015 reiterated this point in a very strong way. It was emphasised in a discussion with representatives of over twelve partner organisations that the ‘Link and Learn’ approach has created an important impact to engender a collaborative, learning approach amongst partners. It was, however, stressed and agreed that this was not enough to build lasting collaboration. One participant described this as being inherent within individual organisations, an “internalised capitalist ethos” that placed themselves and their organisation above others in fear of “loss of funds or a stealing of funders” which compounded a “lack of trust between different organisations” (Durban Roundtable Discussion30). This reflected the mental state of organisations that often exist in survivalist mode, a situation that was likely to continue as long as there was a competitive nature for funding, and duplication of services. Despite the resistance to change, one partner expressed the belief: “We have to change otherwise we are going to die” (Durban Roundtable Discussion31). Although a very difficult time, partners also in the main felt happy with the exit process, as it had occurred to date. In the words of one partner, “we are disappointed that it [the closure] has happened, but we are happy with the way it is happening” (OAU, 2015d). Organisations appreciated the extra support being offered, the notice given, and the professionalism in handling the process. More than one partner indicated that this is much better than any other donor exit they have experienced. In the final reporting period, January to June 2015, the critical challenge of the “funding crisis experienced by partners and South African civil society in general” was acknowledged in light of OAU partnership’s response to this (OAU, 2015c). The exit strategy was implemented in a way that partners were able to organise themselves around OAU’s departure with the partner workshop that outlined a series of action items for partners to take forward. This included a road show that was organised in January 2015 to announce the closure of NLV to partners and to initiate dialogue on the exit. This was followed by a number of follow up discussions during the April/May 2015 monitoring visits, complemented by surveys to ‘check back’ with partners and test the ‘mood’ of their organisations. As emphasised earlier, the results demonstrated that overall, the partnership showed a high level of resilience in the face of the change with partner organisations articulating a high level of confidence in themselves (OAU, 2015d). In the words of a partner during the Durban roundtable discussion, “for communities to be able to stand on their own, it is important to remember and develop a sense of independence through partnerships” (Roundtable Discussion, Durban).

30 Roundtable Discussion with Partner Representatives, Durban, KwaZulu-Natal, 18 November 2015. 31 Roundtable Discussion with Partner Representatives, Durban, KwaZulu-Natal, 18 November 2015.

Participation The question guiding the analysis in this section is: • How have women and men, boys and girls and marginalised groups been involved in the different stages of the program cycle? As demonstrated in the analysis of impact, drawing on MEL data, the NLV program has emphasised the participation of people across the program cycle. Support was provided to all partners to strengthen their ability to include marginalised groups in particular. As to be expected, participation has been a core tenant of NLV, as expressed in MEL reporting. This issue, however, continues to be a major challenge in the South African development space, as articulated in the story below and the subsequent quotes from beneficiaries, grappling with meaningful participation on a daily basis. A representative of Rape Crisis in the Cape Town roundtable provided an example of how youth were being included in its work and, in so doing, had reinforced the broader work of the partner. There are not many opportunities on the Cape Flats for younger and older people. The program highlighted opportunities that existed in the community, supported by the development of a training manual. As many youth live in low-income areas, it was not possible to take children on camps without Oxfam. Through the program “it has been possible to spread information and skills beyond the Camp and bring those skills and information back to the community. Youth keep coming back for more and they themselves are encouraged to take actions in schools, homes and communities. The programme works from the individual through to the community” (Roundtable Participant, Cape Town). Through ensuring participation, youth have up-skilled themselves and this has filtered down to the personal lives, peer groups and youth in church. A number of beneficiaries raised the continual challenges of securing meaningful participation in their lives, as demonstrate by the quotes below. “We are under the authority of Amakhosi [traditional leadership]. Most authority comes from Inkosi, Induna and the Councillor. The Inkosi [Chief] has got custody of the land, the Induna is closest to the people and helps with community problems and crime. The Councillor is responsible for development and gives authority to development projects through working with ward committees. The Councillor is our link to the municipality, at the moment we find it difficult to communicate to the Councillor, he does not even come to the community meetings. Nkanini is slow in development; service delivery is slow because of political dynamics. During Councillor meetings people do not attend, the barrier is that service delivery is politically aligned. Our voices are not listened to” (FGD, Eshowe, KwaZulu-Natal).

“In the community we have old grannies that live in houses that are not in a good condition, there is no proper shelter and good care for them. Older children take their pension money and there is no money to educate the younger children” (FGD, Eshowe, KwaZulu-Natal).

“Some people do not have ID, [the partner] supports and assist those that need to apply for Identity document. People living in farm areas lose their jobs because they do not have identity documents and their employer cannot register them for UIF. The new VISA regulation of home affairs makes it more difficult for farm workers to obtain their identity documents” (FGD, Eshowe, KwaZulu-Natal).

“It is not that we ask for any favours or special treatment; we only want what we as LGBTI deserve as human beings and which the larger community is receiving regardless of who and what they are. We all deserve equal treatment like any other human being regardless of race or gender” (FGD, Ceres, Western Cape).

“People that live with HIV have freedom of speech, take good care of their families, educate their families, take care of child-headed families and connect and communicate with each other continuously” (FGD, Melmoth, KwaZulu-Natal).

Participation remains a significant challenge, whether in a communal area under traditional leadership, or because of one’s sexual orientation, gender or health status. These issues are further interrogated in the subsequent section.

Gender and disability The questions guiding the analysis in this section include: • • •

To what extent has the program supported more equal access by women, men and people with disability and living with HIV and AIDS to the benefits of the activity, and more broadly to resources, services and skills (e.g. by increased access directly, or by removing barriers to access)? To what extent has the project developed the capacity of partner, government and civil society to understand and promote disability inclusion? To what extent has the project developed the capacity of partner, government and civil society to understand and promote gender equality?

Gender The source documents and the formative evaluation were clear about the centrality of gender in NLV. The framework document set out the intentions: all of the work undertaken under this strategy will have a strong gender analysis. Work will be done both internally (with staff) and externally (with partners and other groups) to develop the skills and tools needed to improve poor programming in a way that makes it more ‘gendered’. By this, we mean that the program will recognise and respond to the need to place women more centrally as both actors in development and as primary beneficiaries of our work. It is useful to consider briefly the importance of gender work in programs focused on vulnerability. Gender is the socially constructed roles and relationship between men and women in a given culture or location and the societal structures that support them (Adeoti et al. 201232). As an example of why it is so important to consider gender in the context of vulnerable livelihood systems, Hovorka provides an important perspective on urban agriculture: “... men and women enter into agricultural production, and participate within this urban economic sector, on unequal terms based on socioeconomic status, location, and interactions with the environment. If urban agriculture is to contribute to food security and economic growth, as well as urban sustainability more generally, gender relations of power, as produced and reproduced through socio-spatial and humanenvironment relations, must inform understanding of this phenomenon”. (Hovorka, 200633).

32 Adeoti, A., Cofie, O. and Oladele, O. 2012. Gender analysis of the contribution of urban agriculture to sustainable livelihoods in Accra, Ghana. Journal of Sustainable Agriculture, (36)2:236-248. 33 Hovorka, A. 2006. Urban Agriculture: Addressing Practical and Strategic Gender Needs. Development in Practice, 16(1):51-61.

As articulated in the MTR, the NLV program has a strong gender analysis (2014). Work has been done internally with staff and externally with partners and other groups to develop the skills to recognise and respond to the need to place women more centrally, as both actors in development, and as primary beneficiaries of OAU’s work. This reflects the abiding concern of the program: unless the structural roots of vulnerability are addressed, any change may be temporary and illusionary. In terms of this evaluation, gender work has been largely embedded throughout OAU’s work. The important learning identified in the MTR was that women’s empowerment is often the key to addressing vulnerability, an argument that seemed to be increasingly accepted and recognised across programming from WASH, food and nutrition security, and smallholder farming, to disaster risk reduction and accessing services. The overarching strategy to empower women was through their participation in groups that build community solidarity to address social issues. Group interaction included sharing information, building trust and establishing alliances. Beneficiaries themselves best express the impact of some of this work. “There has not been much change in the roles, power and authority in the community, we would like to see more women in leadership, and we want to see women Indunas [chiefs] in the future” (FGD, Eshowe, KwaZulu-Natal).

“Women do not have a voice in our community. We as women would like to be assisted in some ways on how to stand up in our community and take on leadership roles” (FGD, Bergville, KwaZulu-Natal).

“The Chief has more authority in the community and people are scared of him. We feel the Chief oppresses people. There are always fees to be paid per family for different activities of the Chief” (FGD, Melmoth, KwaZulu-Natal).

“Women don’t use their power in different areas because they are overpowered by men. Men are those who practicing their powers within the different communities, for example chiefs are men. Traditional leaders are making decisions alone without involving other people, and these can lead to young people knowing that men and women have different roles and responsibilities” (FGD, Vhutshilo, Limpopo).

“In our community women are not too involved in community activities. We need women with leadership skills to lead us then we will follow” (FGD, Bergville, KwaZulu-Natal).

“We have learnt to express ourselves as women, we are aware of our rights and can stand up for our rights” (FGD, Melmoth, KwaZulu-Natal).

“In our community we now have Induna as a woman and Indunankulu as a woman. We also have some women that are the police for the Inkosi” (FGD, Melmoth, KwaZulu-Natal).

“We have noticed that women dominate men in our community and we would like to see more women Indunas“ (FGD, Melmoth, KwaZulu-Natal).

“When women leave their homes because of abuse, and when the woman comes back she needs to buy a cow to apologise meanwhile men do not do the same. Men are now being challenged to buy a cow when they want to chase away the woman” (FGD, Melmoth, KwaZulu-Natal).

“These changes (in recognition of women’s rights) have been caused by the way (the partner) facilitates development in the community (through) mobilisation, training, better communication and commitment with community leadership, and when they started their programs, they met with the community’s traditional leaders and their program was endorsed” (FGD, Melmoth, KwaZulu-Natal).

“As a result, women are very active in our community; they are doing caring work, they are loving it, they take care of vulnerable children, deal with housing issues, fight drug abuse and contributed in building a safe home for orphaned and vulnerable children and feeding children. Women have learnt to stand on their own two feet and provide for themselves” (FGD, Eshowe, KwaZulu-Natal).

“We feel all service providers including [the partner] are more focused on the black population and women” (FGD, Ceres, Western Cape).

“There are some changes because people who are doing gardening are women. In different households there are some changes because men and women participate in different activities equally” (FGD, Vhutshilo, Limpopo).

“As a minority group we know how it feels to be ostracised and discriminated against as the LGBTI community… and there have been no changes in the participation (of marginalised groups)” (FGD, Ceres, Western Cape).

Apart from illustrating the diversity of issues, contexts and experiences in the gender work supported under NLV, these quotes demonstrate a vulnerability context characterised by increasing poverty and inequality and also gender inequalities and discrimination, reinforced by socio-economic inequality. As expressed in the MTR, the objective goal to narrow inequalities as essential gendered work, and to provide women with the spaces, not only to bring their own solutions, but also to make their own choices and build solidarity, is pivotal for the ongoing work in South Africa.

Disability The issue of disability has been a key component of NLV since its inception. The first report at the end of 2012 highlighted the decision to engage CREATE to provide tailor-made capacity-building support on disability inclusion, with an early result that some partners developed programs that were both inclusive of people with disabilities and tailor-made to meet the local contexts (OAU, 2012). Woza Moya and Fancy Stitch made structural changes to their buildings, including ramps, handrails and toilets, and changes in the way data was collected and disaggregated. Partner staff members underwent training to understand how to design programs that were more inclusive. Tholulwazi Uzivikele (TU) and Woza Moya trained Home-Based Care Workers on disability issues so that they were better skilled to deal with disability in order to address issues of rights, stigma and discrimination. Three years later, Community Care Workers from Woza Moya who were trained in disability issues were, on a monthly basis, identifying and referring five people, on average, to local clinics for check-ups and further referrals (OAU, 2015b).

Work on inclusion of people living with disabilities continued to increase across partner organisations through the duration of NLV. For example, One Voice South Africa revised their child protection policy by ensuring that it was disability inclusive and that their school materials on human rights included disability rights (OAU, 2015b). RSS secured the services of sign language interpreters for their deaf clientele and further offered interpretation in refugee languages for mentally challenged clients so that they could have full access to the services. CREATE continued to play a strong supporting role assisting KMCRC to organise individual workshops for staff so as to develop related policies for disability inclusiveness. Staff from the Gay and Lesbian Network (GLN) attended sign language training to integrate and facilitate people living with disabilities. In addition, CREATE facilitated a one-day disability awareness workshop, which was attended by representatives of ten OAU partners (OAU, 2015b). Furthermore, CREATE continued playing its role of influencing for disability inclusion with the KwaSokhulu Traditional Authority and Umfolozi Local Municipality, enabling a case study to be developed that further strengthened the municipality’s inclusion of people living with disabilities. In terms of figures, the MEL data reveals that in particular, WASH and DRR made important strides in including people living with disabilities. The additional number of people living with disabilities accessing WASH services increased by 632 in 2015, a dramatic increase considering the total of 1,366 over the duration of NLV. This was a reflection of the capacity-building work around disability inclusion undertaken with WASH partners. Of the total, 620 were female. In terms of DRR, a total of 1,607 people living with disabilities engaged in partners’ activities. The majority of these were reached at the apex of the DRR programming in 2014 with a total of 1,067 reached that year. In the final reporting period, 162 people living with disabilities were engaged. In terms of beneficiaries’ own views on the inclusion of people living with disabilities, some nuance around these figures, and indeed the challenges faced by these groups, became clear. The starkest comment emerged from a beneficiary in Limpopo who argued “community members undermine people living with disability; they also experience discrimination and stigmatisation in the community. Community members don’t even take these people serious” (FGD, Vhutshilo, Limpopo). Another was more accepting but uncertain how people living with disabilities could actively engage food security work, “People with disabilities are sometimes limited in terms of our services; they get involved according to what they are able to do” (FGD, Umgababa, KwaZulu-Natal). Other partners provided a more promising perspective, that deep prejudices can indeed be overcome, as demonstrated by the work of NLV. “People with disabilities are being identified, being taken care of in a loving way that make them feel they belong into the community. We work with their families so that they accept them, take care of them by providing medication and food, get government grants and in this way their dignity is being restored” (FGD, KwaMakhutha, KwaZulu-Natal). Another beneficiary reflected on the impact of the partnership with CREATE, “We are involved in CREATE workshops where we learn about working with people with disabilities; how to treat them in the community, care and involve them on a daily basis (FGD, KwaMakhutha, KwaZuluNatal). This was reiterated by another who stated, “We have people with disabilities that attend our workshops” when asked about how people living with disabilities could be brought into development practice (FGD, Pietermaritzburg, KwaZulu-Natal). Once again the long-term nature of addressing vulnerability comes into focus. CATCH was forthright in arguing, “working with Oxfam has enabled organisations to lessen the vulnerability of children and women through working within existing structures and those who have been victims of crime…the expanded programme has made a huge difference in people’s lives and it has encouraged fresh ideas” (Roundtable Participant, Cape Town). OAU’s contribution to addressing gender dynamics and disability has provided important lessons for partners and others continuing this work; the contribution clearly making shifts in the lives of beneficiaries by shaping the strategic way forward. Simply put, however, the scale of vulnerability is such that it is a strong beginning at best.

Moving towards a conclusion, it is important to ask whether these outcomes would have happened regardless of the particular approach and ethos of NLV. A related question would be to determine whether the way in which NLV was designed, implemented, and monitored led to these changes. OAU has clearly invested and used resources appropriately to ensure it has consistently and successfully built the capacity of its partners to address the complexity that is vulnerability in South Africa. It has used resources to support partners to effectively implement context-specific programming that was co-designed and highly valued by partners and communities alike. It enabled partners to link and learn collectively and engaged appropriate technical expertise both inside and outside of Oxfam. NLV was managed responsively; the large investment in organisational capacity-building support was central to strengthening the organisational architecture that supported and enabled partnersâ&#x20AC;&#x2122; work. This investment was also directed to partners to sustain them beyond the close of NLV and, indeed the close of Oxfam Australia in South Africa. The strong and respectful partnership established between OAU and partners, and between partners, was evident throughout this evaluation. These relationships supported the efficient delivery of NLV and supported a focus on program delivery and collaboration for practical program improvement. It is also clear that the leveraging of partner knowledge, skills and resources has been extremely high, leading to innovation within NLV (including DRR, the use of the Integral Framework, and integration) with several partners forming collaborations outside of NLV. The capacity-building inputs have clearly cascaded to benefit communities, with partners bringing about a range of outcomes for communities, including changes in knowledge, practice and increased access to services. The indications that aspects of NLV will continue to be integral to partner organisations is high. The program design supports a high level of sustainability by using an integrated approach, which aimed to ensure partners continue to deliver services and support after the partnership with OAU has ended. Overall, the program performed well in supporting partners to ensure programs effectively targeted and reached vulnerable groups. This included disability inclusion and child protection. Comprehensive data provided clear evidence of the many benefits experienced by women, children and people living with disabilities across the NLV duration. Indeed, there is clear evidence to indicate that partners deeply valued NLV, and have contextualised it to their programming contexts and communities. This is significant given the wide range of partner organisations and the willingness to embrace a new way of programming, built upon integration. This also reflects how NLV was designed and the appropriateness of the approach in South Africa. Drawing out a set of key lessons and recommendations from such a wide-ranging summative evaluation is immensely challenging and, as such, it seems appropriate to highlight some of the voices of the beneficiaries in addressing and explaining their challenges. â&#x20AC;&#x153;Even though we do not have everything we need because it is about a certain political party, we have a voice about the services in our community. Some people we do not know come and work in our community because they belong to a certain party. Sometimes you benefit if you become a member of a particular political party. We do not know which channels to follow for our needs to be metâ&#x20AC;? (FGD, Melmoth, KwaZulu-Natal).

“At the moment we take long walks to the groups which expose us to crime and rape, there is no public transport in our routes and sometimes people ask us where the money that we save is” (FGD, Melmoth, KwaZulu-Natal).

“Sometimes the members of the groups disclose the share out date [from the savings groups], the police are also involved in crime so we can’t really trust them. We hope these challenges will not last long, we will find other ways as we will not let these challenges stop us from what we have started. Our work is great and we won’t let anything to put us down. We can do and afford anything; we trust the process” (FGD, Melmoth, KwaZulu-Natal).

“Due to the fact that justice was not served in the first two murders of young LGBTIs that took place still leaves us extremely vulnerable and living in fear as the LGBTI community. As far our safety as (the) LGBTI community is concerned, nothing has changed” (FGD, Ceres, Western Cape).

“It is always a struggle to get a safe space which means we can only have meetings at one of the LGBTI members’ houses” (FGD, Ceres, Western Cape).

“Government gives funding for gender equality which is not spent on the LGBTI community. [We were] told that they only assist ‘real women’” (FGD, Ceres, Western Cape).

It is clear that the intractable structural challenges that underpin vulnerability remain strong in South Africa. The experience of NLV has shown how a program with an explicit theory of change embedded in how it works can simultaneously engage the immediate, lived realities of people facing deprivation whilst attempting to link to shifting some of the structures that embed vulnerability. The holistic approach has attempted to avoid the ‘traps’ that characterise this terrain. As seen from the complex context in which OAU has worked, facilitating change depends on working across multiple spheres: individuals within communities (rights holders) holding local authority (duty-bearers) to account; improved access to services; effectiveness of governance through strengthened connection between rights holders and duty-bearers; and reduced inequalities through access to better services. A number of key questions emerge for organisations that might follow OAU in addressing some of these issues. What is the best future strategy to address the structural underpinnings of vulnerability? Is this best addressed through support focused on high-level advocacy and political influence or rather on processes built from the grassroots with an explicit theory of change that pivots on changing consciousness that feeds into collective action which can help challenge the structural issues from multiple dimensions? Is the combination of both an effective strategy, one that allows each dimension to feed off and support the other?

There is no question about the impact that NLV has had, acknowledging its own recognised limitations and, indeed, learning and evolutionary approach. Perhaps one word that captures this success is that of trust. OAU has built relationships of trust across the partnership, embedded in communities and upwards to those in positions of authority. The genesis of this might be found in the early work of the organisation in KwaZuluNatal. As a senior representative of a partner organisation reminded the Cape Town roundtable discussion, “The organisation has built long-term relationships after the ravages of the KwaZulu-Natal civil war in the nineties, which still resonates as the organisation continues working with the trauma of that era. Oxfam’s convening power pulls together organisations and builds a body of strength” (Roundtable Participant, Cape Town). This set of relationships enabled the individuals within NLV to fully comprehend what vulnerability actually looked like in South Africa and to then design and execute a program in response. Another senior representative of a partner organisation concluded in a reflection on what NLV had achieved, “The partnership with Oxfam has been one of collaboration and heightened consciousness to continue the work of the organisation. There was mutual respect and trust. It was about people-centred development that enhances an organisation’s own processes. Linking and learning were high on the agenda. One could describe the outcome of the partnerships as ‘the seed is alive’” (Roundtable Participant, Cape Town).

Adato, M., Carter, M.R., and May, J. 2006. Exploring poverty traps and social exclusion in South Africa using qualitative and quantitative data. Journal of Development Studies, 42:226–247. Adeoti, A., Cofie, O., and Oladele, O. 2012. Gender analysis of the contribution of urban agriculture to sustainable livelihoods in Accra, Ghana. Journal of Sustainable Agriculture, (36)2:236-248. Bunce, B. and Cousins, B. 2015. Communal Area Agriculture and Rural Development: A Consultation Workshop Held at the Donald Woods Foundation, Hobeni, September 15th - 17th, 2015, Institute for Poverty, Land and Agrarian Studies, University of the Western Cape. www.plaas.org.za Case, R., Andrews, M., and Werner, W. 1988. How can we do it? An evaluation training package for development educators. Research and Development in Global Studies, British Columbia. www.wlu.ca/programs-Global-Studies Drimie, S. 2012. The Oxfam Australia Program Framework for South Africa Formative Evaluation, Oxfam Australia (Internal Report). Drimie, S. 2014. Mid-Term Review (July 2012 - December 2013) No Longer Vulnerable Program (July 2012 - July 2015) (Internal Report). Ferreira, L. 2015. FACTSHEET: Social grants in South Africa – separating myth from reality. Available at: https://africacheck.org/factsheets/separating-myth-from-reality-a-guide-to-social-grants-in-southafrica/#sthash.6bJyrUih.dpuf Friedman, S. and Hudson, J. 2011. “Filling the Gaps? The Future of CARE in South Africa”, Report on CARE’s Role in South Africa in the Light of Current Development Needs and Stakeholder Responses Commissioned by CARE South Africa. www.care.org/country/south-africa Hovorka, A. 2006. Urban Agriculture: Addressing Practical and Strategic Gender Needs. Development in Practice, 16(1):51-61 Moolman, A. 2015. An ecological approach to partnership, draft reflection document, OAU, Durban, South Africa. (Internal report) Rao, A. and Kelleher, D. 2005. “Is There Life After Gender Mainstreaming?”, Gender at Work. Available at: http:// policy-practice.oxfam.org.uk/publications/is-there-life-after-gender-mainstreaming-131582 Shackleton, S., Ziervogel, G., Sallu, S., Gill, T., and Tschakert, P. 2015. Why is socially-just climate change adaptation in sub-Saharan Africa so challenging? A review of barriers identified from empirical cases, WIREs Clim Change 2015. Available at: doi: 10.1002/wcc.335. Oxfam Australia. 2012. No Longer Vulnerable Report for period July to December 2012, Durban, KwaZulu-Natal. (Internal Report)

Oxfam Australia. 2012a. The Oxfam Australia Program Framework for South Africa Formative Evaluation, June 2012, Durban, KwaZulu-Natal. (Internal Report) Oxfam Australia. 2012b. No Longer Vulnerable: The Oxfam Australia Program Framework for South Africa, July 2012 to June 2015, Durban, KwaZulu-Natal. (Internal Report) Oxfam Australia. 2013a. No Longer Vulnerable Report for period January to June 2013, Durban, KwaZulu-Natal. (Internal Report) Oxfam Australia. 2013b, No Longer Vulnerable Report for period July to December 2013, Durban, KwaZulu-Natal. (Internal Report) Oxfam Australia. 2014a, No Longer Vulnerable Report for period January to June 2014, Durban, KwaZulu-Natal. (Internal Report) Oxfam Australia. 2014b. No Longer Vulnerable Report for period July to December 2014, Durban, KwaZulu-Natal. (Internal Report) Oxfam Australia. 2014c. Management Response to the South Africa Midterm Review of ‘No Longer Vulnerable’ Strategic Plan 2013 -2105, September, Durban, KwaZulu-Natal. (Internal Report) OAU Partners. 2015. “The Seeds are germinating” Legacy and Lessons: Oxfam Australia in South Africa, Durban, KwaZulu-Natal. (Internal Report) Oxfam Australia. 2015a. No Longer Vulnerable Partnership Satisfaction Learning Note, Durban, July. (Internal Report) Oxfam Australia. 2015c. Oxfam Partner Satisfaction Survey 2015, Results Presentation, Durban, KwaZulu-Natal. (Internal Report) Oxfam Australia (South Africa). 2015d. OAU Program Closure in SA: Where are we at? June, Durban, KwaZulu-Natal. (Internal Report) OAU. 2015e. No Longer Vulnerable Operational Plan 01 July 2015 to 31 March 2016, Durban, KwaZulu-Natal. (Internal Report) Woza Moya. (Prose, N., and Naidu, T.). 2015. Being accountable. Ixopo. Available at: https://www.youtube.com/watch?v=opC5dQehTpY&feature=youtu.be Woza Moya. Mazisize Self Help Group. 2015. Ixopo. Available at: https://www.youtube.com/watch?v=1zFM6n74KD8 Woza Moya. 2015. About Woza Moya. Ixopo. Available at: https://www.youtube.com/watch?v=DpxtpwUilEo

Annexure One: NLV Evaluation partner selection matrix

Partner Name

Location

Main Issue

Engagement in MTR

Number of activities involved in

Attendance at ReVision Regional Indabas

Access to Beneficiaries (over 18)

Africaid

KZN

Health

DBN

High

aln

western cape

Health

afh

kzn

Rights

DBN

bela bela

limpopo

Health

LIMPOPO

biowatch

kzn

Food security & livelihoods

DBN

catch

eastern cape

Health

choice

limpopo

Health

LIMPOPO

create

kzn

Rights

dbn

Med

dhc

kzn

Health

dbn

Low

fsg

kzn

Food security & livelihoods

dbn

High

hact

kzn

Health

dbn

Low

haven wellness eastern cape

Health

n/a

Low

inner circle western cape

Rights

N/A

Low

High

isibane

kzn

Health / Food security & livelihoods

dbn

Low

jaw

kzn

Gender

DBN

High

kmcrc

kzn

Rights

Dbn

High

krcc

kzn

Gender

Dbn

High

lifeline

kzn

Rights / Health

Dbn

Low

lima

kzn

WASH

Dbn

Low

loves and fishes

eastern cape

Rights

N/A

one voice

kzn

Health / WASH

DBN

Focus group (our opinion)

Low

Exclude

Partner Beneficiaries

Partner Name

Location

Main Issue

Engagement in MTR

Number of activities involved in

Attendance at ReVision Regional Indabas

Access to Beneficiaries (over 18)

opup

kzn

Health / Food security & livelihoods

DBN

High

pacsa

kzn

Rights

DBN

High

palabora

limpopo

Health

Limpopo

Low

gln

kzn

Rights

High

project empower

kzn

Gender

DBN

Low

rapcan

western cape

Rights

Low

rape crisis

western cape

Rights

Low

rss

kzn

Rights / Health

DBN

Low

save

kzn

WASH

DBN

High

sinamandla

kzn

Food security & livelihoods

dbn

Low

sinani

KZN

Rights

DBN

High

siyavuna

kzn

Food security & livelihoods

dbn

High

sophakama

eastern cape

Health

dbn

tac

kzn / national

Health

dbn

Low

kzn

Health / WASH

dbn

High

thusanang

Limpopo

Rights

limpopo

High

triangle

western cape Gender / Health

High

tcoe

security western cape Food & livelihoods

High

limpopo

High

vms

limpopo

Health

wfsa

security eastern cape Food & livelihoods

dbn

High

wfp

security western cape Food & livelihoods

High

dbn

High

woza moya

kzn

Health / WASH

Focus group (our opinion)

Partner Beneficiaries

Annexure Two: FGD Guidelines Oxfam Australia’s No Longer Vulnerable Programme Focus Group Discussion Guidelines Introduction The first step is to introduce yourselves and explain what you are doing. The following script provides guidance, but you can adapt this and put it in your own words: Thank you all for coming. My name is ........................ and my colleague is ........................ . We are working on a process that will form part of an evaluation on what is known as the “No Longer Vulnerable” programme supported by Oxfam Australia. The goal is to understand whether the programme has achieved what it set out to achieve. We also hope that the results we generate here together will be useful to you in addressing vulnerability within your communities. The overall question is to identify strategies and approaches that have worked, and which have not, and to make recommendations about how the programme could have been improved. This is so your experiences of the programme can inform similar programmes in the future. We will be using a process called a Focus Group Discussion. This is based on a set of questions to allow you to reflect and discuss what has happened. This is so we can get the insight of people like you, who are part of the process and know it from the inside. We are very interested in hearing different viewpoints. There are no right and wrong answers. We are just interested in what you really think. The answers and information you give in this focus group discussion will be completely confidential. We will explain what people in this community and others think in a report but we will not mention any names. We will be recording this discussion because we want to be sure that we note what you say correctly. Your personal contributions and views will not be shared with anyone else in a way that can identify you. I will be asking the questions and my colleague will be writing down your answers. The discussion will take about 2 hours to complete and we will have some refreshments later. Do you have anything you want to ask me?

Ask the participants about the activities of the organisation that works with them (supported by Oxfam Australia) • What does the work that the organisation does with you look like within your community? • What does it do? • What is the focus of the work?

Ask the participants whether the programme has changed anything in their lives over the past few years • Has anything changed in your lives over the past few years? What are these changes? • Were these changes a result of the programme? In what way? • Are these changes part of what the programme said it wanted to do?

Ask the participants whether they have been involved in the programme • How has the organisation, which works with you, included you in the programme? • Have you had a chance to tell the organisation what you need as a community? • How have women and men been involved in the programme? • How have boys and girls been involved in the programme?

• Are there marginalised groups in your community? Who are they? • How have marginalised groups been involved in the programme? • Have you or any other members of the community had a chance to meet and talk to Oxfam Australia? What was this meeting about?

Ask the participants to discuss whether any roles, power and authority have changed over the past few years • Have any power relations changed in the community over the past few years as a result of the programme? • What are these changes? • What do they think has caused these changes? • Who has most authority in your community at present? Why is this so?

5) Ask the participants to talk about whether they have a voice about the services that are brought to their community • Do you receive services in your community? From who do you receive these services? What are they? • Do you have any influence or voice about the services you receive? • Have you seen changes over service delivery over the past few years? What do the government people do now that is different? • What do you consider as a good service? • Do you expect good service and honesty from the government? Do you feel you can claim the right to access good services? • Do you expect good service and honesty from other organizations and NGOs? Do you feel you can claim the right to access good services? • Do organizations or government promise services they don’t deliver? • Does anything happen if a service is not provided as promised? Do you have anybody to go to for complaints? • Do you have more confidence about asking for better services?

Ask the participants whether they think it is possible to increase women’s power and authority in the community • Who has authority in your community? What is this authority over (land, housing, etc.)? • Have there been changes in who has authority over the past few years? • Has there been any change in women’s power and authority as a result of the programme? • How have these changes happened over the past few years? • What kinds of changes are needed to increase women’s power and authority in the community? • What kinds of changes are needed to increase women’s power and authority in the household?

7) Ask the participants whether they think it is possible to include the participation of people with disability and living with HIV and AIDS in the community • Has there been any change in the participation of people with disability and living with HIV and AIDS in the community as a result of the programme? • How have these changes happened over the past few years? • What kinds of changes are needed to increase the participation of people with disability and living with HIV and AIDS in the community in the community?

• How should these changes happen? Where could they come from?

Ask the participants to discuss the sustainability and aspirations of this work • Are the changes you have experienced going to continue? • How will they be sustained? Who will sustain them? • What are your aspirations or dreams for your community?

Annexure Three: MEL Framework aim: people living and working in south africa are less vulnerable outcomes and indicators improve health outcomes relating to hiv and aids, tuberculosis and water related infections and diseases

increase and sustain food security and livelihoods options available to households

# of people who benefit from integrated approaches to health and wellbeing34

# of people assisted to attend and complete life skills, literacy, technical / vocational / job skills training or leadership programs

# of people provided with voluntary HIV/AIDS and STD testing & counselling and other care services provided (including home based care) # of additional people with increased access to essential medicines and health commodities including HIV treatment # of people trained in the delivery of comprehensive HIV prevention and care services # of prevention services provided 35

# of people provided with increased access to safe water # of people provided with increased access to appropriate sanitation

# of people assisted to participate in sustainable livelihood generation activities36 # of people reporting increased financial security including through establishment of small business # of households reporting an increase in food security # of people able to access social transfers37 # of communities / people assisted to develop community disaster risk reduction plans

increase and uphold access to social protection / rights # of women beneficiaries in leadership roles in partner programs # of programs / training for community members and organisations which include a focus on accountability and transparency of leadership and / or pro-poor policy # of structures / systems put in place to allow communities to engage government and other duty bearers e.g. traditional leaders # of communities reporting an increase in the level / quality of government service delivery attributable to the project or program

# of people with access to hygiene facilities # of people trained in the delivery of comprehensive WASH programs # of people with increased knowledge of hygiene practices

objectives and process indicators strengthen community development approaches with a focus on supporting civil society organisations (both formal and informal groups of people)

support the sustainable delivery of, and increased community participation in, integrated development programs

create and sustain enabling environments with a focus on communities of vulnerable people

# of partners contracted

# of exchange visits facilitated

Proportion of program spend on grants

# of models of integrated programs developed

Social policy monitoring systems developed (OAU)

Proportion of program spend on capacitybuilding and support Proportion of contracts completed by due date # of partners with capacity-building plan in program proposals # of good practice case studies developed Proportion of partners with developed MEL plans Proportion of partners with sustainability plans

# of good practice examples emerging from partner work Increased participation of beneficiaries in partner programs Beneficiaries indicate high levels of satisfaction with services received Proportion of partners involved in collaborative service delivery

# of CSOs supported to work for improved protection of women # of partners supported for advocacy and lobbying media work # of CSOs playing an active role in engaging with government / decision-makers in order to improve increased access to services for the poor # of CSOs supported to work for improved protection of women, children and marginalised groups # of partner organisations engaged with governments on human rights, law and justice

Proportion of partners expressing satisfaction with program management systems and support provided

# of media pieces referencing partner work

Program reports delivered on time

# of research pieces commissioned and disseminated

Program reports meet donor requirements

# of references to Oxfam or partner research 34 35 36 37

including nutrition, infection control, access to health care, oral rehydration, bed nets and Maternal Child Health outreach activities, etc. e.g. condoms, needles and syringes, IEC, antenatal testing and counselling, peer outreach includes formal and informal sectors such as cash or in kind transfers including food

Title:

Summative Evaluation: ‘No Longer Vulnerable’ Program (July 2012 – June 2015)

Published: March 2016 Author:

Scott Drimie

Proofreader: Caitlin Martin Design:

LUMO design & illustration (www.lumo.co.za)

Copyright: Oxfam gives permission for excerpts from this book to be photocopied or reproduced provided that the source is clearly and properly acknowledged. Disclaimer: The views in this publication are those of the respective authors and do not necessarily represent those of Oxfam or any funding agency. The interview and review process was participatory and consent around content and inclusion of personal information was given to Oxfam by interviewees. Contact Details: Oxfam Suite 1B, Strathway Building Strathmore Office Park 305 Musgrave Road Durban, South Africa +27 (0) 31 201 0865 enquire@oxfam.org.au

This Summative Evaluation was supported by the Department of Foreign Affairs and Trade, Oxfam Germany and Oxfam Ireland.