Joint Evaluation of Responses to the Yogyakarta Earthquake by Catholic Relief Services

report CARE, Catholic Relief Services,

Save the Children and World Vision Indonesia

Photo courtesy of David Snyder

Joint Evaluation of Their Responses to the Yogyakarta Earthquake July 2007

Independent Evaluation by: Pauline Wilson and Donal Reilly with support from Ryan Russell, Malaika Wright, Astri Arini, Desideria Cempaka, Eri Diastami, Listya Narulita, Maria Angela Anindita, YB Johan Dwi Bowo Santosa, Yos Handani, and Yustina Tri Wahyuningsih

report CARE, Catholic Relief Services, Save the Children and World Vision Indonesia

Joint Evaluation of Their Responses to the Yogyakarta Earthquake July 2007

Acknowledgements

Many people willingly and actively shared their views with the evaluation team. This included the many women, men, children, teachers and village leaders we had the privilege to speak with in each of the villages visited. Indonesian government (GOI) staff at district and sub-district levels and in department of health and education offices were generous with their time as were UN staff in both Yogyakarta and Jakarta. They all shared their views on the effects of the earthquake and the emergency response work done by CARE, Catholic Relief Services (CRS), Save the Children (SC) and World Vision Indonesia (WVI).

The joint evaluation team would like to thank the staff from the four participating agencies in both Jakarta and Yogyakarta, who took the time to describe their agency response to the Yogyakarta earthquake. The team would particularly like to give great thanks to the steering committee of Adhong Syahri Ramadhan and Maria Josephine Wijiastuti of CRS, Agus Budiarto and Evi Esaly Kaban of Save the Children, Harining Mardjuki and Anwar Hadipriyanto of CARE, and Richardus Indra Gunawan and Yacobus Runtuwene of WVI who invested much time and effort in organising and managing the evaluation process. The assistance of Endang Triningsih and Wiwit Ekawati of CRS, and Oline Sigar of WVI in processing visas and making all travel arrangements for the team was appreciated. Also invaluable was the support from the many drivers of the four agencies who ensured our travel to the villages to meet with the local people at prearranged times.

Special thanks to Adhong Ramadhan, Josephine and Yenni Suryani and the many staff members of CRS Indonesia who hosted the evaluation team so graciously. The CRS Indonesia staff provided us a welcoming environment and much support to complete the evaluation. Special thanks also to Hester Smidt who voluntarily doing proof reading and editing report of the final evaluation. A big thanks also to Guy Sharrock, CRS headquarters senior advisor for monitoring and evaluation, for providing guidance and support to all during the preparatory stage of the evaluation. The evaluation team is grateful to everyone for all their assistance and are indebted to all of you for the time, information and support you provided. This joint evaluation would not have been possible without your participation.

A Joint Evaluation of the Yogyakarta Earthquake Response July 2007

Executive Summary

1. Introduction Following the earthquake in Yogyakarta on May 27, 2006, CARE, Catholic Relief Services (CRS), Save the Children (SC) and World Vision Indonesia (WVI) responded separately to the disaster. Although the agencies worked independently of each other, it was felt that a joint evaluation (JE) of the response would demonstrate greater accountability and the results would be taken more seriously. The objectives of the JE were to assess individual agencies on:

• The impacts of their responses and identify promising practices and indicators on impact measurement. • The appropriateness of agency responses. • Whether the responses had helped the recovery of people and communities. • The level of agency accountability to local people. • Organisational preparedness to respond to emergencies. In addition, learning on joint evaluations was assessed.

2. The Context The Yogyakarta earthquake killed an estimate 5,700 people and injured 27,000. Over 300,000 houses were destroyed or severely damaged and a further 200,000 suffered minor damage. 1.6 million people were left homeless. An additional 1.1million people were affected1.

Recovery is now well underway in the affected areas, as those affected have been provided with some form of shelter assistance, health and education services are operating, and children are back in school and say they feel less traumatised. However, many gaps still remain, particularly due to the limited recovery of economic livelihoods.

3. The Response by the four agencies

At the time of the earthquake, three of the agencies had teams on the ground responding or preparing to respond to a potential eruption of the Mount Merapi Volcano. They began assessments and redeployed NFI kits from the Mount Merapi crisis to earthquake-affected areas. The fourth agency began their response on May 29th 2006. Many staff employed in Yogyakarta had worked in their agency’s emergency response program in Aceh Province. They were able to apply their learning from Aceh to the more recent disaster in Java

Source: UNDP: The Cluster Approach in Yogyakarta and Central Java One Year Review, 2007, p.1.

A Joint Evaluation of the Yogyakarta Earthquake Response July 2007

vii

and work more effectively with local government and community structures in distributing aid to affected people.

All four agencies were credited with working in remote areas that were damaged severely. The agencies carried out rapid assessments and NFI distributions of shelter, hygiene, clothing, household and clean up kits. WVI provided extensive support to reactivate health services at sub-district and village levels. CRS, SC and WVI implemented activities to protect children and reduce their trauma. SC and WVI provided support for elementary schools to restart classes in mid-July 2006 as well as support for schools to operate effectively. CARE, CRS and WVI implemented transitional or permanent housing programs, while water and sanitation activities were implemented by CARE and CRS. The largest activity by each agency was NFI distribution and collectively the agencies reached around 20% of all affected people with shelter and other NFI kits.

4. Conclusions

Conclusions are based the views expressed most frequently by aid recipients, local government officials and staff from village level up to district level.

Appropriateness: Most activities were considered appropriate and justified. The friendliness of staff was appreciated and the fact that all these agencies arrived at the start of the emergency and responded quickly. Agencies were also commended for the high quality of the goods they provided and the fact that they tended to monitor distributions, the selection of beneficiaries and the use of their assistance regularly. Concerns raised were related to the overall response and distribution process. Oversupply and undersupply occurred in some villages. In addition, officials and villagers noted that assessments were carried out in the same location by different agencies, indicating a lack of coordination. Respondents said coordination between agencies on their emergency response needs to improve. Another concern was about the way agencies work with affected people. While noting the positive impacts of assistance, informants said assistance created conflict and dependency in some village locations. Concerns over distribution, and the importance of it being fair and not creating conflict were raised in seven out of the nine villages visited.

Impact2: Agency activities did contribute to positive impacts. As there were many agencies and actors responding to the emergency, impacts cannot be attributed to the specific agencies who participated in this evaluation. The impacts mentioned most often by recipients and village leaders were:

• NFI support helped meet the basic survival needs of affected people. • CRS, SC and WVI children’s activities helped to reduce children’s trauma and increase their selfesteem and confidence. • SC and WVI elementary school support ensured that schools could restart in mid-July and work effectively thereafter.

2 The definition of impact used is from the Impact Measurement and Accountability in Emergencies: The Good Enough Guide, page 4. The Guide also informed the team’s review of accountability. viii

A Joint Evaluation of the Yogyakarta Earthquake Response July 2007

• WVI’s health sector support helped ensure that local people had access to basic health care services quickly. • CARE and CRS water and sanitation activities helped to improve people’s access to clean water and increase their knowledge of hygiene. • Agencies implementing shelter programs helped families to have a place to live that is more earthquake resistant. • CARE and CRS were credited with working in ways that helped increase cooperation and solidarity at community level.

Recovery: Agency activities did help affected people and communities to recover. Recovery levels reflect the support provided by all emergency responders and not just these agencies. Villagers said the contribution by the Government of Indonesia (GOI) to recovery was 50-60%, by the NGOs 25-30% and by others around 10%.

Villagers and leaders said that the elementary school system is 90% recovered. The work of SC and WVI was credited with contributing to this level of recovery. The children’s activities implemented by CRS, SC and WVI were credited with helping children to recover from trauma and respondents said trauma had decreased, though no percentage was given. Drinking water sources were said to be back to normal though sanitation and access to latrines was said to have recovered by only 50%. In most villages housing reconstruction is only between 30-50% and similar figures were given for economic recovery. Accountability to local people: The four agencies did work with local leaders and involved them in assessments, planning, monitoring and decision-making while at the same time involving the communities to varying degrees in these processes.

However, women in villages where three of the agencies worked said they wanted to be more involved and have more information on agency activities. All informants stressed the importance of regular information to all in a community, backed up by on going monitoring of the assistance programs implemented to ensure fairness and to avoid conflicts. Monitoring and Evaluation (M&E): Of the regular M&E activities, there were some good practices which are exemplary. Of note was the child-led evaluation carried out by SC, in which children were trained to actually do a program evaluation. Other agencies had carried out internal reviews and one agency also conducted an external evaluation of their post emergency program. All agencies were able to produce solid input and output data, and some like CRS had some easy to use outcome level indicators.

Emergency preparedness: The overall speed at which the agencies responded to the disaster was significant, mainly due to the fact that three of the agencies were already mobilized on the ground in Yogyakarta to respond to a potential eruption of the Mount Merapi Volcano. Otherwise, the response time may not have been so swift. Joint evaluation: The joint evaluation had advantages, in bringing together the organizations involved and providing them opportunities to learn from one another about each other’s programs. Results are more holistic than a single agency evaluation. The way the process was carried out enabled these agencies to be accountable to government, affected people and others working in A Joint Evaluation of the Yogyakarta Earthquake Response July 2007

the Yogyakarta response. However, such evaluations need to be done one or two months after an emergency program work ends.

Recommendations

Recommendations on activities for future sudden onset emergencies a) Continue to do the type of programme activities carried out in this response. Carry out assessments to ensure aid meets the needs of affected people and to agree with them procedures for distribution and beneficiary selection. Provide good quality items, distribute quickly and follow simple procedures. b) Better coordinate NFI programs between all actors/stakeholders to ensure equal distribution across areas and application of distribution methods that promote fairness. Monitor the assistance well by ensuring that staff participate in distributions and beneficiary selection processes. c) Carry out joint assessments so that the same information is not collected a number of times in the same location by different organizations.

d) Start recovery activities earlier e.g. transitional and permanent housing plus activities to restore livelihoods. e) Complete a study on the transitional and permanent housing designs and approaches used by these four agencies, other INGOs and the GOI in Yogyakarta to draw out learning that can be applied in Indonesia when responding to future emergencies where shelter is a huge need. Recommendations on economic recovery activities

a) Provide more support in helping affected HHs and communities to restore their economic livelihoods.

b) Learn from work done in other countries prone to sudden onset emergencies to identify appropriate economic livelihood activities to support in future emergencies in Indonesia. Recommendations on local accountability

a) Provide information to the wider community: men, women, beneficiaries and non-beneficiaries, on a regular basis so that people are aware of the work being conducted by the agency with them so reducing opportunities for misuse of information. b) Establish a complaints system that clearly defines how people can complain about the work being done by an agency if they need to do so.

c) From the start, involve women as well as men in planning, implementation and evaluation of programs. Recommendations on emergency preparedness

a) Complete country emergency preparedness and contingency plans and ensure that all staff are aware of their existence and content. This could be done through country program emergency

A Joint Evaluation of the Yogyakarta Earthquake Response July 2007

response simulations, followed by an interagency simulation once all agencies are comfortable with their own plans.

b) Create a joint database on the capacity of different agencies regarding the location and type of pre-positioned NFIs. Examine the feasibility of holding joint stock in shared warehousing. c) Better prepare staff who do not have emergency experience and ensure new hires receive appropriate training and supervision. Recommendations on joint evaluations

a) Once joint assessments are complete, plan for a joint evaluation to start within one to two months of emergency program completion. Use the Good Enough Guide s to inform the JE process.

b) Commit enough experienced program staff to the entire period3 of the JE so that the team has sufficient experience for an in-depth review of a few sector specific activities.

3 The two CRS team members were highly qualified emergency staff and did an excellent job. Transfer of knowledge as one member turned over their work to the other in the middle of the evaluation proved challenging. A Joint Evaluation of the Yogyakarta Earthquake Response July 2007

xii

A Joint Evaluation of the Yogyakarta Earthquake Response July 2007

BAKORNAS â&#x20AC;&#x201C;PB BAPPENAS CARE CFS CIMO CRS DINAS DINSOS DRR ECB FGD GOI HDI HH IASC INGO JEER MBR MCH MOH MOU M&E NA NFI OCHA SATKORLAK SC SPA SWS ToR UNCC WVI

Abbreviations

National Coordinating Board for the Management of Disaster National Planning Board

Cooperative for American Relief Everywhere Child Friendly Spaces

CARITAS Implementing Organizations Catholic Relief Services

Department of Education

Department of Social Welfare Disaster Risk Reduction

Emergency Capacity Building Focus Group Discussion

Government of Indonesia

Human Development Index Household

Inter-agency Standing Committee

International Non-Governmental Organisation

Jogyakarta Earthquake Emergency Response Market Based Relief

Maternal Child Health Ministry of Health

Memorandum of Understanding Monitoring and Evaluation

Not Applicable

Non-Food Item

Office for the Coordination of Humanitarian Affairs Provincial Board for the Management of Disaster Save the Children Safe Play Area

Safe Water Systems

Terms of Reference

United Nations Coordination Centre World Vision Indonesia

A Joint Evaluation of the Yogyakarta Earthquake Response July 2007

xiii

xiv

A Joint Evaluation of the Yogyakarta Earthquake Response July 2007

Table of Contents

Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . v

Executive Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii

Abbreviations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii

1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

2. The joint evaluation team, field work locations and methods . . . . . . . . . . . . . . . . . . . . . . . 2

3. Structure of the Report . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

4. The Yogyakarta earthquake emergency context . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

5. The emergency and recovery programs of CARE, CRS, SC, and WVI. . . . . . . . . . . . . . . . . . . . 8

6. The response by these agencies was largely considered appropriate . . . . . . . . . . . . . . . . . 10

NFIs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

Children Activities . 11

Water and sanitation activities 11

Support to government education and health services . 12

Transitional and permanent housing . 12

Recommendations from stakeholders on program activities: . 13

7. Agency activities contributed to impacts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

8. Activities implemented by these agencies and other actors helped people and their

communities to recover . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

Recommendations on recovery 17

9. Accountability to local people was mostly well done . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

Recommendations on local accountability . 19

10. Some good practice on monitoring and evaluation (M&E) is found . . . . . . . . . . . . . . . . . . 20

11. Agencies were prepared to respond because of Mount Merapi . . . . . . . . . . . . . . . . . . . . . 22

Examples of good practice: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

Critical gaps . 23

Recommendations on emergency preparedness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

12. The joint evaluation process had a number of advantages . . . . . . . . . . . . . . . . . . . . . . . . 25

What went well . 25

Advantages of a joint evaluation approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

Learning from the process . 26

Recommendations on joint evaluations . 27

13. Annexes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

A Joint Evaluation of the Yogyakarta Earthquake Response July 2007

xvi

A Joint Evaluation of the Yogyakarta Earthquake Response July 2007

Introduction

Early in the morning of May 27, 2006 an earthquake measuring 5.9 on the Richter scale struck central Java. There was extensive damage in eight districts of Yogyakarta and Central Java Province resulting in loss of life and injury to people. It is estimated that 2.7million people were affected by the disaster. Within the first 24 hours CRS, Save and WVI were responding to the Earthquake and CARE started its emergency response work on May 29th.

In May 2007, these four agencies commissioned an independent evaluation of the emergency response and recovery work they had each carried out from May 27, 2006 to May 27, 2007. The agencies had not worked together during the response. The joint evaluation in Yogyakarta started as a discussion between two agencies – Catholic Relief Services and Save the Children. Together they developed a terms of reference for the evaluation in January 2007 . A few months later, CARE and WVI confirmed their interest in participating. The agencies believed that a joint evaluation would demonstrate a greater level of accountability and objectivity and thus the results would be taken more seriously and could be used for advocacy purposes. The agencies formed a Steering Committee to oversee the joint evaluation and ensure that its focus would meet the needs of the four agencies involved. The committee agreed to the evaluation objectives as well as the methods and processes that would be used. The objectives of the evaluation were to assess individual agencies on:

• The impact of their work on the people and communities they served and identify promising practices and indicators on impact measurement. • The appropriateness of agency responses. • Whether their responses had helped the recovery of people and communities. • The level of agency accountability to local people. • Organizational preparedness to respond. In addition, the evaluation team was asked to make recommendations on future joint evaluations.

Please see Annex One for the joint evaluation terms of reference.

A Joint Evaluation of the Yogyakarta Earthquake Response July 2007

The joint evaluation team, field work locations and methods

The joint evaluation took place from May 28 through June 20th, 20075. The team was led by two members, an independent evaluator, and experienced emergency CRS regional staff members who were not involved in the Yogyakarta emergency response6. A member of the Emergency Capacity Building (ECB)7 project was with the team for the first ten days to assess the joint evaluation process. In addition, three facilitators, three note-takers and two translators bilingual in English and Bahasa Indonesian, were recruited locally. The level of experience on the team defined the scope and depth of work the team completed.

The evaluation team went to nine different villages in seven sub-districts. Villages were selected on the basis of the following criteria: (1) villages severely damaged by the earthquake, (2) those where agencies had carried out a significant level of work and (3) villages where more than one of the four participating agencies had worked8. The JE team visited three villages where CRS and CARE each had a significant level of work and two different villages where SC and WVI had a significant level of work. To compliment discussions and observations in the field, documents of each of the agencies were reviewed9 and discussions were held with agency staff.

In each village, separate focus group discussions (FGD) were held with men and women and in seven of the locations with children. In addition, the team met separately with the village leader in each location and interviewed non-beneficiaries to crosscheck the information provided by beneficiaries in the FGDs. The total number of people the team interviewed across the villages was 31810. Semi-structured interviews (SSI) were completed with six government and four UN officials11. Two interagency workshops were held, one in Yogyakarta and one in Jakarta to review and confirm the findings from the field with staff. Staff confirmed that the findings were representative of what we would have heard in other locations where they worked.

By triangulating information from various methods and sources, the team was able to bring together sufficient information to draw conclusions and make recommendations on the emergency response by these four agencies. These conclusions were discussed at a multi-stakeholder event on June 20th in Yogyakarta which included beneficiaries, GOI representatives, local and international 5

Please see Annex Two for a detailed schedule of the joint evaluation.

6 One CRS regional staff person was with the team for the first ten days. Another CRS regional staff person replaced them for the final part of the evaluation. 7 The ECB Project is a collaborative effort of the seven agencies of the Inter-agency Working Group on Emergency Capacity: CARE International, Catholic Relief Services, the International Rescue Committee, Mercy Corps, Oxfam GB, Save the Children, and World Vision International. For further information on ECB please see their website ww.ecbproject.org. 8 Villages visited were across the most severely earthquake affected sub-districts. This purposive sampling approach was used to ensure we spoke with people affected severely by the earthquake who had received sufficient relief assistance from these agencies to have strong views. 9

10 11 2

Please see Annex Three: References, for a list of the documents reviewed.

Please see Annex Four for a summary of those we spoke with in each village. Please see Annex Five for a list of key informants.

A Joint Evaluation of the Yogyakarta Earthquake Response July 2007

NGOs, and staff from the four agencies. They reviewed and amended the conclusions and made some recommendations to INGOs regarding future responses. Their views are captured in the body of this report.

A Joint Evaluation of the Yogyakarta Earthquake Response July 2007

Structure of the Report

The report starts with a summary of the affects of the earthquake and the response by all actors. It acknowledges that the success of the response was due to many factors and cannot be accredited to the efforts of any one particular agency. There were many actors who responded to the disaster. Section five describes the type and scale of activities carried out by these four agencies. Section six details the appropriateness of their activities while section seven focuses on the impact of their activities as perceived by beneficiaries and local leaders. Section eight explains peopleâ&#x20AC;&#x2122;s views of their level of recovery from the affects of the earthquake. This level of recovery is attributed to all those who responded to meet the needs of people affected by the earthquake. In section nine the agency efforts to be accountable to local people and local leaders during this emergency are described as well as the challenges that arise. Section ten sets out some of the good M&E practices employed by these agencies. Section eleven reviews the preparedness status of these agencies at the time of the Earthquake and their current preparedness to respond to a large-scale emergency in Indonesia. The final section concludes the report with a brief summary of the advantages and learning from this joint evaluation.

A Joint Evaluation of the Yogyakarta Earthquake Response July 2007

The Yogyakarta earthquake emergency context

The number of people affected by the earthquake was large. Over 5,700 people were killed and 27,000 injured. Over 300,000 homes were destroyed or damaged beyond repair and a further 200,000 houses suffered minor damage. This left an estimated 1.6million people homeless and an additional 1.1million affected12. People lost houses and belongings which in many cases were vital in earning a living. The widespread damage to personal property, businesses and infrastructure in the affected areas has increased the number of households (HH) below the poverty line in this part of Indonesia.

The most severely affected areas were heavily populated rural villages13 where most people lived in brick-walled houses which were close together and not earthquake resistant. This was a major reason for the extent of damage by a medium scale earthquake which left three times more people homeless than in Aceh Province as a result of the tsunami. The large scale damage to housing led to the earthquake being characterised as a â&#x20AC;&#x2DC;shelter-ledâ&#x20AC;&#x2122; emergency. The two worse affected districts were Bantul (the epicentre) in Yogyakarta Province and Klaten in Central Java Province. The extent of damage spread out from there to other districts and included damage across the districts to water and sanitation infrastructure, schools, health centres, roads and businesses. The total estimated damage and losses was calculated to be US$3.1billion.

The Government of Indonesia (GOI) set the provision of health assistance and emergency and transitional shelter as its response priorities. The GOI immediately began to organize food distribution and health services. The IASC agencies in Indonesia agreed to follow a cluster approach to coordinate the response by the humanitarian community. Ten clusters were established in June 2006. Cluster meetings were soon jointly chaired by a GOI representative and a member of the IASC. Many local and international NGOs participated actively in relevant cluster groups. As of May 2007 all coordination tasks related to earthquake recovery were handed over to provincial and district government with support from UNDPâ&#x20AC;&#x2122;s Economic Recovery Assistance program. The GOI was the largest responder deploying personnel from all government departments as well as the military. The GOI made it clear that it would provide funds and the delivery mechanism for permanent housing, asking others to focus on pressing emergency and recovery needs. Over the last year, 546 organisations provided assistance. Organisations included UN agencies, commercial organisations, donors, universities, military departments plus 248 national NGOs and 127 INGOs14. The emergency assistance provided by others complimented that of the GOI. Collectively this group of agencies is estimated to have provided US$175million (23% of all assistance).

The first responders though were the people affected by the earthquake, their neighbours, the families and friends of those affected. Help from neighbours was prevalent in this part of Java

Source: UNDP Cluster Approach One Year Review, p1.

UNDP: Cluster Approach One-Year Review, p. 3.

13 Average population density in Bantul District is 1,611 people per square kilometre and in Klaten it is 1,724

(Source: Bappenas of the Provincial and Local Governments of DIY, 2006).

A Joint Evaluation of the Yogyakarta Earthquake Response July 2007

Joint Evaluation FGD area

Joint Evaluation FGD Areas

where the social tradition of ‘gotong royong’ or communal labour exchange became widespread15. The efforts of local people were supported by truckloads of volunteers who came from surrounding cities and universities to help. Such action was re-enforced by provincial authorities who were quick to act and encourage people to clean up rubble and construct temporary shelter. The message from the GOI was clear; people should not become dependent on external aid. The level of assistance and the quick response by the GOI and others helped to mitigate suffering and bring the increasing levels of serious diarrhoea and other communicable diseases under control by July. The cooperation at local level between people, and between the GOI and other actors facilitated a quick and effective response that has assisted in the recovery of many of the people affected.

One year on, recovery is well underway in the majority of the affected villages. Much infrastructure is either already rebuilt or the GOI has taken clear responsibility for completing its reconstruction. Overall the response is considered a positive success. The reasons for this are many. A few are described in the box below. A longer list of conditions facilitating the success of this response are listed in Annex Six. Factors contributing to the success of the response

• The GOI mobilized people and resources quickly. • Local people helped one another through the tradition of ‘gotong royong’. • There were many responders including affected people, the GOI, local NGOs, INGOs, the UN, the private sector, neighbours, and individuals. • The response was quick and villages were accessible. • Agencies had competent staff with emergency skills that they deployed from Aceh.

However, gaps still remain. As of April 2007, 40,000 families still live in temporary shelter and some await promised funds for housing reconstruction from the government. The agricultural sector and the wider village economy have not yet fully recovered16. The need for further assistance to restore economic livelihoods was commonly expressed by most people we spoke with in seven out of the nine villages visited during this evaluation. It was also a main concern raised by participants during the workshop on June 20, 2007.

15 In the assessment done by Harjanto for CRS in September 2006, 37% of people reported that they had built their transitional shelters with support from neighbours via ‘gotong royong’. 16 Information in this section of the report is adapted from the UNDP paper ‘The Cluster Approach in Yogyakarta and Central Java: One Year Review’ (draft). The report provides details on the achievements of each cluster and the status of each sector as of May 2007. A Joint Evaluation of the Yogyakarta Earthquake Response July 2007

The emergency and recovery programs of CARE, CRS, SC, and WVI

All four agencies have extensive experience of working in Indonesia. Only CRS had experience of working in Java which is considered an economically better off area of Indonesia17. At the time of the earthquake, CRS had a team on the ground responding to a potential eruption of the Mount Merapi volcano. Save the Children and WVI also had assessment teams for Mount Merapi on the ground. CARE sent an emergency team in to respond to the earthquake on May 29, 2006. The agencies quickly began rapid assessments and redirected materials they had available for the Mount Merapi volcano crisis to earthquake-affected areas. Agencies joined the UN-coordinated shelter cluster immediately. CARE, CRS and WVI remained active members in the shelter cluster18. SC joined the education and child protection clusters.

All four agencies spoke of learning from emergency work in Aceh and based on this experience made greater efforts to work with local people and relevant government staff when making decisions. They partnered with local NGOs, supporting them to organise distributions at Many apply learning from Aceh locations close to the affected people. Their • The GOI encouraged local people to take the level of coordination with others, including lead and help themselves and their neighbours local communities, was appreciated and helped – interdependence was promoted. to ensure that assistance was provided where it • GOI policy was to provide a cash subsidy to was most needed. families who had lost their house. They asked that permanent housing not be provided to families as the experience from Aceh was that it was costly and took a lot of time. • Agencies mobilised people to help themselves. ‘The attitude was that people are not passive recipients’ . • Agencies did not use cash-for-work. Most families had help from neighbours and local volunteers. They were rapidly cleaning up rubble and putting up temporary shelters. • Based on their experience in Aceh, one agency established a better complaints handling mechanism.

All four agencies worked in more remote areas where damage was severe and other agencies were not yet doing much work. They all carried out NFI programs based on rapid assessments in severely affected sub-districts. They provided detailed information at cluster meetings and worked with UN, GOI and other NGOs to identify gaps and help fill them. They tried to compliment the efforts of affected people and the Indonesian government and so respected the humanitarian charter as set out in Sphere. This charter encourages NGOs to recognise and respect the primary role of people affected by calamities to meet their basic needs through their own efforts as well as the role of the state to provide assistance when people’s capacity to cope has been exceeded19 and for NGOs to define their role accordingly. 17 Indonesia is ranked 108 on UNDP’s 2006 Human Development Index, a medium HDI ranking. Life expectancy is

67years, adult literacy levels 90% and GDP per capita US$ 3,609.

18 19 8

Each of the agencies joined other cluster groups.

Sphere Humanitarian Charter and Minimum Standards in Disaster Response (2004). P.18.

A Joint Evaluation of the Yogyakarta Earthquake Response July 2007

Active coordination and a fast response by these agencies and many other actors resulted in temporary and transitional shelters being provided quickly. By August 2006, 80% of homeless people were reported to have received emergency shelter roofing materials and many were in temporary shelters though 40% of them were living in conditions below Sphere minimum standards20. Shelter provision was critical. Informants said ‘Once we had some form of shelter we could start to focus on rebuilding other aspects of our livelihoods’21. Except for CARE, coverage by each agency on NFI kits was relatively low but together they reached over 20% of affected families with various NFI kits including shelter, hygiene, clothing, household and clean-up kits22. WVI provided extensive support to reactivate health services at sub-district and village level ensuring access to basic health and MCH services for 300,000 people. CRS, SC and WVI implemented activities to protect children and reduce their trauma. SC and WVI provided support for elementary schools to ensure they could restart classes by mid-July 2006 and operate more effectively thereafter. CARE, CRS and WVI carried out transitional or permanent shelter programs for families who had lost their homes and were prioritised as most in need of housing assistance. CARE and CRS supported water and sanitation activities. Three of the agencies were satisfied with the scale of their overall response and felt that it enabled them to ensure a high quality response. They said their response was adequate given resources at their disposal and their allocated roles, as agreed to at cluster meetings. One agency said they wanted to do more but had budgetary limitations. Budgets of each agency were between US$ 2.6 to 3.1million. Program Coverage by the Four Agencies Activity

CARE

CRS

WVI

NFI Kit Distribution

50,000 HH

7,500 HH

9,954HH

9,000 HH

Support to Elementary schools

13,436 children 2,400 children

% Reached

Children’s activities Shelter

Water/sanitation

13.4% NA

307 HH

1,050 HH

300 children 300HH

300HH

2.7%

2,900 children NA

2.4%

3,300 children 315 HH NA

WVI completed its Yogyakarta emergency response and recovery programs at the end of May 2007. CARE will complete its program in August 2007. CRS will finish its activities in March 2008 and SC by June 2008. All of the agencies acknowledge that this part of Java is relatively well off and that their agency resources can now best be used in poorer, more vulnerable areas of Indonesia23. 20

OCHA Field Situation Report No 21: Indonesia Earthquake Update, 10 August 2006, p.5.

21 For more details on the response to the emergency by all actors please see Annex Seven: Brief chronology of the

response to the Yogyakarta earthquake.

22 Coverage figures are based on an estimated 1.6million homeless people. The average household size in this part

of Indonesia is 4.3 people. Using these two figures the number of households needing immediate support is estimated

at 372,000HH.

Summaries of the findings of each agency are in Annex Eight.

A Joint Evaluation of the Yogyakarta Earthquake Response July 2007

The response by these agencies was largely considered appropriate

At village level people and officials were asked what was done well by these agencies and what they want them to do differently next time. This section summarises the responses of people 24 and also includes the views of staff and information from documents. Each activity section starts with what agencies did well in relation to an activity. Concerns are noted at the end of each activity section.

Based on what people said, the majority of activities carried out by these agencies were appropriate – meeting people’s immediate needs and respecting local culture25. These four agencies were recognized locally as arriving at the very start of the crisis and staying on to work. District government officials know the agencies will complete their programs in this part of Java soon and these officials said ‘we would welcome these agencies back should a similar type of emergency occur in future’ . NFIs Due to the lapse in time and the number of different agencies that had provided NFIs, communities had some difficulty in recalling which agency had given what NFI kits. However, in eight of the nine villages either the men, women, children or leaders said that the quality of the items provided by these INGOs was good and met their needs26. Distribution processes were said to be simple, fast and fair and based on the results of rapid assessments. NFI distributions were carried out at locations close to where affected people lived and distribution methods re-enforced the local selfhelp tradition of ‘gotong royong’. One agency provided vouchers for food through a market-based relief (MBR) scheme. This voucher approach was considered appropriate as it helped local businesses to recover (the voucher approach is discussed in more detail below).

Concerns about NFI activities were raised in a few villages and by beneficiaries and NGOs during the multi-stakeholder event on June 20th, 2006. The concerns related to the emergency response as a whole and all the actors involved, and not specifically these four agencies. They were: 1) there was oversupply of NFI materials in some villages and undersupply in others and 2) assessments were done by different agencies in the same locations. Some recommended that agencies start doing

Summaries of views heard in each village are provided in Annex Nine.

25 The definition of appropriateness is as defined in the OECD/DAC 1999 ‘Guidance for Evaluating Humanitarian

Assistance in Complex Emergencies’, p.22.

26 These views were backed up by evidence from internal reviews done by the agencies on their NFI activities.

The most complete assessment on NFI’s was done by CRS. This was an independent review. FGDs were carried out in a sample of 13 out of the 200 villages that were randomly selected from the list of villages where CIMO agencies operated. Individual interviews were done in a sample of 90 of these villages. 10

A Joint Evaluation of the Yogyakarta Earthquake Response July 2007

joint assessments and at least share assessment information quickly and 27 improve coordination with the government to ensure that NFI kits are distributed across areas and between households equitably. The issue of equity is exemplified in the box below, by the way a village dealt with the receipt of food vouchers for only 75% of families. The importance of fairness was strongly expressed by children who said â&#x20AC;&#x2DC;distribution needs to be fair so conflict does not occur and some families go withoutâ&#x20AC;&#x2122;. During discussions in one village, informants said tools provided in clean up kits were of poor quality. In another village, informants said they had difficulty in ensuring tools in clean up kits were returned in good condition and unbroken. Otherwise, informants said that NFI programs implemented by these agencies were done well.

MBR vouchers wer eree only only pr pro ovided for 75% of families rather than for all. The community took it upon themselv themselves to redistribute edistribute the vouchers in a way the they y considered consider ed fair and in or order der to pr preevent conflict though though this caused str stress. ess.

Children Activities

Activities carried out by CRS, SC and WVI with children were appreciated by the children themselves,

their parents and village and district leaders. Many informants gave such activities high marks.

They were said to be fun, educative and made the children happy. Parents said that such childrens

programs allowed them to get on with the work of rebuilding their houses as they knew their

children were safe and actively involved in the

activities.

Save the Children Children Safe Safe Play Play Areas Areas The main concern expressed by parents and children

was the sustainability of such activities. While in

many villages, people had publicly said they would

be responsible for continuing childrenâ&#x20AC;&#x2122;s activities, few were confident that these activities would be sustained after the departure of the agency. Others wondered why such programs were only provided for children. Parents, LNGOs and GOI officials asked that in future emergency responses, trauma-healing programs be provided for adults and teachers as well.

SC established 50 saf safee pla play y ar areas. eas. Initial assessments wer eree carried out with community repr epresentati esentativ ves to identify sites sit es and select 150 volunteers olunteers for training tr aining along with staff of fiv five LNGOs on trauma tr auma counselling and child pr prot otection. ection. Centres Centres ran fiv five da day ys a week, based on a schedule agr agreed eed betw between een the children children and volunteers. olunteers. Children Children needing special attention att ention wer eree referr eferred ed to the corr correct ect services in MOH.

Water and sanitation activities

CARE implemented water and sanitation activities that assisted villagers in obtaining clean drinking water. Such work was welcomed since water sources everywhere were polluted for the first three months following the earthquake. Jerry cans, equipment and education on filtering and boiling water were provided. Help and advice to clean up rivers and protect them from pollution were also implemented by CRS. Health and hygiene promotion activities were continued by both agencies after the immediate relief phase. 27 The ECB is doing a Data Resource Collection Project that aims to develop common assessment tools that could facilitate a joint assessment process. A Joint Evaluation of the Yogyakarta Earthquake Response July 2007

One of the agencies provided chlorine to purify drinking water. Households in four out of the nine villages visited during the evaluation had received the solution. Women and children said the solution was used for washing clothes. They did not like its smell or taste. Support to government education and health services

SC and WVI provided support to get elementary schools up and running and ensure they could operate effectively. Temporary shelters, furniture, books and other school supplies were provided. WVI constructed six new schools and SC trained 760 teachers in trauma counselling, child protection and planning. Such support was appreciated but educational activities of both agencies received very limited mention during the interviews with men, women, children or leaders, which could be due to the length of time which had passed since implementation of the activities. However, many said that support to schools did ensure that children did go back to school. WVI provided extensive support to reactivate sub-district and district level health services and such support was appreciated by all and is discussed in more detail under the impact section below. There were no concerns raised about support to government health and education services. Transitional and permanent housing

The transitional and permanent shelter programs implemented by CARE, CRS and WVI were much appreciated and in most cases carried out in ways that targeted those considered most in need by local people. Two agencies followed government policy in carrying out transitional shelter programs and involved families in reconstructing their houses28. Families are already sleeping29 in these houses even though construction is not complete. CARE used a voucher system whereby families could obtain building materials for housing construction from local vendors. This approach was welcomed as it allowed people to control the quality of building materials themselves, as well as support local businesses.

All three agencies carried out shelter surveys with community leaders and encouraged village groups to select beneficiaries themselves based on agreed criteria. In most cases, discussions to select beneficiaries were public with village leaders, agency and local group representatives present. This level of transparency was valued.

28 Government policy was that NGOs should not provide permanent housing. The GOI committed to providing a cash subsidy for families to reconstruct their homes. They asked NGOs to provide tools, technical support and transitional shelter support and encourage families to be responsible for reconstruction themselves. 29 Participants in the June 20th were adamant that were only sleeping in these houses because they were not complete and in most cases had no kitchen or furniture so they would not say they were living there. 12

A Joint Evaluation of the Yogyakarta Earthquake Response July 2007

CRS Transitional Housing The transitional housing model used by CRS was rated as one of the best by local people. A lightweight metal frame including a roof and cement floor were provided and constructed over a two-day period. Families were expected to assist with putting up the frame and completing the walls, windows and other parts of the house themselves. This transitional shelter can be converted into a permanent house easily. Recipients said it was earthquake resistant, went up fast, was easy to construct, and that the material and design was of good quality. Other families in the neighbourhood were said to be using the design to construct similar houses. People said they felt safe and comfortable in the new houses especially as â&#x20AC;&#x2DC;the roof acts as an alarm system as you hear the rain and feel an earthquakeâ&#x20AC;&#x2122; . However, others mentioned that with a metal roof the house is very hot.

In two villages, discussions by the agency at the beginning of the shelter program were done well. However, follow up monitoring and meetings were carried out mostly with village leaders. Men and women said that criteria for housing support were no longer clear. As a result, selection of recipients was manipulated. They asked that information be provided by the agency on a regular basis to prevent misinformation, and that monitoring be carried out thoroughly by agencies so that processes remain fair.

The three agencies used different approaches for working with villages on transitional and permanent housing. They each had a different housing design, as did other INGOs. As Indonesia is prone to natural disasters that affect shelter, a deeper learning review with GOI participation of shelter programs implemented by all INGOs in response to this earthquake would be beneficial. It could assist agencies and the GOI to make better decisions on both the process and housing models to use in future emergency responses. Recommendations from stakeholders on program activities:

a) Continue to do the type of activities carried out in this response. Carry out assessments to ensure aid meets the needs of affected people and to agree with them procedures for distribution and beneficiary selection. Provide good quality items, distribute goods quickly and follow simple procedures.

b) Better coordinate NFI programs between all responders and stakeholders to ensure equal distribution across areas and application of distribution methods that promote fairness. Monitor the assistance well ensuring staff participate in distributions and beneficiary selection processes. c) Carry out assessments jointly so that the same information is not collected a number of times in the same location by different organisations. d) Start recovery activities earlier e.g. transitional and permanent housing plus activities to restore livelihoods.

e) Complete a study on the transitional and permanent housing designs and approaches used by these four agencies, other INGOs and the GOI in Yogyakarta to draw out learning that can be applied in Indonesia when responding to future emergencies with a high shelter need.

A Joint Evaluation of the Yogyakarta Earthquake Response July 2007

Agency activities contributed to impacts

For the purposes of this evaluation, impact is defined as the difference made by the activities carried out30. To identify impacts local people involved in the work of these agencies were asked: ‘what difference did the activities implemented by these agencies make’? This question was asked of all informants both in FGDs and SSIs. Their responses are summarised below.

The four agencies contributed to the impacts or immediate effects described below. However, agencies were only one among many delivering NFIs and shelter in any village. It would be difficult to credit them with bringing about some of the impacts independently, but they definitely made a positive contribution to producing results. All informants said that the NFI programs helped meet the basic survival needs of affected people. These programs were said to have lifted people’s spirits and to have helped communities to recover faster.

The children’s programs supported by CRS, SC and WVI helped reduce children’s trauma and encouraged them to return to formal schools. Children said the activities had helped increase their self-esteem and confidence. The support provided to elementary schools helped ensure schools could restart classes in mid-July 2006. In villages where new schools were built, they are of better quality than the preearthquake buildings. The children have a better environment to study in ‘a better building with Help to Restart Government Health Services good lighting and more textbooks are available to World Vision re-equipped and provided tents children’ . According to the heads of sub-district to ensure that 12 puskesmas could restart their education departments, teachers trained by services. Six of these ‘puskesmas’ later received SC now understand the new GOI education support to rebuild their structures. 665 curriculum and are able to plan their work more ‘posyandus’ were re- equipped to provide MCH effectively. services and run supplementary feeding for Support to the health sector by WVI was credited by health officials at district and sub-district levels and by local people with ensuring that local people had access to basic health services quickly. The work they did is described in the box31.

People said the water and sanitation activities improved their access to clean water.

children under the age of five. Many health staff from these facilities were trained in physical rehabilitation and dealing with trauma. Women informants said they appreciated the support to posyandus as ‘they promoted exclusive breastfeeding by mothers and ran supplementary feeding programs for children which prevented malnutrition of children under five years of age’. Health officials said, ‘The recovery for the society was faster because WV supported health staff to get back to work faster and more efficiently’.

30 This definition is from the Impact Measurement and Accountability in Emergencies: The Good Enough Guide (see page 4 of the Guide). It is also based on the definition of impact as set out in the OECD/DAC 1999 guidance paper on evaluating emergency response. 31 Each sub-district has a puskesma or basic health care centre. Posyandus are located at village level. They provide MCH services. 14

A Joint Evaluation of the Yogyakarta Earthquake Response July 2007

The shelter programs were mostly targeted to families that had lost a house in the earthquake and were considered economically in need of support. Most of these families had received the GOI cash subsidy for housing reconstruction. Together with the help from the agencies they have started to rebuild their home. These families said ‘we now have a house and are able to start concentrating on work and on restoring our livelihoods’. In most cases families knew that the transitional houses built with the support of these agencies were earthquake resistant. Men and women said they felt safer and more comfortable in these houses. In addition, people said that because of the housing programs many people in the surrounding areas now know how to build earthquake resistant houses and others are beginning to use the housing models designed by at least two of the agencies. The way CARE and CRS worked with communities was credited with ‘increasing cooperation and solidarity in this community’ .

A Joint Evaluation of the Yogyakarta Earthquake Response July 2007

Activities implemented by these agencies and other actors helped people and their communities to recover

Teams walked with leaders through each of the villages and asked them to describe the effects of the earthquake and what had taken place since. Leaders were asked to estimate the level of recovery since the earthquake and to describe those that had contributed to recovery. This is what we saw and heard.

In the nine villages visited, recovery from the effects of the earthquake are well underway. The reconstruction of houses was visible everywhere. A good number of houses are rebuilt, other houses are well on the way to completion and others are still only frames with much work to be done. Roads are being repaired. There are many government offices and schools already reconstructed by the GOI, donors, the private sector and NGOs. Wells are clean and water sources back to normal.

In two villages local leaders estimated that overall recovery was at 90%. In the other seven villages, local leaders estimated that recovery was between 30-50% and this was reconfirmed during discussions with beneficiaries on June 20th. Most participants at the June 20th event said that housing reconstruction in their villages was less than 50% and only in one village was housing said to be back to pre-earthquake status. Similar percents of 30-50% were given for livelihood recovery. Informants said families do not have enough capital to buy fertilizer, seeds or goods for petty trading activities. They felt that the GOI is not supporting economic recovery programs.

‘There is only 30% recovery in this village because there is little economic activity going on. Everyone is focused on fixing their house and this needs funds but the government will only provide 20 million rupiah and only 9 .4 million rupiyah has been distributed per household so far’ .

Teachers and village leaders plus participants at the June 20th event estimated recovery of elementary schools to pre-earthquake conditions at 90%. Schools are rebuilt and repaired though some still lack furniture. Children are back in school. In addition, children said their trauma was reduced and parents confirmed this during FGDs. Both children and women said ‘we suffer less from nightmares about the earthquake’ . Water sources for drinking purposes was said to have recovered up to 90% though sanitation remains a problem as many households still do not have latrines. Villagers and leaders estimated that recovery of latrines was only at 50%.

When asked to estimate the contribution of various responders to recovery, leaders said that support from the GOI was between 50-60%, that from NGOs 25-30% and support from others 10%. The biggest gap remaining in the majority of villages is economic restoration, more specifically the recovery of livelihoods. This was expressed in seven out of the nine villages visited and again strongly expressed during the multi-stakeholder event on June 20th. 16

A Joint Evaluation of the Yogyakarta Earthquake Response July 2007

Recommendations on recovery32 a) Provide more support to HHs and communities to restore their economic livelihoods.

b) Learn from work carried out in other countries prone to sudden onset emergencies to identify appropriate economic livelihood activities to support in future emergencies in Indonesia.

32 Bangladesh is prone to sudden onset emergencies. Some of the agencies participating in this JE operate welldesigned disaster loan programs in Bangladesh. Such programs capitalise micro-finance institutions to extend loan periods at low interest or concessionary rates when disaster affects communities. Such an approach is only possible when a micro-finance institution exists before the crisis. Would it have worked in Yogyakarta especially as villagers were asking for loans and not for grants? A Joint Evaluation of the Yogyakarta Earthquake Response July 2007

Accountability to local people was mostly well done

All informants were asked how the agencies involved them in activities and kept them informed. This is what we heard. All four agencies worked with local leaders involving them in assessments, planning and decision-making. Meetings were held frequently enough to keep leaders informed and aware of the status of activities. Most of the agencies worked in ways that encouraged local people to rely on one another and to make their own decisions about who to prioritise as beneficiaries. Decision-making processes were bottom-up, although women in five different villages said they would have liked to have been more involved in decisions and had better access to information about activities carried out by the participating agencies. Women in these villages said they had to rely on their husbands and local leaders to find out what was going on.

A Village Cadr Cadre e System stem that Ensur Ensured ed Women’s omen’s Inv In vol olv vement

CARE or orgganised gr groups oups in villages wher heree they the y wor ork ked and ask asked ed each gr group oup in a neighbour neighbourhood hood clust cluster er to select leaders: one man and one woman. These cadr cadres es assessed beneficiary needs in their clust cluster er.. Women said they the y could get complet completee information information about the pr progr ogram am from from cadr cadres es and that there there was nev ne ver an any y missing information. information. CARE staff met with cadr cadres es and village leaders once a week to discuss acti activities vities and mak makee decisions. All pr project oject documents wer eree sho shown wn to the cadr cadres. es. The leaders, men and women interview interviewed ed pr praised aised the method of wor orking king with cadr cadres, es, claiming it to ha hav ve ‘…str strengt engthened hened links and cooperation ooperation betw between people and to hav havee enc encour ourag aged ed mor moree mutual assistanc assistancee than than befor before’ e’.

All four agencies respected the government’s request to work with the local administrative structure33. Rapid assessments and NFI distributions were organised through this structure and local NGOs were asked to support communities in these processes. The structure worked relatively well in most places. In seven out of the nine villages people regarded the processes used by the agencies as fair and jealousy or conflict was generally avoided. This view of how well agencies work with local people was however challenged by village level participants and LNGOs during the June 20th workshop. Their concerns are in the following box.

A final concern related to procedures. Proposals were requested by two agencies for specific types of support. Informants said proposal writing was complicated and they would prefer simpler processes.

In seven different villages “fairness” was a word used by women, children, men and leaders to describe how activities were carried out and should be carried out in the future. Regular information, on-going monitoring and transparent beneficiary selection processes were perceived as critical in maintaining fairness and to varying degrees these agencies applied such processes. Such processes were said to help ensure that goods were provided in ways that encouraged ‘fair or equal distribution’ and ‘avoided jealousies and conflict’. ‘Aid should be distributed equally because those who do not get assistance like to complain . Agencies should ensure there is a complaints system’ 33 The GOI structure starts with neighbourhood clusters or RTs with an estimated 20-30HHs. This has an elected leader. Ten to twenty RTs make up a RW which has an appointed leader. There are three RWs in a sub- village. 18

A Joint Evaluation of the Yogyakarta Earthquake Response July 2007

(children’s focus group). Women and men in other villages also recommended that a complaints mechanism be established by NGOs.

CRS has a formal complaints mechanism34 and CARE had one in the early months of the emergency. CRS has informed beneficiaries and leaders that they can send an SMS message to the agency’s M&E officer or their shelter officer. Cell phone numbers of the respected officers were provided to village leaders and groups. All complaints are discussed at weekly program staff meetings and appropriate and necessary action is taken. Twenty complaints were received and addressed immediately.

‘NGOs come and provide goods for fifty families when there are a hundred families in this village . We divided the goods equally but then the most affected families don’t have enough . The GOI set criteria and said distribution of goods does not have to be equal but fair and based on need . This helped us in making decisions about distribution . But every choice created a problem; conflicts arose and stress was created . The earthquake was a test from God but the support from NGOs has proven to be a bigger one . Responding to an emergency is not just about providing assistance but also about communicating with us and motivating and providing us psychological support’ .

Recommendations on local accountability

a) Provide information to the wider community: men, women, beneficiaries and nonbeneficiaries, on a regular basis so people are aware of the work being conducted by the agency with them so reducing opportunities for misuse of information. b) Establish a complaints system that clearly defines how people can complain about the work being carried out by an agency if they need to do so. c) From the start, involve women as well as men in planning, implementation and evaluation of programs.

The complaints procedure was adapted from one tried in Aceh.

A Joint Evaluation of the Yogyakarta Earthquake Response July 2007

10.

Some good practice on monitoring and evaluation (M&E) is found

All four agencies had log frames for their projects with good input and output data. CRS had some clear and simple outcome indicators. They mentioned they had used the ‘Impact and Accountability in Emergencies: The Good Enough Guide’ to help them define their M&E plans. Three of the agencies had monitoring and evaluation staff in their Yogyakarta project teams and all four agencies conducted a review of their emergency or post emergency programs and produced review reports.

Each agency exhibited some good practice on M&E. CRS and SC conducted reviews of their NFI programs with CRS completing the analysis in October 2006 and sharing the report widely in the shelter cluster meetings. Villages were randomly selected, the review was carried out successfully and the quality of the report was excellent. It is assumed that the report assisted others to understand the usefulness of the NFI assistance. WVI completed an independent evaluation of their post earthquake assistance program and shared the results widely. CARE held an externally facilitated two-day workshop with all local stakeholders to confirm what program activities were successful and what could be improved in the future. SC trained and supported children to conduct a program review. The Child-Led Review

Fifty-one children between the ages of 8-16 were trained by Save the Children to conduct a review.

They volunteered from two schools. The children were briefed on the program and its three main objectives and trained in collecting data, data analysis and in presenting findings. The children defined the indicators and designed pictorial tools for discussions with other children, parents and teachers to find out what was liked and what was not liked and why. Based on the data, the children drew conclusions on the impact of the program. Their conclusion was that the activities implemented by SC had encouraged children to go to school, despite the difficult conditions, and that they had helped reduce children’s trauma.

Based on CRS’s work on outcome indicators and the information provided by informants on the difference activities made, the following indicators are suggested for use in similar types of emergency programs. Verification of results would use the approach as set out in the ‘Good Enough Guide’ and applied in this joint evaluation. This approach relies on qualitative assessment methods and asking a sufficient number of participants in a program of their views on its appropriateness and effects.

A Joint Evaluation of the Yogyakarta Earthquake Response July 2007

Suggested Impact Indicators for emergency responses to natural disasters Activity

Impact Indicator

NFI Distribution

• Basic survival needs met • Goods provided in ways considered fair by local people

Child protection activities

• Trauma reduced • Children say their happiness has returned

Support to restart health and education services

Services operating at the same level they were at before the disaster

Watsan

People have access to clean water

Shelter

Shelters constructed and people are living in them

A Joint Evaluation of the Yogyakarta Earthquake Response July 2007

11.

Agencies were prepared to respond because of Mount Merapi

An objective of the evaluation was to assess the level of emergency preparedness of these four agencies. The evaluators were asked to identify examples of good practice and critical gaps, and make recommendations for improving emergency preparedness and response in the future. To achieve this, agencies were asked through interviews and written submissions to reply to the following questions: •

Did the agency have an emergency preparedness or contingency plan in place prior to the May 27th Earthquake? If so what did the plan entail?

•

Does the agency have specific standards and operational procedures during an emergency response and what are they?

•

What worked well and what could have been done better?

•

What resources, material, financial and human does the agency have in place to respond to a humanitarian emergency?

The agencies were also asked to outline actions, if any, taken by their organization over the last year to improve their level of preparedness, and to give one or two recommendations or steps they would like to see their agency take to improve emergency preparedness. A summary of responses to these questions is provided in the table in Annex Ten. The key findings are summarized below. Emergency Planning;

Prior to the earthquake 3 out of the 4 agencies did not have emergency preparedness plans in place to respond to the affects of an earthquake in Indonesia. Emergency Standards and Operating Procedures;

Three of the four agencies have adapted specific emergency operating procedures that are used during an emergency response. Resources Available;

Financial: - Each of the agency’s country offices can access agency funds to support an initial response to an emergency. The amount of funds available and the conditions for access vary from agency to agency. Materials: Three out of the four agencies have propositioned essential NFIs in various parts of Indonesia.

Human: All of the agencies have rosters of international experts or Emergency Response Teams (ERTs) that can be called upon to support a country program during an emergency response. All of the agencies reported that they were able to call upon experienced national staff (many from 22

A Joint Evaluation of the Yogyakarta Earthquake Response July 2007

Aceh) to support earthquake response activities. Two of the agencies have national staff emergency rosters. One of these specifically includes a mechanism to fill gaps of deployed staff. Examples of good practice:

Access to funds to support initial assessments and response activities allowed each of the agencies to quickly mobilize and respond to the Yogyakarta Earthquake. The pre-positioning of NFIs allowed the agencies to quickly meet the needs of affected people. It also meant that for the initial distributions they were not paying post-emergency inflated NFI prices. Pre-positioning also helped one agency to develop relationships with local vendors and develop knowledge of prices, quality, availability of items, and transportation options prior to the emergency. Being able to call on national staff with experience of working in emergencies was cited by all agencies as key to the success of the agenciesâ&#x20AC;&#x2122; response in Yogyakarta. Critical gaps

During the evaluation, communities and government officials praised INGOs for how quickly they mobilized and responded to the earthquake. In part, this perceived preparedness was due to a possible eruption of Mount Merapi Volcano. Agencies were gearing up to respond to the high alert and had assessment teams on the ground and NFIs on route from Yogyakarta. The speed of the response to the earthquake did not therefore accurately reflect their ability to respond rapidly to a sudden onset emergency such as an earthquake or tsunami. Through out the evaluation process staff referred to learning from their emergency response programs in Aceh. However, the fact that only one of the four agencies had an emergency preparedness plan in place at the time of the earthquake (two and a half years after the Tsunami) indicates that the lessons learned from Aceh have yet to be institutionalised in three of the agencies. Rather than being institutionalised, these lessons from Aceh came with the people redeployed from Aceh to Yogyakarta.

While being able to call on experienced national staff from Aceh was cited as key in the success of the response, the need to adequately train and supervise new recruits was also cited as important and crucial. At least one agency said it would invest more in new staff capacity development in the event of a future disaster. Notable from interviews with field-based staff in Yogyakarta was their lack of clarity regarding agency preparedness plans. This suggests that more needs to be done to disseminate an understanding of emergency preparedness plans down to field staff. Since the Yogyakarta earthquake, the three agencies that did not have preparedness plans in place, have, to varying degrees, taken steps to improve their emergency preparedness. Nonetheless much still needs, and can be done, to position the agencies to ensure a more effective and coordinated response to future emergencies.

A Joint Evaluation of the Yogyakarta Earthquake Response July 2007

These agencies worked together on other ECB activities35. The ECB provides a platform for greater interagency cooperation in emergency response. It is a mechanism for coordinating and developing strategies for joint assessments, shared pre-positioning of NFIs and joint implementation strategies and evaluations. Will agencies use the relationships they have built during the joint evaluation to do more emergency work together? The recommendations below come from comments made by staff and the conclusions which identify existing gaps. Recommendations on emergency preparedness

a) Complete country emergency preparedness and contingency plans and ensure that all staff are aware of their existence and content. This could be done through country program emergency response simulations, followed by an interagency simulation once all agencies are satisfied with their own plans.

b) Create a joint database on the capacity of different agencies regarding the location and type of pre-positioned NFIs. Examine the feasibility of holding joint stock in shared warehousing. c) Better prepare staff that do not have emergency experience; ensuring that new hires receive appropriate training and supervision.

35 This group of agencies had worked together in the ECB project on DRR during 2005 and 2006. Over this time, much trust was developed between the agencies. 24

A Joint Evaluation of the Yogyakarta Earthquake Response July 2007

12.

The joint evaluation process had a number of advantages

CRS planned a joint evaluation in their Yogyakarta emergency strategy written in July 2006. The decision was influenced by the ECB Project initiative on accountability and impact measurement that had supported joint evaluations between INGOs in Niger, Guatemala and the tsunami affected countries during 2005 and 2006. Planning for the Yogyakarta JE got underway in January 2007 when CRS and SC jointly developed a terms of reference for the evaluation. A few months later, CARE and World Vision confirmed their interest in participating and a steering committee was formed with representatives from each of the four agencies.

The evaluation was managed by the steering committee. CRS agreed to take responsibility for overall management of the evaluation and to chair the steering committee. As the lead agency, CRS hired the evaluation team; gathered key documents from each agency, the UN and the GOI on the emergency response and sent them to the evaluation team; negotiated the schedule of activities and the budget; organised logistics; and led discussions on methods with the lead evaluator. All steering committee members jointly agreed to major decisions. Costs of carrying out the evaluation were shared between the agencies and ECB. What went well

In general, the joint evaluation process went well. There was effective inter-agency communication between staff with a high level of trust amongst those involved. Communication infrastructure was adequate with reliable access to telephones, e-mail, instant messenger service, and geographic proximity to all the participating agencies making face-to-face meetings relatively easy.

The lead agency carried out its vital management responsibilities well. The steering committee chair was successful in securing the commitment and trust of his colleagues. CRS staff did a good job in organising all evaluation logistics, hosting the evaluation team and providing overall guidance to the evaluation team on the context of the humanitarian response, and the applicability of methods and questions to explore in the field. Each of the participating agencies had sufficient monitoring and evaluation capacity, with three out of the four having M&E officers within their Yogyakarta Emergency Response Teams. They helped create openness within their organization to this evaluation, ensure rapid sharing of relevant documents and provided good advice to the JE team on methods. The agencies have benefited from a supra structure that is supportive of joint evaluations and collaboration in general. In particular, agency staff in CRS at Jakarta and headquarters level provided

A Joint Evaluation of the Yogyakarta Earthquake Response July 2007

strong encouragement and support to staff in Yogyakarta to lead the JE process. The culture of collaboration promoted by the ECB in Indonesia on disaster risk reduction between these agencies encouraged them to try a joint evaluation in order to continue to learn from one another. Advantages of a joint evaluation approach

There was a significant amount of learning and relationship building between the agencies involved in the process. The sharing of documentation and discussion when preparing for the evaluation provided an opportunity for steering committee members and M&E staff to learn about other agencies’ programs and approaches. Relationships among these individuals were strengthened. The agencies expect that findings from this evaluation, being more holistic than an individual evaluation, will make a useful contribution to the humanitarian community’s understanding of emergency work in Indonesia and beyond. It is also demonstrates their accountability since they have subjected themselves to the scrutiny of their peers, local people and government during this evaluation. It is possible to use the findings for advocacy purposes as the report provides a perspective on the overall results of the response by a number of agencies. This may give the report more authority as its conclusions originate from a group of agencies that have assessed the larger context and tried to understand how their responses have affected people and communities. Learning from the process

• A joint evaluation is more time-consuming in terms of planning and management than an individual one. It takes time to get agreements on decisions and changes between the agencies. The demands on staff time of the lead agency are particularly high. • The evaluation team had to visit sufficient locations where each of the four different agencies worked. This left less time to focus on an individual agency and assess their activities in-depth. However, the findings are broad and provide a perspective on the effectiveness of emergency work conducted by all agencies.

• The agencies wanted an independent team that would be seen as objective. An independent evaluator was hired for this purpose. A regional CRS staff member with experience in emergency was assigned to the team. Local facilitators, note takers and translators were hired. While their hard work was invaluable and they fulfilled their roles excellently, local team members were relatively new to NGO work. The limited amount of emergency program experience on the team meant that specific sector areas of work were not assessed in-depth36.

• In-country agency staff were not assigned to be part of the evaluation team. This limited the level of inter-agency learning between the participating agencies and the depth of sectoral analysis. In addition, a national consultant was not hired. This meant that the JE team had to depend heavily on the lead agency for advice on methods and the larger context. 36 Ideally, a JE team will have four experienced people: the team leader and three emergency staff members from the various participating agencies, with different technical specialities. Members of the ECB interagency standing team created to spread good practice on a Good Enough Approach to M&E in emergencies would be ideal. Each member needs to stay for the duration of the JE. 26

A Joint Evaluation of the Yogyakarta Earthquake Response July 2007

â&#x20AC;˘ â&#x20AC;˘

A JE should be carried out by one or two months after an emergency program ends in order to capture the necessary information from the affected people on what was done and by who, and what went well and what did not before the details of the response are forgotten. Other INGOs have noted their interest in being part of such joint evaluations and the four agencies agreed that other INGOs should have been involved. There are great benefits to doing an evaluation of the work of all INGOs as many of the emergency response activities overlap and are similar. A broader and more encompassing JE would take more time to plan and objectives would have to be limited in order to carry out an effective evaluation.

Recommendations on joint evaluations

a) Once joint assessments are complete, plan for a joint evaluation to start within one to two months of emergency program completion. Use the Good Enough Guide to inform the JE process. b) Commit enough experienced program staff for the entire period of the JE so that the team has sufficient experience for an in-depth review of a few sector specific activities.

A Joint Evaluation of the Yogyakarta Earthquake Response July 2007

Above: Pauline Wilson, team leader of JE presented the draft finding to the program staff of CRS, CARE, Save the Children and World Vision Indonesia. Below: Nining and Ella, facilitating Women group during FGD process at CARE project village.

Above: Male FGD process at one of the Save the Children project village. Below: Childrens at the â&#x20AC;&#x153;Safe Play Areaâ&#x20AC;? at Save the Children SPA tent.

Above: Ryan Russell, together with Pauline and Donal, facilitating field team after field work for data compilation, probbing, review and data crosscheck at CRS Yogyakarta office. Below: Children at WVI project area exiting with the visit of â&#x20AC;&#x153;buleâ&#x20AC;? [bulai] (bahasa slank term for all foreigner) in their village.

Above: Joint evaluation sharing results presented at Hotel in Yogyakarta attended by more than 100 participants from beneficiary, Government, university, international and national NGOs. Below: Transsect walk and interview carried out by evaluator team at one of CRS project village.

13. Annexes Annex One:

Terms of Reference for the joint evaluation

Annex Two:

Schedule of the joint evaluation

Annex Four:

Summary of those we spoke with in each village

Annex Three: References Annex Five: Annex Six:

Key informants

Conditions facilitating success of the response

Annex Seven: Brief chronology of the response to the Yogyakarta earthquake Annex Eight:

Summary of findings of each agency

Annex Ten:

Summary table on emergency preparedness of each agency

Annex Nine:

Summary of findings from each village

A Joint Evaluation of the Yogyakarta Earthquake Response July 2007

Annex One:

Terms of Reference for the joint evaluation

TOR FOR JOINT EVALUATION OF NGO RESPONSES TO THE 2006 YOGYAKARTA EARTHQUAKE (final ver. June 20th, 2007)37

1. Background On May 27th 2006 an earthquake measuring 5.9 on the Richter scale hit Yogyakarta and its surrounding areas. Some 6,000 lives were lost and a further 1,600,000 made homeless by the quake and the aftershocks that occurred. International and local NGOs responded very quickly to the needs of those affected by the quake providing shelter kits and non-food items and setting up emergency response programmes covering all the normal programming areas. At the same time the area was poised for an eruption of Mount Merapi, a volcano only a few miles from the earthquake zone. To date this eruption has not occurred and yet the area remains on alert.

Post-crisis review and reflection activities, such as independent evaluations, Lessons-Learned Workshops and After Action Reviews (AARs), have been shown to promote better quality humanitarian programming by providing practical learning opportunities and increasing accountability of participating agencies. Attention is now being given to interagency evaluations and AARs as a way to strengthen accountability and learning through peer review, and to minimize the impact of such activities on those affected by the disaster. While being designed to measure impact of agency interventions and ensure accountability, resulting lessons-learned are used for organisational learning and policy development. In March 2005, the Interagency Working Group composed of CARE International, Catholic Relief Services, Save the Children, International Rescue Committee, Mercy Corps, Oxfam GB and World Vision International launched a two-year “Emergency Capacity Building” project funded by the Bill & Melinda Gates Foundation designed to strengthen humanitarian response. One of the focus areas for this project is to improve agency accountability and improve our ability to measure the impact of our interventions. To help achieve this, it was decided to support opportunities for joint learning and accountability activities that would promote institutional learning at both a country and institutional level. Wherever feasible, it is planned that such exercises are designed to link with and support the on-going work of learning and accountability networks such as HAP-I, ALNAP, Sphere and People in Aid. ECB experience to date with interagency evaluations suggests the following possible benefits: •

37 34

Strengthened coordination systems. The lack of an effective NGO coordination system in, for example, Niger prior to the interagency evaluation had resulted in scattered individual agency Finalized with the steering committee and the external consultant team leader.

Yogyakarta Joint Evaluation Annexes July 19, 2007

efforts. In the final report it was noted that, â&#x20AC;&#x153;Opportunities were thus lost for recognition of comparative advantages between the partners, establishing joint advocacy positions and for peer training prior to emergency actionsâ&#x20AC;?38.

â&#x20AC;˘ Modelling cooperation at upper levels can strengthen cooperation in the field so that it relies less on ad hoc, informal contacts. 2. Timing of the Joint Evaluation

To coincide with the one year anniversary of the Yogyakarta earthquake. 3. Objectives of the Joint Evaluation

This consultancy will provide the following:

i. An assessment of the quality of the response, measured in terms of the impact, timeliness, coverage, and appropriateness of the respective emergency responses of the participating agencies; ii. An appraisal of the extent to which agency interventions mitigated the effects of the earthquake via their contribution to an accelerated recovery for affected communities; iii. A judgement about the degree to which agencies improved preparedness for potential future emergencies;

iv. An evaluation of the effectiveness and coherence of the coordination between key stakeholders, including government, donors, UN agencies, NGOs and people affected by the disaster, identifying examples of both good practice and missed opportunities;

v. Identification of examples of good practice and critical gaps coupled with recommendations for improving emergency preparedness and response in future at both a country and global (institutional level); and vi. Recommendations for the future conduct of interagency evaluations. 4. Specific Issues for Consideration

IMPACT: There remains a lack of industry standards and definition regarding impact as applied to humanitarian actions, and responses are usually undertaken without appropriate baseline line information or monitoring systems in place. Since this evaluation will examine evidence of changes (positive and negative) attributable to the aid intervention, it will also make suggestions regarding indicators for measuring impact and provide examples of promising practice in the monitoring of impact.

QUALITY ASSURANCE: The evaluation will examine the extent to which beneficiaries were supported and encouraged to participate in all elements of the project cycle. Particular attention will be paid to the effectiveness of participatory accountability systems, put in place by each agency, designed to ensure that beneficiaries are aware of their entitlements and have full access to a feedback system to register complaints. 38

Niger joint evaluation, p. 3

Yogyakarta Joint Evaluation Annexes July 19, 2007

APPROPRIATENESS: The evaluation will examine whether the intervention and the resources provided were relevant to the need, context and culture, with particular emphasis on the restoration of livelihoods.

5. Methodology i.

Data gathering – Data will be collected via a combination of the following approaches.

• review of relevant literature; • field observations, and • key informant interviews and/or focus group discussions with: a) community members affected by the earthquake (focus group discussion will be grouped by gender (male and female) and age (adults and children); b) selected agency staff (in the field, in country and regional offices and at headquarters) and, where possible, key staff who have left the programme; and c) key external stakeholders (host government officials, UN, NGOs, donor representatives). ii. Ethical considerations – The Evaluation Team will take all reasonable steps to ensure that the security and dignity of affected populations is not compromised and that disruption to ongoing operations is minimized; iii. Confidentiality of information – All documents and data collected from interviews will be treated as confidential and used solely to facilitate analysis. Interviewees will not be quoted in the report without their express permission.

iv. Communication of Results – The Evaluation Team will present the preliminary findings and recommendations to key stakeholders in order to provide immediate feedback to operations managers, and to afford the Evaluation Team an opportunity to validate findings. Following publication of the evaluation results, the Steering Committee will present key findings and recommendations to different stakeholder groups, including government authorities, donor representatives, NGOs as well as representatives from the communities themselves. v. Use of Results – The results of this joint evaluation are intended not only to increase the quality of programming and coherence between agencies responding to the Yogyakarta disaster at an institutional level, but also to guide similar joint activities in future. The findings of the evaluation will be placed in the public domain to promote improved evaluation quality throughout the wider humanitarian community. Stakeholders targeted by specific recommendations will be expected to outline plans of action wherever appropriate and agency focal points listed below will be responsible for monitoring follow-up at a country and institutional level as appropriate. 6. Management of the Joint Evaluation

The evaluation will be jointly managed by a Steering Committee comprised of in-country agencies designated by each participating agency, with the option of inviting key stakeholder(s) to participate as observers. The Steering Committee will be the primary point of contact for the evaluation team. It is anticipated that at strategic intervals there will be progress update meetings between the evaluation team leader and the Steering Committee. 36

Yogyakarta Joint Evaluation Annexes July 19, 2007

i. Coordination and administrative support - In view of capacity, it has been agreed that CRS will assume responsibilities for facilitating the Steering Committee for the evaluation, providing administrative support (e.g. issuing consultancy contracts for consultants) and coordinating logistical support to be provided by each agency for the evaluation team. ii. Technical support – In-country M&E capacities will be backstopped by support from the Accountability & Impact Initiative Advisers to the IWG’s Emergency Capacity Building project.

iii. Cost sharing arrangements – Shared costs for the evaluation will be equally divided between participating agencies. This study will focus on areas where participating agencies are intervening. 7. Team Composition

Overall responsibility for leading the evaluation (including drafting and editing each version of the report) resides with an external team leader consultant. Other team members will include Standing Team members and representatives from each organisation who will carry out the evaluation. A request was made to the ECB2 management team for two Standing Team members to participate in the joint evaluation39. Team members will be selected on the basis of their abilities to fulfil the tasks outlined in this TOR, with and suitable balance of appropriate technical skills, gender and geographical distribution. The joint evaluation team will consist of:

• An international consultant Team Leader

• A staff member from one of the participating agencies who was not involved in the Yogyakarta Emergency Response. 8. Schedule & Milestones (March through June 2007)

This consultancy is foreseen as requiring a total of 20 days40 with final deliverables due before June 30th, 2007. The proposed calendar of activities is as follows: • Drafting TOR, Service Request and Budget

• Selection of consultants and Standing Team members

• Document Research, Fieldwork

• Draft report circulated to agency interviewees

• Report finalised and communicated to stakeholders

March 2007

April 2007

June 2007

July 2007

39 The request for a standing team members was met by one agency that sent a regional staff another for the first 10days and a second regional staff member for the remaining period of the evaluation. The original ToR requested a staff member from each of the participating agencies. Staff were not available. Local facilitators, note takers and translators with Bahasa language skills were hired instead. 40

Excluding travel days to/from Yogyakarta.

Yogyakarta Joint Evaluation Annexes July 19, 2007

The duration of the evaluation will be 20 days, excluding travel to/from Yogyakarta (2 days each way). . Deliverables

The evaluation team will produce a draft and final report in MS-Word within the time lines specified by the management committee. The report will consist of an Executive Summary of no more than 5 pages that covers the main findings of the evaluation. The main text should consist of no more than 30 pages, covering methodology, findings and recommendations, with annexes. This report will be circulated to participating agencies for comment prior to finalization and publication. The report will be produced in English, and at the minimum the executive summary will be translated into Bahasa Indonesia to enable National and Local Government, partner agencies and the local community to read and understand the findings. An appropriate budgetary provision will be made for translation. The final report will be presented to key stakeholders as agreed by the Steering Committee. 10. Agency Focal Points Agency

Steering Committee

CARE International in Indonesia

Harining Mardjuki Harining_Mardjuki@careind.or.id

CRS

Save the Children

World Vision Indonesia

ECB II Advisers

Adhong Syahri Ramadhan sramadhan@id.seapro.crs.org

Dane Fredenburg dfredenburg@crsert.org

Agus Budiarto pm_scukyogya@yahoo.com

Emma Roberts e.roberts@savethechildren.org.uk

Richardus Indra Yacobus Runtuwene

Guy Sharrock gsharroc@crs.org

Richardus Indra Richardus_Indra@wvi.org

Yacobus Runtuwene yacobus_runtuwene@wvi.org

Yogyakarta Joint Evaluation Annexes July 19, 2007

Annex Two:

Schedule of the joint evaluation

28th May 29 May 30 May

31st May

Joint Evaluation team meets in Jakarta with Yenni Suryani, CRS to review schedule

Meetings with senior staff in: CRS, and World Visions to understand their response to the Yogyakarta Emergency and their level of emergency preparedness. Meetings with UNDP and Save the Children UK.

• Evaluation team meets in Yogyakarta to agree how they will work together • Evaluation team meets with the Steering Committee to understand the big questions they want addressed and review the methods and locations where the fieldwork is to be done. • Collection of important documents including those from government and donors.

• Meetings with various government officials in Yogya • Individual meetings with staff of each JE participating agency to understand their response, preparedness and the type of accountability systems they have for working with local people. • Individual meetings with M&E staff to discuss their M&E system.

1st June

• Fieldwork in Sawit, Gantiwarno, Klaten to test questions and methods.

3rd June

Day Off

2nd June

th June

5th June 6th June 7th June 8th June 9th June

• Evaluation team meeting to review process so far and amend methods and questions, agree division of work for remainder of Joint evaluation, and continue document review. • Morning review of findings to date • Field work in Piring, Srihardono, Pundong, Bantul by one team • Fieldwork in Gaduh, Patalan, Bantul by other team. • Morning review of process and findings to date. • Fieldwork in Canden, Jetis, Bantul by one team. • Fieldwork in Sengon, Cucukan, Klaten by other team.

• Morning review of process and findings to date. • Fieldwork in Katekan, Gantiwarno, Klaten by one team. • Fieldwork in Sawit, Gantiwarno, Klaten.

• Morning review of process and findings. • Fieldwork in Brangkal, Wedi, Klaten by one team. • Fieldwork in Sukorejo, Wedi Klaten by other team.

• Fieldwork in Sukorejo by one team, and Sumberharjo by the other team. • Discussion with district health and education officials. • Discussion with sub-district health officials • Data analysis by two team members

Yogyakarta Joint Evaluation Annexes July 19, 2007

10th June

Data analysis by two team members and document review

12th June

Writing and preparation for interagency event on 13th June

11th June 13th June

14th June 15th June 16th June 17th June

18th June 19th June

20th June 21 June 22 June 30 June

6th July

Workshop with field facilitators, note takers and translators to review village summaries and draw conclusions. Interagency staff workshop in Yogyakarta to review findings and draw conclusions and recommendations. Report writing by team to incorporate the views from interagency staff meeting.

Preparation and facilitation of interagency workshop in Jakarta with senior staff who make decisions on emergency interventions (2 hours) to share findings, conclusions and recommendations and hear their views on what else needs to be included in the report. Evaluation Team travels back to Yogyakarta and continues writing report.

Day Off

Continue writing draft report.

Meeting with steering committee to prepare for multi-stakeholder event on June 20th Continuing writing and preparing for the event on the 20th June

Sharing of Joint Evaluation findings with wider stakeholder group and launch of GEG in Yogyakarta Complete draft report and travel Yogyakarta to Jakarta Team Leader Departs Indonesia from Jakarta

Final comments from agencies on the JE Report.

Team Leader incorporates final comments on draft report and sends final report agencies. Guy Sharrock places Joint Evaluation in the public domain e.g. ALNAP and ECB websites

Yogyakarta Joint Evaluation Annexes July 19, 2007

Annex Three: References

CARE International Indonesia (November 2006) – JERP EOP Workshop PowerPoint presentation

CARE International Indonesia (December 2006) Market-Based Relief (MBR) Program – PowerPoint presentation. CARE International Indonesia (January 2007). - Java Earthquake Response Program (JERP) End of Year Report June – December 2006

CARE International Indonesia CARE (April 2007) Central Java Recovery Program GRIYO Project, Progress Report. CARE International Indonesia (April 2007) Sendang Desa Program – PowerPoint presentation.

Choularton, Richard. (March 2007). Contingency planning and humanitarian action: A review of practice. London: ODI Humanitarian Practice Network.

CRS (July 2006). CRS/INDONESIA YOGYAKARTA EMERGENCY RESPONSE & RECOVERY STRATEGY 1 June 2006 – 31 March 2008. Dwiyanto, Dr. Agus, et al. (2007). World Vision Indonesia Post-Earthquake Assistance Program (Yogyakarta Earthquake Emergency Response Phase II) Evaluation Report. Yogyakarta: Gadjah Mada University.

Harjanto, Wahyu. (October 2006). Monitoring and assessment of Yogyakarta Earthquake Response Programme in selected areas of ERST members in Yogyakarta and Central Java. CRS Yogyakarta: Caritas Implementing Partners. IASC (July 5, 2006). Indonesia Earthquake: 2006 Response Plan Revision. Jakarta: UN.

OECD/DAC (1999), Guidance for Evaluating Humanitarian Assistance in Complex Emergencies.

Paris: OECD. Ramadan, Adhong Sy (July 19, 2006). Distribution Monitoring and Gap Assessment: Yogyakarta Earthquake Emergency Response. Yogyakarta: Catholic Relief Services. Save the Children-UK (July 2006). Yogyakarta Six-Month Operational Plan. Jakarta

Save the Children (November 2006). Yogyakarta Earthquake Emergency Response – Final Program Report. UNDP (May 25, 2007). The Cluster Approach in Yogyakarta and Central Java: One-Year Review (Draft)

World Vision Indonesia (July 27, 2006). Initial Emergency Report on the Yogyakarta Earthquake Response. World Vision Indonesia (January 10, 2007). Final Report for First Phase (May 26 – November 25, 2006) Yogyakarta Earthquake Emergency Response (JEER). Yogyakarta Joint Evaluation Annexes July 19, 2007

World Vision Indonesia (June 2007). Report of the Second Phase of the Yogyakarta Earthquake Emergency Response (draft). World Vision International for the Emergency Capacity Building Project (2007). Impact Measurement and Accountability in Emergencies: The Good Enough Guide. Oxford: Oxfam Publications.

Yogyakarta Joint Evaluation Annexes July 19, 2007

Annex Four:

Summary of those we spoke with in each village

FGDs

People

Vil-1

Vil-2

Vil-3

Vil-4

Vil-5

Vil-6

Vil-7

Vil-8

Vil-9

Men

Women

SSI

205

10 3

Children

10 7

Men (L)

Women

Men

Couples

Total

Boys

The joint evaluation team went to nine sub-districts in seven different sub-districts. Those villages are: Pundong, Patalan, Jetis, Cucukan, Gantiwarno, Prambanan, Wedi . In all we spoke with 318 people.41

26 FGDs were conducted using open-ended questions. Total participants in FGDs were 256. They were mostly beneficiaries of the programs carried out by one or more of these agencies. We intentionally tried to keep FGDs to around 8 to 10 people so that we could understand the perspective of each FGD participant. Similar questions were used for separate semi-structured interviews with 49 individuals who were mostly non-beneficiaries of programs implemented by these agencies. In addition, we asked similar questions of 13 village leaders, plus we asked them what it was like after the earthquake to understand the baseline conditions and how far the village has recovered from the affects of the earthquake. People were friendly and happy to speak to us about the work done by these agencies.

This was two separate FGDs with women.

Yogyakarta Joint Evaluation Annexes July 19, 2007

Annex Five:

Key informants

Name

Organisation

Function

Achmad Judi Wirjawan

Catholic Relief Services

Logistic Coordinator

Adjie Fachrurrazi

CARE

Technical Program Leader DRR & Environmental Health

Adhong Sy Ramadhan

Catholic Relief Services

Program Coordinator

Agus Budiarto

Save the Children UK

Program Manager

Ardhiani Dyah P.

Save the Children UK

Senior Program Officer Nutrition

Anwar Hadipriyanto Asif Sarwar

Bambang Yulis Priambodo Dian Asmarani Dr. Cahyono

EJ Heri wahyudi

CARE

Save the Children UK World Vision JEER World Vision JEER

Community Health Centre, Gantiwarno, Klaten UNCC - Yogyakarta

Grants and M&E Officer

Deputy Director (Support) Distribution Staff

Child Protection Officer

Head of Community Health Centre

Liaison and Coordination Officer for Central Java Province

Dr Suyatno

Education Department of Jetis Sub-district

Head of department

Gendut Sudarto (Dr)

District of Bantul

Executive Secretary

Evi Esaly Kaban

Harining Mardjuki

Hastamik Purbatin Wahyuningsih Hendri Puryanto

Jimmy Nadapdap Jhon Purba

Save the Children UK CARE

CARE

M&E Coordinator

Team Leader of Central Java Recovery Program

Sendang Desa Project Manager

CARE

Construction Officer

World Vision

HEA Manager

CARE

Watsan Technical Specialist/ Behavioural change officer

Kristanto Sinandang

UNDP

Senior Program Officer

Lusi Margiyani

Save the Children UK

Program Coordinator Education

Maria Josephine Wijiastuti

Catholic Relief Services

Crisis Prevention & Recovery Unit Monitoring and Evaluation Officer

Yogyakarta Joint Evaluation Annexes July 19, 2007

Muhamed Taufikurohman

World Vision JEER

Shelter Officer

Nur Zainab

Department of Health Bantul

Director of Health Department

Muhamed Zuhri Paidi Suparno

Petrus Hendra Qurotul Aini

Retno Winahyu

Catholic Relief Services Sawit Village

World Vision JEER CARE

UNDP

Shelter Officer

CBO leader (“Fathers’ Group”) Area Coordinator for Bantul Health Officer

Team Leader Early Recovery Assistance Program

Richard Balmadier

Catholic Relief Services

Country Representative

Ruth Meigi Panggabean

World Vision

Program Officer

Richardus Indra Gunawan Sasmoyo Hermawan Sekti Mulatsih

Shewangezaw Lulie Sri Yatini Subardi

Sugeng Santosa

Tulus Budiyanto Wilfridus Nahak

Yacobus Runtuwene Yenni Suryani

World Vision JEER

Save the Children UK UNCC

Save the Children UK World Vision JEER

Department of Education

BAPEDA District of Klaten Village Leader CARE

World Vision JEER

Catholic Relief Services

Yogyakarta Joint Evaluation Annexes July 19, 2007

Program Officer

Senior Education Program Officer

Information and Liaison Assistant Emergency Program Manager Infrastructure Facilitator

Principle of an elementary school BAPEDA, Klaten staff Sawit, Gantiwarno

Griyo Project Manager Team Leader JEER

Deputy Country Representative

Annex Six:

Conditions facilitating success of the response

As the one-year anniversary of the Yogyakarta earthquake passed, many marvelled at the quick recovery in the area. These are some of the reasons we heard for recovery.

• People in central Java trust one another. It was easy for NGOs to work with them to make decisions and ensure goods and services were provided in ways considered fair. • Local officials all the way up to provincial level encouraged local people to act themselves and not depend on aid. People did act for themselves and provided each other mutual support as the local tradition of ‘gotong royong’ was revived. • GOI quickly mobilized to provide resources – rice and a small cash grants were provided to those who lost their homes.

• Within six months the GOI was providing cash subsidies to families who had lost their homes for housing reconstruction. • Yogyakarta City suffered little damage so transport and communications networks were working. As it is a manufacturing and trading centres, it was easy to find supplies of NFIs, building materials, etc. Prices of goods went up only after a few weeks and even then traders did not try to exploit the situation by raising prices unreasonably.

• There were many emergency responders including government, local NGOs, the private sector (34% of goods in some places) and individuals. Truckloads of volunteers came from Solo and other surrounding cities and universities to help.

• INGOs, UN and GOI had teams on the ground and emergency supplies in Yogyakarta to assess the affects of a potential eruption of the Mount Merapi Volcano. Both were immediately redeployed to respond to the affects of the earthquake. • There were many local NGOs already working at grassroots level in Yogyakarta who were able to facilitate NFI distribution in the communities. • Many agencies had competent staff with emergency skills that they redeployed from Aceh and other staff were hired from the many universities in Yogyakarta.

• Coordination between the GOI, UN and NGOs was relatively effective with most providing information on what they were doing and where they were working in order to facilitate coordination and to identify and fill gaps and not compete but cooperate. • FGDs indicate that NFI kits were timely and efficiently distributed helping to quickly meet affected people’s basic survival needs. Little oversupply or undersupply of goods was reported.

• Affected areas were accessible.

Yogyakarta Joint Evaluation Annexes July 19, 2007

Annex Seven:

Brief chronology of the response to the Yogyakarta earthquake

Date

External Events

Response by JE Agencies

Mid-April 2006

GOI raised the alert level for Mount Merapi Volcano from level II to III. GOI estimates that 30,000 people need to be evacuated and 71,000 will be effected by fall out if volcano erupts.

CRS office in Yogyakarta sends in team to provide support to resettle those affected by volcano activity.

Earthquake measuring 5.9 on the Richter Scale hits at 05:53AM in Yogyakarta Province and Central Java Province. An estimated 5,760 people are killed, 37,339 injured and 1million people estimated to be homeless. Public infrastructure damaged includes telecommunications, schools, roads and bridges, airport, electricity supplies, government buildings, mosques and churches. Estimated value of infrastructure destroyed is US$3.1billion.

CRS starts to procure and distribute shelter, hygiene and family kits to 5,000 people in Pretek, Pudong and Prambanan, plus provides funds to local NGO to run its mobile clinic. SC joins coordination meeting in Bantul. WVI starts providing medical supplies to hospitals, NFIs to families. Program continues to expand as the days go by.

GOI declares a 3-month state of emergency and allocates 1.7 trillion rupiah (US$190million) for recovery and rehabilitation activities. UN agencies, government and NGOs complete rapid needs assessment of the affected area. Priority needs identified include emergency shelter, medical supplies, clean water, sanitation, agriculture and food. People in Aceh collect donations, clothes and blood to send to Yogyakarta.

CARE deploys staff to Jogya

May 14 May 27

May 28

May 29

UN and BAKORNAS staff deployed to Yogyakarta to monitor and support preparedness efforts related to possible eruption of Mt. Merapi.

President Susilo Bambang Yudhoyono temporarily relocates his office to Yogyakarta in order to coordinate the emergency response effort. Response teams, medical teams and military units from around the country begin moving into the affected provinces to work with BAKORNAS and provincial and district authorities. UN led coordination cluster meetings start up.

Yogyakarta Joint Evaluation Annexes July 19, 2007

Date

External Events

Response by JE Agencies

May 30

The airport in Yogyakarta reopens and commercial flights resume A UNDAC team is deployed to support operations in Bantul and Yogya. UNICEF and government partners do a child protection assessment.

CARE signs cooperation agreement with LNGO Yayasan Dian Desa

GOI pledges rice and funds for clothing and HH goods to affected people. Reports of theft from damaged and abandoned homes are reported. Concerns over spread of disease continue.

SC begins setting up safe play areas for affected children and completes a rapid child protection assessment survey. WV sets up a temporary office in Yogyakarta and deploys a rapid relief assistance team.

May 31

June 1

June 2

June 3 June 5

June 6

Primary school examinations scheduled for the first week of June postponed indefinitely in the most affected districts. Increase in cases of malaria, dengue, ARI plus diarrhoea and tetanus are reported. 22 countries are now involved in the relief operations.

CRS with other CARITAS members in Indonesia meet and agree an emergency response strategy. SC starts NFI distribution (hygiene kits)

MOH instructs hospitals to provide free medical treatment to affected people. Emergency Response Plan (IASC) issued requesting US$103million for immediate relief needs for UN run by programs.

WVI starts providing cleaning kits to families to clear rubble and set up temporary shelter.

GOI establishes crisis centre at Yogyakarta Airport. All aid agencies are instructed to register and report their activities to the centre.

CARE 1st distribution of SWSs

GOI reduces emergency response period to one month after which the recovery period is to start. There are 5,000 military in the area to monitor the situation and prevent looting. GOI Social Affairs Ministry increases number of reported dead from the earthquake to 6,234, those injured to 50,000 and the number of internally displaced to 647,000. UNOCHA sets up office in Yogya

Yogyakarta Joint Evaluation Annexes July 19, 2007

Date

External Events

Response by JE Agencies

June 8

Earthquake of 4.3 on the Richter Scale occurs in Klaten. Provincial health authorities send out mobile clinics to address rise in tetanus and run measles immunization campaign. Market prices of construction materials increase.

WVI opens its first child friendly spaces in Prambanan, Bantul for 420 children and these centres are gradually set up in 9 locations in Bantul and 11 locations of Klaten and reach 3,300 children.

Mount Merapi is at alert level.

WVI, CARE, Plan, Oxfam, Islamic Relief, CARDI do a consolidated appeal for more funds.

June 12 June 14 June 16 June 23

June 30 July 2, 2006 July 3 July 5

BAKORNAS creates earthquake compensation plan.

Affected people in some remote villages are yet to receive assistance.

CRS starts 10-day assessment in Klaten and Bantul to develop its immediate and long term plans

34% of people who lost their homes are rebuilding them from previous materials. Tetanus case numbers are stabilizing though diarrhoea and ARI remain problems. Finding clean water becomes difficult.

SC starts ToT on disaster management for local partners.

The President declares the emergency phase over. BAKORNAS leaves Yogyakarta and coordination meetings in Jakarta stop.

Rehabilitation phase is to go to December

2006 and reconstruction to December 2008.

SC local partners start training teachers on

disaster management.

Revised Emergency Response Plan (IASC) issued requesting US$80million noting that 2.7million people are affected (631,000 HHs) as their homes were damaged. Of these 1.6 million (345,000 HHs) are homeless. The number of primary schools estimated needing immediate emergency assistance so they can start on July 17 are 1,232. The plan is to complement efforts of GOI.

CARE distributes food vouchers to 10,500 people and plans to distribute vouchers to 10,700 more people in August and September WV trains youth volunteers to work with children in child friendly spaces in Klaten

60% of affected people have received GOI assistance of 10kg of rice and R.90, 000.

Presidential Decree 9 defines GOI executive and implementing teams for post earthquake rehabilitation and redistribution for a two-year period.

Yogyakarta Joint Evaluation Annexes July 19, 2007

CARE conducts an Action and Capacity Review.

Date

External Events

Response by JE Agencies

July 6

GOI estimates that 39% of affected families (600,000 people) have received shelter assistance (tents or tarpaulins) from national and international humanitarian agencies and GOI. Given scale of need GOI extends emergency phase to end of August

CARE – permanent team leader in place

July 14

July 17 July 27 July 31

GOI announces plan to assist 300,000 HHs by providing them a cash subsidy to build earthquake resistant house. NGOs are asked to complement this by providing tools. Humanitarian agencies are operating in 58 out of 65 affected sub-districts. Reports of suicide and depression continue. New school year begins. School fees reduce attendance so GOI waives some fees. OCHA reports 69% of affected population have received emergency shelter.

CRS emergency program ends to be continued by a post emergency phase until Sept 30 and a rehabilitation phase to March 31, 2008.

WV trains youth volunteers to work with children in child friendly spaces in Bantul WV starts supplementary feeding program to 3,000 children in 5sub-districts of Bantul and 5 sub-districts of Klaten to continue to November.

August – 1st week

UN agencies and others do livelihood assessment. Harvesting of crops begins.

MidAugust

An estimated 80% of people affected have CARE distributes 2nd round of food

vouchers.

emergency shelter. However, 40% of these people are living in conditions below Sphere

minimum standards.

Provincial Governments in process of finalizing their housing recovery assistance packages for HHs whose homes were destroyed.

EndAugust Sept

Response plan for Mount Merapi is set aside due to Merapi’s lower activity since May. Provincial government nears finalisation of housing policy to compensate those whose houses were damaged by the Earthquake.

SC starts to build temporary school shelters and continues this work to December.

Yogyakarta Joint Evaluation Annexes July 19, 2007

Date

External Events

Response by JE Agencies

October

Monsoon rain starts Planting season for farmers in Central Java starts.

CARE distributes 3rd round of food

vouchers.

Strong rainfall triggers cold lava and cloud flows at Mount Merapi that reach villages within 3kilometre. UN coordination centre in Yogyakarta closes at end Nov.

CARE completes end of project evaluation

(external).

October 19 November

December

40% of people who lost their houses in the Earthquake remain in inadequate shelter to protect them from the rains .

January 07

February May

Windstorm strikes the city of Yogyakarta

Future earthquake response coordination handed over to UNDP Early Recovery Agency or the GOI.

Yogyakarta Joint Evaluation Annexes July 19, 2007

CRS starts its transitional shelter program

(core house structures).

SC ends emergency response program. Plans to continue program to June â&#x20AC;&#x2122;08 and focus on longer term development issues for children. SC SPAs handed over to communities. SC starts internal evaluation for emergency phase. CARE - Begins recovery program (incl. MBR â&#x20AC;&#x201C; shelter) to be completed by end of August 2007. SC starts Child Led Review for emergency phase.

CRS distributes 700 claw hammers and 350 shovels to clean up rubble.

WVI Emergency Program Ends. Yogyakarta Office to close down in June. Staff member to remain in DoH office to monitor phase over.

Annex Eight:

Summary of findings of each agency

CARE – RELIEF AND RECOVERY PROGRAM MAY 27 2006 TO MAY 26 2007 Program Objectives

Activities Completed

Outcome/results

Provide immediate relief to earthquakeaffected communities and assist vulnerable households to rebuild their houses.

Distribute to approx. 50,000 HHs • 20,143 tarps • 93,246 blankets • 138,141 SWS • 55,605 jerry cans • 299,310 bars soap

Food distributions to 10,700 recipients (3 times) through 30 local vendors (Voucher scheme). One time distribution of hygiene products to 5,914 households and health promotion to 18 villages. Through MBR vulnerable families have additional support (8.8m RP) to rebuild their destroyed houses. Empower communities to mobilize and strengthen community awareness and practice of safe, clean drinking water use.

Community volunteers trained to lead trainings and disseminate best practices water management information to 1,050 households.

Contributed to meeting the basic needs of earthquake-affected communities.

Basic food needs of targeted communities met.

Increased awareness and knowledge of targeted communities in ways to prevent illness from poor sanitation and hygiene practices. 307 vulnerable families have the financial capacity to rebuild their destroyed houses.

Communities take responsibility to ensure HH uses safe drinking water.

Narrative:

Prior to the earthquake CARE did not have a presence in Yogyakarta, however they were quick to mobilize and commence rapid assessments within two days of the earthquake to determine the extent of damage and identify emergency and relief needs. By day five CARE had developed an emergency relief strategy. CARE focused on technical areas in which it had prior experience, especially with regards to recent emergency relief work conducted in Aceh. Geographically, CARE provided emergency relief in four sub-districts of Klaten and Yogyakarta distributing nonfood items (NFI), providing food distribution through a Market-Based Relief (MBR) scheme,

Yogyakarta Joint Evaluation Annexes July 19, 2007

and supporting communities with health, water and sanitation interventions. Their emergency program in Yogyakarta is from May 2006 to August 2007. CARE partnered with three LNGOs, namely, Dian Desa, Prakarsa and KOMPIP to deliver relief services. Activities were also coordinated with local government officials, community leaders and community groups. Appropriateness:

The direct distribution of NFIâ&#x20AC;&#x2122;s was considered fair and met recipientsâ&#x20AC;&#x2122; immediate needs. Overall the MBR method for distributing food received positive comments from the community and village leaders. They felt that the system was timely, appropriate and the materials received through it were of good quality. The system also helped local shops to recover. One village leader cited how the system reduced pressure on him by not having to organise distributions. The provision of food whether via voucher or directly was appreciated by communities as it allowed them to focus their limited resources on other important activities such as shelter reconstruction. However, some interviewees said that by not targeting 100% of the community the system was unfair and created stress within the community. Participants in one focus group said that they would have preferred direct distribution of food to beneficiaries though the village leader preferred vouchers as it left the responsibility of collecting goods with households. It is evident that people see advantages and disadvantages with this approach and further discussion would help clarify whether to use such an MBR approach in future.

CARE support to permanent housing was very much appreciated. The use of vouchers allowed beneficiaries to control the supply and quality of materials. Informants said it allowed them to hire and pay local labourers who were trained by CARE on earthquake resistant construction and control the quality of the material construction materials. They said local people now have the knowledge of the methods involved in constructing earthquake resistant houses. Impact:

Through the support given by CARE, other NGOs and the GOI, the immediate and basic household needs of affected communities were met. In general the MBR program had a positive impact on local vendors and was perceived by most as a fair and efficient way to distribute relief items.

People recalled the health promotion and promotions around the better management of drinking water, and credited these programs with improving their health.

The shelter support program gave additional support to vulnerable families to rebuild their houses. The program also increased the communities understanding of how to construct earthquake resistant houses and built the capacity of over 150 local construction workers in appropriate building techniques. By having greater involvement of the community in activities, CARE was given credit for increasing cooperation and solidarity with in the community.

Yogyakarta Joint Evaluation Annexes July 19, 2007

Level of Recovery: When asked to compare where they are now against their situation immediately after the earthquake, communities and leaders rated their recovery in the region of 30 to 50% regarding shelter and livelihoods. Accountability to local people:

The cadre system employed by CARE was regarded by the community as the fairest system to ensure the participation of both men and women in the program and as a means of informing the communities about the program. A number of the interviewees commented on how CARE encouraged people to work together and get involved in their own relief efforts. CARE coordinated its activities at the district level through district coordination meetings and was praised by one district official for being the only agency to have a MOU with the district authority.

Not directed specifically at CARE, but in one community where CARE worked the community felt that the NGOs needed to better coordinate distribution activities in order to ensure fair and equitable distribution between various districts. The fact the shelter beneficiaries had control over the selection of shelter material resulted in less complaints, however one Focus Group noted that people were afraid to complain in case the shelter program was stopped. They did not specify what complaints they had. CARE did have a complaints system during the initial months of the program. However, staff interviewed did not know about the system and it appears to have been stopped. Communities were also pleased with the level of on-site supervision provided by CARE staff and the friendliness of the staff. Monitoring and Evaluation:

CARE conducted a two-day end of program evaluation workshop that included 70 stakeholders; including beneficiaries, community leaders, local government officials, vendors, community volunteers, and other NGOs including partners to assess what worked well and what did not. They used the information to adjust activities accordingly. Concerns:

In the interest of fairness and in order not to create internal community conflict, vouchers were redistribution by at least one community. The redistribution targeted all the community members equally. However, some of the more affluent members felt humiliated knowing they were receiving vouchers originally intended for the poor.

While there were a number of positive comments on how CAREâ&#x20AC;&#x2122;s health promotion activities help improved hygiene behaviour, there was concern about the effectiveness of chlorine water purification distribution (SWS). It appears that in many cases people did not feel it was necessary to use the SWSs. They did not like the smell or the taste it left in the water, and preferred to use the chlorine for washing clothes. The health and hygiene program was appreciated, although both men and women said the proposal process for getting funds was complicated and needed to be simplified. 54

Yogyakarta Joint Evaluation Annexes July 19, 2007

It was noted by some that the assistance for shelter came late as households had already started rebuilding their houses.

CARE appeared to use two different methods for selecting beneficiaries for the shelter support program. For the first phase the communities were involved in the selection process, however for the second phase beneficiaries were selected by the CARE staff along with staff of the local NGO KOMPIP. CARE staff said that the list of selected beneficiaries was confirmed during a public village forum. The community was however unsure as to why this had happened and said that they preferred the process of beneficiary selection used in the first phase. While appreciating the support of NGOs in meeting basic needs most interviewees would like to have seen more support for recovering livelihoods. Preparedness

No emergency preparedness or contingency plan was in place for Indonesia. For further details on emergency preparedness please see annex 7. Recommendations to CARE from stakeholders (What they could do better next time):

• Needs to be better coordination between NGOs to ensure the equal and fair distribution of relief supplies between villages/sub-districts. • All members of the communities, including the more affluent, should be included in distribution activities.

• Review beneficiary selection procedures with community representatives to confirm and ensure that they are acceptable to local people before they are applied.

• Make the community proposal submission/preparation system for wat/san projects simpler. • Upfront all or some of the funding for the vendors to purchase MBR items.

Yogyakarta Joint Evaluation Annexes July 19, 2007

CATHOLIC RELIEF SERVICES RELIEF AND RECOVERY PROGRAM MAY 27, 2006

TO MAY 26, 2007

Program Objectives

Outputs

Outcomes/Results

Target families are able to meet their basic needs.

• -

7,500 HHs live under conditions per Sphere standards within a month of the disaster

Target families have improved living conditions.

Children in target communities live in harmony. Target families resume their livelihoods.

Narrative:42

7,500HHs receive and use NFI kits Clothing kits (10,000 HHs) Shelter kits (7500 HHs) Kitchen Utensil Kits (6250 HHs) Household Kits (7500) Hygiene kits (5000 HHs) Group Tool Kits (250)

• 300 HHs have transitional shelter frame/roof constructed

300 targeted HHs complete house and live in it within their community

• 2 groups of 30 children formed in each village with transitional housing program • 75% of children attend bimonthly practice sessions for soccer, volleyball and peace building.

50% of children demonstrate an increased participation in children’s group activities.

• One person/HH trained on BES housing standard • 1committee per 10 HHs formed to complete transitional houses • 215 HHs in need of support provided HH latrines

300 HHs provided capital and equipment to restart their businesses.

80% of targeted HHs have a stable source of income6.

CRS had worked in the Yogyakarta area since 1997. At the time of the Earthquake, they had one staff member working on peace building and a team working on a response to the Mount Merapi volcano. With a team on the ground, they were able to identify needs and begin NFI distribution immediately. CRS worked in locations badly affected by the earthquake and where few agencies had reached. The sub-districts and districts where they worked were: Patuk and Kretek sub-districts of Bantul, Prambanan in Klaten and Prambanan in Gunung Kidul. Their relief program ran from May 27 to August 1 2006. It was followed by a recovery program which will run until March 2008. CRS worked closely with three local NGOs – LBKUB, Lintas and YSBD. They also worked with local churches, CARITAS members, UN agencies and local government and community groups. 42 The provision of equipment and funds to families for the livelihood program is pending funding approval as of June 14, 2007. 56

Yogyakarta Joint Evaluation Annexes July 19, 2007

Appropriateness: NFI assistance was based on a rapid assessment and the assistance provided met the priority needs of affected people. Distribution and the quality of items recieved was considered good. Distribution procedures were simple, fast and efficient. The process of distribution was close to where affected people lived and they had indicated their preferred distribution method. The distribution of assistance was considered fair (all respondents). The clean-up kits were useful though in some places the community did not have procedures to control the proper use of tools resulting in some tools not being returned or being damaged. In some locations the quantity of goods was said to be insufficient. However, the dominant response was appreciation and not to complain and accept what was provided (p11 Harjanto-Assessment). The housing program was appreciated by UNCC staff who considered it a good model for core housing as it fit well within GOI defined housing policy in which NGOs were asked to compliment the GOI’s housing subsidy. The shelter survey was considered valuable for ensuring that the right families were selected, focusing on. those most in need as well as promoting ‘gotong royong’. Local people worked together to help one another to complete their houses. Procedures for getting a housing frame were considered simple and quick. The transitional shelter model provided by CRS was rated as one of the best by local people. They noted it was earthquake resistant, went up fast, it was easy to construct, and the material and design was of good quality. ‘The roof acts as an alarm system as you can hear the rain and feel an earthquake’ (woman respondent). Water and sanitation support and latrines were provided to some households. In addition financial management training was provided to those who will be involved in the livelihood program and some DRR training was carried out with local people. However, there was little mention of these activities by the communities, with the exception of a group of men who said CRS had helped stop river pollution in their village, making it difficult to assess their appropriateness. Children said the activities for them were well organized and fun.

Respondents said that CRS monitored the work well and that were communicative and kept them informed. Staff were said to be polite and friendly (women/children respondents).

UN and government officials at district and local level said that CRS collaborated with them and provided updated information which helped ensure that assistance was well coordinated and that duplication was avoided. Coordination by CRS at local level was also appreciated and considered successful by leaders and men. CRS participated in the shelter, Watsan and livelihood clusters and UN informants said this participation was active and that information was shared openly by the CRS representatives. CRS was seen by UN as making efforts to reach remote areas. GOI officials and village leaders said they had coordinated with them well. Impact:

The assistance provided by CRS helped people and communities to recover. NFI support helped to ensure that the basic survival needs of families were met.

Those who received core housing spoke of feeling safer and less afraid of earthquakes and being more comfortable in their new houses than before. ‘People don’t run out of the house now when there is an earthquake’. Some families now have a better quality house than they did before the

Yogyakarta Joint Evaluation Annexes July 19, 2007

earthquake. People spoke of being healthier since moving into the new house. ‘Families have a house to live in now where before they were staying in tents and many people were sick.’

Men said that the program approach used by CRS promoted mutual assistance in their village and that mutual support among families has continued. They said that once their house was built they could start concentrating on earning a living.

In terms of the soccer, volleyball and peace building activities provided to children, boys spoke of soccer clinics as fun and giving them a greater sense of confidence. ‘Since joining this program we feel more ‘’funky’ and cool’. These games and peace building activities with girls and boys helped reduce children’s trauma. Boys spoke of getting higher scores in sports activities at school since participating in games. Village leaders and men said that the Watsan promotion activities by CRS have resulted in the river being kept clean.

Recovery: In villages where CRS worked children said they were feeling better and less traumatized. In one village men, women and leaders said that at least 50% of the households had recovered in terms of housing. In the other village most families now had houses but most had nothing in them. In one village some said that livelihoods were back to normal and others said they were not. In the second village people said unemployement was a huge problem. In both places people wanted support to restore livelihoods. Accountability to local people:

CRS worked through the local leadership and male group structures. CRS met with all at village level and encouraged people to choose those who would participate in the groups that CRS would work with. Men were chosen by each of the participating households. Group members and village leaders then decided which families would receive priority both for NFIs and for the core-housing program. The process was bottom-up and decisions were made at local level. However, women said they would have liked to be more involved in decision-making.

With NFIs, CRS’ local NGO partners verified what local people had received and what need still remained. As so many different agencies were distributing materials, ongoing needs assessments were necessary. CRS partners then participated in group meetings as well as in distributions. Distributions were carried out by local people with partners accompanying the process to ensure it remained transparent and fair. For housing, leaders and affected households prepared and submitted a proposal to CRS for core housing assistance. CRS visited and reviewed the proposal with the villagers. The system worked well as decisions on prioritizing beneficiaries was decided by local people themselves. CRS provided a roof and the housing frame and people were expected to use the housing subsidy from the GOI to complete the house. This level of support encouraged families to help themselves in completing their house. Information was provided to leaders and a complaints mechanism was set up so that anyone could send an SMS message to the M&E officer and or the shelter officer. Their cell phone numbers were provided to villagers. All complaints were discussed during program staff meetings and necessary actions were taken. Twenty complaints were received and addressed immediately. 58

Yogyakarta Joint Evaluation Annexes July 19, 2007

CRS claimed they used Sphere standards for temporary shelter and hygiene program. During the village visits families were living in the houses provided and people were satisfied with the good quality of the houses and the good quality of NFI materials received.

Monitoring and Evaluation:

Input, output and outcome data is clear and well maintained. The indicators for the emergency and recovery program are clear and realistic. Plans for collecting information at outcome level are easy to implement, for example observation and random discussions with recipients. The M&E officer said they used the Good Enough Guide to develop their M&E plans and thus the system was simple. CRS completed a monitoring and assessment of the Yogyakarta Earthquake Response Program in September 2006. The review used random sample methods to assess the NFI program carried out by the CARITAS implementing agencies (CIMO includes CRS, Cordaid, Caritas Switzerland and Caritas Germany) in 200 villages. This assessment was written up and the report was shared in cluster meetings. It is expected that the quality of the data assisted others to understand the usefulness of NFI assistance provided by various responders and to make decisions on gaps in needs at village level. In addition, CRS planned for a joint evaluation with other NGOs. They followed through with this plan and have done an excellent job in coo rdinating the joint evaluation process between the four agencies. Concerns:

• Women did not feel involved enough in program decisions

• Villagers were asked for information regarding the livelihood program. They have participated in surveys, but have yet to be provided information about when activities will be carried out.

• Staff felt the emergency response was limited in scale and scope because of delays in getting approval for relief activities.

Preparedness:

CRS did not have a preparedness plan for Indonesia. They did have many experienced emergency staff members working in Aceh who were rapidly deploy to Yogyakarta. For further information on emergency preparedness see annex 7. Recommendations to CRS from stakeholders (or what they could do better next time):

• GOI officials – move quicker on implementing a recovery program and spend less time on assessment for this. • Be faster in implementing livelihood support activities

• Inform relevant villages about the status of the livelihood program as they have provided information to CRS but have not heard back what will happen next. • Explore and find better ways to involve women directly • Complete house construction

• Work with communities to find ways to sustain soccer/volley ball program

Yogyakarta Joint Evaluation Annexes July 19, 2007

â&#x20AC;˘

Develop an emergency preparedness plan for Indonesia that includes provision of an emergency response team.

Yogyakarta Joint Evaluation Annexes July 19, 2007

SAVE THE CHILDREN – RELIEF AND RECOVERY PROGRAM MAY 27 2006 TO MAY 26 2007 Program Objectives

Activities Completed

Outcome/results

To ensure that 50,000 people (including 30,000 children) from the earthquake affected areas have access to adequate shelter and are protected from harm; and that their recovery is facilitated by opportunities for recovery, play and development.

• 9,954 HHs received and used NFI kits (nearly 43,000 people) - Shelter kits (9954 HHs) - Hygiene kits (2974 HHs) - Household kits (6523 HHs) - Clean up kits (121 HHs) • 2,900 children in 50 safe play areas were provided an environment to recover from the earthquake • 14,000 children in 99 schools were provided with emergency education facilities - transitional school shelters plus tents, desks, school materials • 760 local teachers trained in psychosocial support, emergency preparedness and planning.

- Families have access to a secure and dignified environment that protect them from the elements

Narrative:

- Children have access to a safe, secure environment where they played, socialized with other children, and received help to recover from the trauma of the earthquake. - Created an environment of normalcy and helped children and teachers recover from the trauma of the earthquake

SC did not have a program in Yogyakarta at the time of the 2006 earthquake. It did however have a team from Jakarta based in Yogyakarta to assess Mount Merapi volcano situation and had decided to provide displaced people with basic hygiene kits. On the day of the earthquake ten truckloads of hygiene kits were on the road from Jakarta to Yogyakarta. These were rerouted to Earthquake affected people. SC’s emergency response program started on May 27 2006 and completed on November 30 2006. It was followed by a recovery program that will continue until the end June 2008. Save the Children focused their response in the districts of Bantul and Sleman in Yogyakarta Province, and the district of Klaten in the Central Java Province. SC collaborated with local partners Lestari, KPI and KSPI for child protection programs in Klaten; Indriayanati and Humana for Child Protection Program in Bantul; and Persepsi and LSPPA for the Education Program. Appropriateness:

The NFIs distributed were considered to be appropriate and the distributions took place in a timely manner. In general the quality of the material distributed was considered to be good, however reference was made to the poor quality of some of the tools in the shelter kits. The distributions were also considered to be carried out in a fair manner, as they did not create jealousy within the community. This is possibly because distributions targeted all households in a single location. Yogyakarta Joint Evaluation Annexes July 19, 2007

The SPAs and associated activities were very much appreciated and considered a success by the communities in reducing trauma in children. The provision of temporary schools, plus school materials was important in providing conditions that allowed children to return to school. In two sub-districts, the heads of the education department praised SC’s training of teachers and their strengthening of school boards and student organisations. They said that with SC support teachers were socialised on the new GOI curriculum, ‘teachers were trained on staff roles and responsibilities and planning and this has helped to do their jobs more effectively’. Impact: Due to lapse in time since the distribution of NFIs, communities had difficulty in recalling exactly which agency gave what items. In general people interviewed were very appreciative of the appropriateness and quality of the items distributed and on how they met their immediate needs and allowed people to start the recovery process. From Save the Children’s own evaluation, 99% of NFIs reached their intended beneficiaries. The safe plays areas were particularly credited with having a significant impact on reducing trauma in children and encouraging them to return to school. Typically communities commented on how SC’s activities not only reduced trauma, but also increased children’s self esteem, made them more creative, helped children return to school, and improved their school reports.

SC’s educational activities received very little mention during the interviews with community members, possibly due to the time lapse since their implementation. There is no doubt that these activities also contributed to the positive responses regarding rapid recovery of children from the effects of the earthquake and the return of children to the formal school system. The Head of the Department of Education in the Sub-district of Jetis and a School Principle both gave very positive reviews of the impact of the Save the Children’s educational activities, particularly in overcoming trauma in children, building the capacity of teachers, and strengthening school boards (dewan sekolah) and student’s organisations. Adults also appreciated the fact that having safe play areas and getting children back to school allowed them space to focus on meeting other basic needs such as shelter and recovery of their livelihoods. Level of Recovery:

When asked to compare their current situation with life before the earthquake, people cited the overall community recovery rate as being in the region of 40 to 60%. However, regarding the recovery of the primary education system, interviewees estimated it to be in the region of 90 to 100%. The recovery of children from trauma and the effects of the earthquake are also estimated in the region of 90 to 100% In one FGD the participants said that the earthquake was a blessing as they are better off now than they were before the earthquake. Accountability to local people:

Save the Children works well with local communities and local and district education officials. Distribution lists for NFIs were gathered from community leaders and the decision to distribute 62

Yogyakarta Joint Evaluation Annexes July 19, 2007

NFIs to 100% of the targeted communities reduced the possibility of inter-community conflict. The use of children in the monitoring and evaluation of NFIs was a success and appreciated by the parents who were asked their opinion on the programs. The involvement of children in the selection of SPA activities was noted and appreciated by all, not least the children. The educational program was well coordinated with local officials.

One female FGD said that the aims of SC’s activities were not always clear and that they would have liked to have seen more involvement of the mothers in the program. Although not specifically targeted at Save the Children, one female FDG in Bantul commented on the fact that the community received too much aid and, that people had become lazy, the rich had become richer, and that people were better off than they were before the earthquake.

Overall the communities and local officials reported that they were satisfied with the level of coordination and information SC provided on project activities.

Monitoring and Evaluation

M&E staff were given log frames that were complicated with many objectives and indicators and not following the Good Enough Approach. However the M&E officer appreciated having a framework for the program. Save the Children did train children and had them complete a review of the SPA and NFI programs. This was innovative, enjoyed by the children and appreciated by the parents. It is a good practice which other agencies may wish to use. Preparedness

At the time of the earthquake, Save the Children Indonesia did not have an emergency preparedness plan in place. However, over the past year Save the Children launched a 2006 to 2010 global emergency response plan, which is now starting to filter into the country program’s preparedness plan. For further information on emergency preparedness please see Annex 7. Recommendations to Save the Children form stakeholders (What they could do better next time):

• Ensure better quality tools in shelter tool kits; possibly achieved through pre-positioning when better quality control can be realized.

• Communities wanted to see the SPA activities extended past the duration of the project and would have liked a greater emphasis on sustainability achievable through the training of the youth to manage the program. This request comes despite the training of 150 community volunteers and 16 district officials. It may not be representative of all project areas. • Increase the involvement of mothers in the SPA activities. As one female FDG requested, ‘so as they can become more creative and have greater value in their communities’.

• In areas of operation coordinate with other agencies to ensure a more integrated response that includes other basic needs such as shelter and livelihoods. Also ensure that people are involved in the recovery process and not receiving more than is necessary to meet their basic needs and maintain their dignity.

Yogyakarta Joint Evaluation Annexes July 19, 2007

â&#x20AC;˘

Ensure that community-based activities respect local culture and practices (for example, allocating time for prayer).

Yogyakarta Joint Evaluation Annexes July 19, 2007

WORLD VISION INDONESIA EMERGENCY RESPONSE AND RECOVERY PROGRAM MAY 27, 2006 TO MAY 31, 2007 Program Objectives

Outputs

Outcomes

Target families have access to basic and survival assistance

• 9,156 HHs receive and use NFI kits Family, hygiene, clean-up, baby and children kits provided

People’s basic survival needs are met

Target families have access basic to health care services

• 2 Puskesmas assisted with equipment/tents to re-function • 6 Puskesmas and 4 Polindes provided new semi-permanent buildings

People in 12 sub-districts have access to basic health care and MCH services (estimated population 300,000)

• 20 CFS established to provide psychosocial support to 3,300 children • 100 CFS youth volunteers trained in psychosocial support and child protection • 10 temporary and permanent elementary schools built and refurbished for 2,457 students • 2,400 students provided school kits

• Targeted children express their hopes and experiences. • Children have a normal environment in which to study

Children have access to the opportunities or services provided to them to be able to express their experiences and hopes.

Target families have a safer shelter to provide a better livelihood.

• 665 posyandus in 10 sub-districts equipped to re-function as MCH and supplementary feeding posts • 40 Puskemas and 1,448 posyandus staff trained on physical rehabilitation

• 315 HHs have newly constructed houses

Houses completed for families with children under 5yrs of age

Narrative

At the time of the disaster, WVI had a team in Yogyakarta to assess the affects of the Mount Merapi volcano alert. The team responded immediately to the earthquake and began purchase of NFI items locally for distribution. They worked in more remote and severely affected areas where few other agencies were working. The sub-districts and districts where they worked were: Jetis, Dlingo, Imogiri, Pleret, and Sewon in Bantul, Prambanan, Gantiwarno, Bayat, Cawas, and Trucuk in Klaten. Their immediate relief program ran from May 27, 2006 to November 30, 2006 and the recovery program from December 1 2006 to May 31 2007.

Yogyakarta Joint Evaluation Annexes July 19, 2007

Appropriateness: Most goods and services provided by World Vision met the priority needs of affected people. In all cases goods and services were provided at locations convenient and close to affected people. The quality of the support both for infrastructure, NFIs and services was highly rated. Beneficiaries and stakeholders agree that WV responded quickly and met needs as identified in assessments: ‘They were quick to follow-up compared to some agencies . The assistance was useful and distributed equally at the beginning.’ The health support provided by World Vision was particularly appreciated and mentioned by district and local officials as well as local people as being valuable and efficient. Women appreciated the support to posyandu’s which promoted exclusive breastfeeding by mothers and carried out a supplementary feeding program for their children. Many said that, ‘WV helped to reactivate health services at Puskesmas and Posyandu levels’.

Children gave child-friendly spaces a 9 out of 10 rating and said that activities were fun and educative. The training of youth volunteers was said to be effective. Both children, teachers and parents considered the schools provided by WV to be of a better quality than those that existed before the earthquake. Women enjoyed the training activities provided to them: sewing, cooking, etc.

It was evident from discussions with government officials that they saw WV assistance as complimenting their own and successfully proving support in locations where it was much needed due to the limited assistance from other agencies and the severe damage to houses and infrastructure. They valued how WV coordinated with government and provided them with detailed information on a regular basis. Impact:

Local people and local and district officials said that the assistance provided by WVI helped people, families and communities to recover faster and helped to ensure that people’s basic needs were met.

Many said that the support to reactivate severely damaged health facilities and train and motivate health staff ensured that people had access to basic health services quickly and that the quality of health services was good. Some said that the supplementary feeding program had prevented malnutrition of children under 5 years of age. ‘The recovery for the society was faster because WV helped health staff to get back to work faster and more efficiently’.

Local people, officials and children themselves agreed that the child-friendly space program assisted children to overcome and/or reduce trauma from the earthquake, helping to increase children’s knowledge and self-esteem. CFS activities helped them to create friendships and gain confidence in themselves. Children said the CFS activities made them happy and feel more secure, ‘We don’t have nightmares about earthquake as we did before’. Support to schools have ensured that children had a comfortable and suitable environment to study in. In places where new schools were built children now have a better quality environment to study in than before the earthquake (‘a better building with good lighting and more text books available for children’).

Yogyakarta Joint Evaluation Annexes July 19, 2007

Women said that the training provided has given them new skills. One respondent mentioned that with the new skills she has started a business.

For some families, ‘the program was a blessing’, as they now had their own house whereas before the disaster they did not. With a house these families are now able to concentrate on work and on restoring their livelihoods

Recovery: Children said they have recovered from trauma. Leaders in both villages estimated that recovery is 90% and many houses are now mostly rebuilt, although sanitation remains a concern since not all families have latrines. In one village where WVI provided a school, teachers said the school was 90% recovered and most of this was due to the support from WVI. Leaders estimated that recovery overall was 60% due to GOI support, 30% due to WV and 10% due to others. Accountability to local people:

WVI worked through the local leadership, keeping village and sub-village leaders informed of plans and regularly having meetings with leaders. Assessments were carried out with service providers and most assessment results were integrated. Community participation was high. However, WV did make some decisions independently – the design of schools, the hiring of contractors and labourers for housing, and the selection of CFS groups. This kind of independent decision-making subsequently had a negative impact on people’s sense of ownership (Gadjah Mada evaluation).

At the start of the program, WV was said to have informed all the stakeholders through meetings. However, as time went by information went mostly via the village leaders. This resulted in misinformation particularly on the housing program. People were not clear on criteria for eligibility of receiving a house, leading to reported cases of abuse. Such cases were said to have created jealousy in the community and agencies were asked to avoid this. In addition, women said they did not know who did surveys or who made decisions for the WV program. Women said they found out about activities after the work was started from village leaders. WV has followed the good practice of informing government and local people that their program has come to an end. They will keep one staff member on to monitor their phase down and ensure that their work is responsibly handed over to suitable parties. Monitoring and Evaluation:

WV had clearly defined plans with outputs and outcomes. Input and output data is available and sufficient. WV did not define any impact indicators. They did follow the good practice of having a final project evaluation carried out and completed by an external party, namely the Gadjah Mada University. The evaluation was successful and is being shared locally. Concerns:

The shelter program had created jealousy in the two different villages visited. People were not clear about the criteria for beneficiary selection on housing. Leaders had manipulated data. There were complaints about the materials for the roofing frame and while WV staff said the materials were replaced people brought the issue up again during the FGDs. A complaints system was not in place and even the children recommended that a complaints system should be established for emergency response programs so that jealousy can be avoided and aid distributed equally. Yogyakarta Joint Evaluation Annexes July 19, 2007

Many are interested in seeing CFS activities continue and are not certain if they have the ability to continue the good work started by WV. Preparedness:

WV had an emergency preparedness plan for Indonesia. For further information on emergency preparedness please see Annex 7. Recommendations to WV from stakeholders

• Ensure more community involvement – a bottom-up decision-making process is needed to ensure people are well informed and involved in the decisions, making them more acceptable and fair.

• Provide information to all people in each location on a regular basis so that mis-information does not occur and transparency is increased. • Monitor programs and ensure selection criteria for housing and other assistance is applied consistently and that criteria is clear to all and based on selecting those most in need.

• Increase community involvement, including women, in each stage of the project — planning, implementing and monitoring. Establish a complaints mechanism so that people are clear on progress, jealousy is avoided and aid is distributed in ways considered fair.

Yogyakarta Joint Evaluation Annexes July 19, 2007

Annex Nine:

Summary of findings from each village

VILLAGE ONE, KLATEN JUNE 7, 2007 Type of discussion

Gender

Number

Characteristics

FGDs

Men

Beneficiaries

SSI

Men

Village leaders (current and previous leader)

Women Women

Man/Woman Total # interviewed

Men

Affects of the Earthquake

9 2 2 2

Beneficiaries – labourers, traders Beneficiaries of house

Husband/wife of shop benefiting from voucher system Non-beneficiaries

• 992 houses of which 656 were destroyed completely and 202 badly damaged • 38 people were killed

Support received: GOI housing subsidy of 4 million and 5.4 million with balance to 20 million pending, reconstruction of primary school and mosque. Red Cross bamboo houses and housing equipment; IOM bamboo houses; JRF did a survey and will build some houses; LPTP toilets, tents and bamboo house; German Red Cross food for each family; affluent people from the city donated food and clothes; PMI food, toolkits and counselling for children; GTZ money; CARE jerry cans, tents, hygiene and HH equipment, support for water purification, and housing frame and roof.

Recovery: Recovery is about 50% in terms of housing and people’s mental state. 50% of all recovery is due to GOI support and the rest is due to the support of CARE and from the many other agencies. 60-70% recovery of housing here is due to CARE. Infrastructure here has not recovered.

Pending: Women said they are still in fear when they think about another earthquake, while men said they had recovered mentally. Houses need to be completed. What people liked about the work of the agency • The housing survey was satisfactory.

• Process of selecting families for house was considered fair and quick (claimed by both beneficiaries and non-beneficiaries – everyone knew the criteria for beneficiary selection). • Aid was spread equally and appropriately to what was needed by the community.

Yogyakarta Joint Evaluation Annexes July 19, 2007

• System of labour for house construction was good. CARE paid for the labourers, as the community did not have money to hire them. • The design and selected material for housing construction was of good quality • NFI distribution filled gaps left by others and the aid was helpful. • Women were satisfied with the services provided by CARE. • Clean water program was equal, open, and inclusive. What difference they said it had made

• Feeling of security returned after receiving shelter assistance in the form of earthquake resistant houses. There is no hesitation about living in new house, house is safe for children

• ‘Once I occupied my house, I could start to concentrate on other work’

• Beneficiaries are very satisfied with the CARE house, as people were able to control supply and quality of materials. The design is used by others in the area as it provides a uniform house size and shape • Water is clean and good for drinking

• The cooperation increased solidarity between people in the village (women, men – and the solidarity helped them to address economic problems) • Support after the earthquake motivated people; gave people the spirit to live again. How people were involved in the process

CARE set up a field office and their local partner KOMPIP worked with villagers and leaders in selecting beneficiaries for the housing program. The selection was agreed upon with leaders and the program was explained to all. CARE provided vouchers to the selected beneficiaries to be used in securing construction materials from local shopkeepers in the town. The shopkeeper had an agreement with CARE to provide supplies. The shop keeper was informed of who the housing recipients were and was involved in their selection at RT level. Shopkeeper(s) appreciated the quality control system used by CARE with their shop. The process was beneficial for the shop as they could sell goods. Women were not involved in the decision-making but attended the monthly meetings run by KOMPIP and were informed of program activities. Concerns raised

There was no complaints system and sometimes the community members were only able to voice their complaints to leaders. People said they were afraid to complain in case the shelter program was stopped.

What they want done differently next time

• Asked for shelter work to continue as many people still need housing. • More than the roof and frame of a house should be provided. 70

Yogyakarta Joint Evaluation Annexes July 19, 2007

• Assistance should be provided more quickly after the earthquake (assistance arrived after families had already started building their houses). • Provide support for economic recovery and mental health programs

• Continue to ensure that the aid is equally provided and involve the community leaders. The community preferred a higher level of involvement like in the first phase of the project, where the community helped to select the recipients of the first 150 houses. Community members disliked the level of involvement in the second phase, where CARE selected the families to receive the houses.

Note: Water purification solution is used to wash vegetables and clothes and not for drinking. Men said there was no water problem in the area after the earthquake so such support was not needed. They didn’t like the smell of the water solution. Some people noted getting sick after drinking the water with the solution so the community members became afraid of using the solution.

Criteria for recieving housing assistance was based on prioritizing the most vulnerable: widows and old men, very poor, house totally destroyed, owner of land and previous house. Village leader, sub-leader met with families at RT level and discussed and decided housing recipients together. Following this, CARE conducted a survey to verify the list. The system of gotong royong was used for houses not built by CARE.

A women house recipient said that she now has a solid home, made from stone. The house was reconstructed with help from CARE, the GOI and with her own money.

Yogyakarta Joint Evaluation Annexes July 19, 2007

VILLAGE TWO, BANTUL JUNE 5, 2007 Type of discussion

Gender

Number

Characteristics

FGDs

Men

Parents of child beneficiaries

Children

Students, 3- boys 4 girls

SSI

Women Men Men

Man/Woman Man Total # interviewed

Men

Affects of the Earthquake • •

Parent of child beneficiaries

Village Leader and a sub-village leader

Volunteers in SC program

Two couples 1 beneficiary, 1 non-beneficiary

non-beneficiary

NFI distributors

In the sub-village there were 100 houses of which 98 were destroyed. The two remaining were badly damaged 8 deaths

Support Received Government: 15 million for every family for permanent shelter construction. Habitat, Makronokia, JRF, P2KP, PEMBACA KOMPAS, IOM involved in providing either temporary shelters and/or permanent housing. UNICEF: water purification, tarps, tents and toilets. JICA health equipment, Hidayah provided a temporary hospital and tents for every family SC toys, sports equipment, and organized activities for children. HUMANA (SC local partner) distributed similar type items for children. WV, IMC, SC distributed various food and non-food items.

Recovery: The sub-village leader said the community had recovered. Others said it was not 100% but that they were well on their way. Pending: Livelihoods remain a problem (men, women). What people liked about the work of the agency: •

Basic needs were met; items distributed were of good quality, useful and came on time. (FGD men)

•

Children very much liked the picnic/review. (FGD children)

•

Children were very happy with the SC activities. (FGD female/children)

Yogyakarta Joint Evaluation Annexes July 19, 2007

What difference they said it had made • Children are happier, more confident, more creative, school reports are better, and they have more skills. (FGDs children/female/men, SSIs) • Women that are pregnant become healthier and birth mortality rates have decreased. (FGD female) • Some people now have houses that are smaller than before the earthquake (VL) • Life is getting back to normal (VL, SSI 5 males)

• People received housing assistance from two sources, allowing some to build two houses while others build one big house. (FGD female). • The program helped children to be more independent and brave. (SSI male sub village leader) • They better off now than they were before the earthquake (FGD women). How people were involved in the process:

• Children were involved in deciding which activities to include in the program. (FGD children)

• Local youths also participated in the program by supervising and teaching younger children. (FGD female)

• Adults were kept informed through parent meetings. (SSI 2 female, 1 male) Concerns raised;

The participants in the FGDs raised no major concerns about the kind of aid provided. However, there were some concerns regarding laziness (VL and FGD women). The village leader noted that people were becoming lazy and used to receiving aid from NGOs. At times when there was no aid they targeted their anger at the village leader. The same was echoed in the women’s FGD, where it was noted that the community received too much support from donors and NGOs, which made them lazy. What they want done differently next time

• Include activities for adults such as livelihood activities. (FGD men, sub village leader)

• Conduct a seminar on child abuse and children rights for parents (SSI male volunteer SC program) • Support should be provided until people are 100% recovered. (VL) • Give more lessons in school subjects and sports. (SSI 2 men)

• SC should provide more information about the program at its inception (women).

Yogyakarta Joint Evaluation Annexes July 19, 2007

VILLAGE THREE, BANTUL JUNE 4, 2007 Type of discussion

Gender

Number

Characteristics

FGDs

Men

Beneficiaries, many jobless

Children

Students 6 girls, 8boys

SSI

Women Man

Man/Woman Man

Total # interviewed

Women

Effects of the earthquake:

Housewives

Village Leader

Husband and wife-non-beneficiaries

60 year old non-beneficiary

Widows non-beneficiaries

•

168 houses of which 163 houses were destroyed by the earthquake

•

Many wells were damaged and water was polluted

• • •

26 people were killed

Irrigation system was damaged Livestock were killed.

Support received: YTB bamboo house, food, hygiene and family kit; ACT manure; WUSHU tarps, Mukhtadin blankets, Hajar Aswat money and praying kits; Trans TV mosque and jerry cans ; IOM & CHF bamboo houses; SCTV school uniform and kits; Papua shoes; WV food, tents, posyandu support, clothes, blankets, hygiene kits, school supplies, CFS, permanent houses and training for women. Recovery: Village leader estimated that they have enough aid and situation is 90% recovered and many houses are completed. Children said they have recovered from trauma. Watsan remains a problem as only 50% of HHs have latrines.

Pending: Some houses still need to be completed. Sanitation remains a concern as many do not have latrines and water also remains a problem. People still feel vulnerable and are afraid there will be another earthquake. What people liked about the work of the agency •

Children enjoyed the CFS activities as they made them happy. They gave WV 9 out of 10 for their work with children.

•

Women enjoyed the training activities provided for them: sewing, cooking, etc.

•

Children activities were fun, educative and staff did their job well (children)

Yogyakarta Joint Evaluation Annexes July 19, 2007

• The quality of all the aid was good, useful, complete and distributed equally at the beginning (women) • The quantity of the aid was more than enough (men) • The response was quick (men)

• All the needs of the people were met (women)

• The coordination and information was good at the beginning What difference they said it had made

• CFS activities reduced children’s trauma (children, women),

• CFS increased their self-esteem, gave them knowledge of team working, helped them create friendships between one another (children) • Children gained confidence even in public speaking and learned how to organise activities • Training for women provided them with new skills

• Some who did not have a house before the earthquake now own a house. • As immediate needs were met it helped families to recover How people were involved in the process:

During the distribution, male youngsters helped out. WV selected the people in-charge of CFS and shelter program. Those in charge were actively involved in the program. WV worked with village groups and kept them informed and received information on needs from them. Information was also provided to village leaders. Women did not know who did surveys or made decisions for the WV program. Women said they found out about activities after the work was started from the village leaders.

Concerns raised

• Aid should be distributed equally to limit complaints from those who may not receive assistance were it not distributed equally. Ensure there is a complaints system (children) • The materials for the houses were not of good quality What they want done differently next time

• Children wanted the library and warehouse separated

• Increase transparency and involve the community more in program decisions

• Ensure the criteria for selecting beneficiaries for houses are applied consistently and procedures for getting aid is clear to all • Keep leaders informed of progress so they are not blamed for any unfairness • Coordinate more with the community so jealousy is limited

• Ensure that the good ways of providing information at the beginning of the program continue to be applied Yogyakarta Joint Evaluation Annexes July 19, 2007

• For other NGOs – IOM etc. it is better not to give bamboo shelter as families started to build permanent shelter – so just give construction materials to build permanent houses. • WV to finish the houses they constructed before leaving

• Ensure shelter program is timed properly so the results are achieved according to schedule. • Involve the community more in checking and assessing housing provisions.

Yogyakarta Joint Evaluation Annexes July 19, 2007

VILLAGE FOUR, KLATEN JUNE 6, 2007 Type of discussion

Gender

Number

Characteristics

FGDs

Men

Beneficiaries, labourers

Children

Students, 1-boy, 3girls

SSI

Women Men Men

Woman Total # interviewed

Boys

Affects of the Earthquake

10 2 2 1 2

Beneficiaries, 8housewives, teacher Village Leader, ; sub-village leader Non-beneficiary

30+year old beneficiary

One beneficiary and 1-non-beneficiary

• 487 houses of which 167 were completely destroyed and 300 partly destroyed

• 2 primary schools and one kindergarten partly destroyed and 1 kindergarten completed destroyed • 2 streets destroyed and one bridge was broken • 54 people were injured and 2 people killed • Water was polluted for three months.

Support received: GOI subsidy of 4 million and 5.4 million received with balance to 20 million pending, government office street reconstruction and 20 houses for poorest of poor. University technical assistance on building bridge and mosque and helping people clean up the rubble; Red Cross bamboo housing and trauma healing; UNICEF school tents and equipment; PKPU primary school; PMI bamboo houses, blankets, food; JRF permanent houses; CARE drinking water, jerry cans, food, HH kits and education on filtering water, medical health free for 6 months, CRS 100 core houses, food, tents, tool kits and soccer and volley ball program for children Recovery: Children feel they are better. Less than 50% of families have recovered their physical houses but some said that livelihoods are back to normal though others said they were not. Leader estimated that recovery was about 35%, and men said it was 30%. Pending: Unfinished houses and restoration of livelihoods.

What people liked about the work of the agency:

• CRS shelter survey was good for reaching the right families • CRS shelter frame was quick to go up

• The quality of the houses were good. The roof acts as an alarm system as you can hear the rain and feel the earthquake (women) Yogyakarta Joint Evaluation Annexes July 19, 2007

• Monitoring of the program was done well by CRS/service is good (everyone) • CRS staff were nice and could teach children well (children)

• Liked that CRS was communicative – kept them informed (VL)

• Liked the CARE health program as it met a direct need • CARE encouraged people to do things for themselves. What difference they said it had made

• CRS house: psychologically families feel safer, happier more comfortable because they know the house is earthquake resistant (VL, women) • Physically, families have a house to live in where before they were staying in tents

• Children learned games and they are getting better scores in sports at school and made new friends (children, men) • CARE drinking water education: people now know how to consume safe drinking water and the river is now kept cleaner because of the CRS Watsan program (VL, men)

• CARE people were encouraged to work together

How people were involved in the process and informed: CRS and CARE called meetings with the village leader who in turn spoke to the sub-village leaders. The sub leader then spoke to the community members. CARE formed a group for men and one for women to run their project. CRS involved the leaders and men. Women were not involved in the CRS meetings. For CRS housing, village leaders identified needs and CRS checked the identified beneficiaries through a survey. Beneficiaries were encouraged to form groups of 10 people (owners of the houses) which the sub-leader was in contact with. Men said they were involved in all decisions on housing program. CRS stimulated people help themselves by only providing housing frame. CARE provided education and community members were expected to actively provide for the remaining needs. For complaints to CRS, people could go directly to CRS. CRS explained its limitations in terms of how many houses it could provide. However, CRS has yet to inform the communities of the status of the livelihood program. There was some jealousy with regards to the housing but the leaders carried out socialization to improve the situation. With regards to CARE there were no complaints as people themselves were doing everything.

Concerns raised

• NFI met needs but distribution was not equal between sub-districts. What they want done differently next time

• NGOs and GOI need to coordinate better in the future. In this particular village there was a large NFI distribution, but in other sub-districts the distribution was minimal.

• Limit the level of demand from each location so unequal NFI distribution across sub-districts does not occur

• Health work started late and should begin immediately. Health work needs to be conducted by all NGOs 78

Yogyakarta Joint Evaluation Annexes July 19, 2007

• CARE should make the proposal process simpler, it was felt to be too complicated. There should be no limitations to giving assistance. In terms of lending assistance, no proposal process should be needed.

• Continue to ensure priority is given to those most in need so that jealousies are not aggravated • Provide support for livelihood restoration

• Help to complete houses as families do not have the means to complete them themselves • Increase volleyball and soccer practice sessions from every other week to once a week Note: Children said the water purification material was used for washing clothes.

Yogyakarta Joint Evaluation Annexes July 19, 2007

VILLAGE FIVE, BANTUL, JUNE 4, 2007 Type of discussion

Gender

Number

Characteristics

FGDs

Men

Beneficiaries, many jobless

Children

Students, boys 8-14yrs old

SSI

Total # interviewed

Women Man

Woman Woman

Effects of the Earthquake

Beneficiaries, housewives Village Leader

30+year old non-beneficiary

60+ non-beneficiary

• 223 houses of which 208 were destroyed by the earthquake • 28 people were killed

• Water in the wells reduced and the water was polluted when water levels started rising again. • Sanitation badly affected as latrines were destroyed and water was polluted.

Support received: GOI subsidy of 15 million to all families with destroyed houses. GKI wood for housing, MDS construction materials, PKPU food, kitchen utensils, blankets; churches, Red Cross; CRS tents for temporary shelter, hygiene kits, jerry cans, tool kits, core house to 46 families, soccer and volleyball games and kits for children Recovery: Village leader estimated situation is 18% recovered. Many families now have houses or have started one with GOI subsidy. People don’t have furniture and unemployment remains a big problem. The village leader estimated that GOI support had contributed 40% towards recovery, society 25% and NGOs particularly CRS had contributed 35%. Pending: Sanitation is a problem as many do not have latrines. Women do not have kitchens in the reconstructed houses. Livelihoods are the biggest concern as many don’t have regular jobs. Most informants mentioned unemployment as a problem. What people liked about the work of the agency: • NFI support from CRS was fast

• Construction of the housing frame was fast, only taking two days (everyone) • House frame is a good quality construction (VL, women, men)

• Procedures for getting the house were simple and quick (women, men)

• Distribution of aid was fair (women, men – ‘there was no misunderstanding between local people and CRS or among the people in the community’) • Staff of CRS were polite and friendly (women)

Yogyakarta Joint Evaluation Annexes July 19, 2007

•

Games for children were well organised and run.

What difference they said it made

• Housing structure is earthquake resistant and people feel safer, less afraid of earthquake and more comfortable (everyone- ‘People don’t run out of the house now when there is an earthquake’ ) • The houses are of better quality than before the earthquake (VL)

• The shelter program was conducted through mutual assistance and this is continuing. (men). • Having a house allows people to concentrate on earning a living

• People living in new house are healthier than before when they were living in tents and many people were sick (women, men)

• After games children are happier, healthier and get higher sports scores at school ‘they feel more funky and cool after joining this program’ (children) How people were involved in the process:

Leader was informed and coordinated with the people. He called the heads of the households together (men) who needed housing. Three groups were formed, families worked together to clear the land and build foundations. Groups decided themselves which families would receive priority. Priority was given to old people, pregnant women and injured. Village leader said process worked well and men said CRS met with them regularly to discuss and monitor the program. CRS had a complaints system but there was no complaint as level of community decision-making was high, allowing the community members to decide most things here (e.g. the families to prioritise for housing). Women said CRS met with them after the program was decided to inform them about the shelter program. They received more complete information on the program from their husbands. Concerns raised

• months ago village members including women were asked about their economic needs. CRS requested data and the village has provided the information required, but until now there has been no follow up. What they want done differently next time

• Provide support for livelihood restoration. The core house and 15 million is not enough for the complete construction of a house. Families are selling their livestock in order to purchase housing construction materials. For instance, women said some families have had to sell 3 cows to get enough money to complete their house. Many women lost livestock and their small businesses during the earthquake and need loans to restart. • Pay more attention to women’s needs on housing. The space provided is not enough to make a kitchen and latrine (men, women).

• More oversight of aid distribution and control by CRS. Though the control and monitoring process was conducted mainly by local people themselves and considered to be fair, the NGO should also oversee the process (men). Yogyakarta Joint Evaluation Annexes July 19, 2007

VILLAGE SIx, KLATEN JUNE 1, 2007 Type of discussion

Gender

Number

Characteristics

FGDs

Men

Beneficiaries, mixed farmers/labourers

Women

Beneficiaries

Women Children

SSI

Man

Woman Man

Total # interviewed

Man

Effects of the Earthquake

Beneficiaries and CARE cadres

Students 6 girls, 5boys 12-13yrs old Village Leader

Beneficiary 35yrs old

Beneficiary –farmer

CARE CBO leader

•

600 houses of which 400 houses were destroyed43

•

Water was badly polluted

•

14 people were killed

Support Received: GOI subsidy for those who lost their house (4 million in a first instalment and 5.4 million in a second instalment. Balance is pending to bring total subsidy to 20 million). University education on building earthquake resistant housing. JRF and ACF equipment for building houses; PMI bamboo houses and tents; IFRC water filtration support and bamboo houses; P2KP housing; UNICEF temporary schools and school supplies; PT Astra Honda a permanent school and school furniture; TNI school uniforms; CRS HH kits; WVI baby foods; and CARE drinking water equipment, jerry cans, medicines, blankets, kitchen kits, education on ensuring water safe for drinking, revolving loan fund to buy water equipment.

Recovery: Less than 50% have recovered shelter to the level before the earthquake. Leader estimated that government has contributed 50% of resources for recovery and that society and NGOs contributed the rest.

Pending: Water quality remains a problem and wastewater management is still needed. Houses have yet to be completed and household items such as furniture are still lacking. Some children are still afraid though many children no longer suffer from trauma. There are still unfinished houses and people have yet to recover their livelihoods – finding work is a problem. What people liked about the work of the agency: • 43 82

Water quality survey was done well Before the earthquake most houses did not have iron rods in the walls; they were not earthquake resistant. Yogyakarta Joint Evaluation Annexes July 19, 2007

• They provided HHs drinking water containers that enabled water sedimentation • They formed groups well and kept the groups informed (VL/men)

• CARE team directly oversaw the distribution and monitored the water program effectively • Distribution was good and timely, and the quality of the goods was high (everyone).

• Distributed items met the peoples needs and were what the people really needed (everyone)

• Staff were friendly (children)

• Cadre system worked well (cadre, women) and procedure for getting aid was clear • Training on how to use and treat water was effective What difference they said it made

• Support from all the agencies resulted in many families having new houses • People know how to construct earthquake resistant houses (VL)

• CARE’s advise increased people’s knowledge on how to improve water quality for drinking (VL, cadre) and changed their behaviour on water usage • CARE’s group formation and information provision strengthened links and cooperation between people (VL), and encouraged mutual assistance more than before (men).

• Water quality has improved – water used is cleaner (men, women)

• People’s awareness on hygiene increased – hand washing (children). How people were involved in the process:

CARE organised the groups – one person per RT. Cadre assessed beneficiary needs. CARE trained the members of the cadre on proposal preparation, outlining the assistance they needed. All beneficiary needs related to water were mentioned in a proposal. Cadres met with women to discuss their needs (women). Women said they could get information about the program from the cadres and there was no missing information. Proposal could be up to million. CARE informed village leaders and groups through meetings and cadre system. Cadre met with CARE staff once per week and staff also informed village leaders. All project documents were shown to the cadres. CARE conducted a workshop with children on the benefits of sanitation, in particular washing hands. Concerns raised

• The water container provided by CARE was too small

• Tents that came from various agencies came too slowly so people had to share the tents with many. • Children said the help from CARE came late – only in December What they want done differently next time

• Provide more support for livelihood restoration e.g. soft loans to renovate fishponds, etc. • Make the proposal process simpler (not so complicated – men, cadres) Yogyakarta Joint Evaluation Annexes July 19, 2007

• All the NGOs must ensure distribution is fair (some displaced people did not receive food items because distribution was not fair -children) • Provide toilets in the camps for displaced (children)

• Continue to coordinate the work through cadre groups in the community and ensure program purpose is clear, and needs are assessed as they were this time.

• Continue CARE programs and expand the program to include a wider focus other than just water. Note: Children said they had little to keep them active just after the earthquake. Many of them begged along the road for money. Baby food provided was traded as it was not needed and there was an oversupply.

The water purification liquid provided was not used for drinking but for washing clothes (women).

Yogyakarta Joint Evaluation Annexes July 19, 2007

VILLAGE SEVEN, KLATEN JUNE 5, 2007 Type of discussion

Gender

Number

Characteristics

FGDs

Men

Beneficiaries, mixed cadres/farmers

Children

Beneficiaries - students

SSI

Women Man Man

Boys Man

Woman Total # interviewed

Men

Affects of Earthquake

10 1 1 1 1 2

Beneficiaries (7, non-beneficiaries (3) Village Leader

School teacher

Non-beneficiary - students high school Beneficiary of house

non-beneficiary 75yrs+ Non-beneficiary

•

500 houses of which 450 houses were destroyed

•

The road and two primary schools were completely destroyed including all furniture and equipment in all buildings.

• •

22 people were killed

Health of the society worsened.

Support received: GOI subsidy of rupee 4mil and 5.4million received. Balance to bring to 20million pending. GOI one primary school and government office, Habitat housing; CHF bamboo houses; PMI house material; Red Cross construction materials and blankets, UNICEF tents, Saudi Arabia food; SC a temporary bamboo school and trauma counselling, CARE family tents, WV a permanent primary school with library, school supplies for children, materials for health post, food, HH and hygiene kits, child friendly spaces and houses for 95families. Recovery: Teacher said that the primary school is 90% recovered and 70% is due to WV support. Others in FGD said the earthquake was a blessing as it helped to improve their situation; before they did not have a house and now they do. Most houses are rebuilt and the economy is recovering. Leader estimated that 60% of recovery was due to GOI support, 30% was due to WV and 10% due to others. Pending: Houses still need to be finished. Livelihood restoration is ongoing.

Yogyakarta Joint Evaluation Annexes July 19, 2007

What people liked about the work of the agency44: • The primary school’s building is of good construction and construction was carried out quickly (VL, teacher) • The health care support was needed and much appreciated (VL)

• Programs for children were done well and their needs were met (children) • WV trauma healing for children was very effective (women, children, men) • NFI support from various sources, including WV, helped cover basic needs • The houses constructed are of better quality than before the earthquake • Everyone liked the shelter programs.

What difference they said it had made • Once families have a house they can begin to focus on other needs

• Some families who did not have a house before the earthquake now have their own house

• School constructed is of better quality than the one before the earthquake and has created a better learning and studying environment (children, teacher, and leader). • The CFS program has reduced children’s fear of earthquakes (women) • Children don’t have nightmares about earthquake as before (children) How people were involved in the process

WV called leaders and village administrators together and briefed them on the work and the criteria for beneficiaries of housing. Leaders in turn were to inform other villagers. The men said they were always informed of any aid coming into the village including on the shelter program. There were some complaints to the leader about the housing program but the leader did not pass on these complaints to WV. Work on the primary school was communicated through the head master who was to inform other teachers and parents. There was no involvement in the school construction or design. The WV officer made occasional field visits to the village to inform the community of the program. Communication between SC and the community was minimal; SC came, built the temporary school and left. Concerns raised

• Housing program created jealousy between families in the village (non-beneficiaries, women) as leaders managed all beneficiary data. Some got two houses from two different sources as the program was not monitored effectively. Some leaders requested beneficiaries to give money in order to receive a house from an agency. One family got five houses from five different sources (women group). • Non-beneficiaries said that aid was concentrated in one sub-village and politically manipulated.

For the FGDs, informants focused on the shelter program mostly as the other programs had gone well.

Yogyakarta Joint Evaluation Annexes July 19, 2007

• The quality of the roof frame materials for housing was not monitored and controlled, resulting in the recipients refusal to accept the roof frame materials. What they want done differently next time

• Provide a complete housing package; as the WV houses have no completed floors or toilet facilities. ‘Better to complete a smaller number of good quality house, than many that are not’ • Employ professional labourers. Ensure the housing materials are of good quality

• Directly inform local people especially on shelter program so there is no misinformation. Coordinate with surrounding villages and non-beneficiaries to reduce jealousies •

Ensure all aid goes to those most in need.

Notes: A number of community members, including a teacher and a housing recipient did not

know whether or not WV houses were earthquake resistant. The housing recipient said he did not get help from GOI and as a result received a WV house. The leader of the village had appointed him as a beneficiary. The recipient did not know what the exact criteria for WV housing was. Other than providing food to the labourers, the recipient received a house and its key from WV. He was not living in the house as he said it was not yet finished e.g. windows not yet completed. He does not know what difference it will make as he has not moved into the house. The recipient believes otheres in the area who received WV housing have not moved into their houses. Teacher said that WV defined criteria for receipt of housing and WV told them to village leaders. Criteria –No help from others including government, a child less than 5 yrs old, family is living with another family, land to build house.

Yogyakarta Joint Evaluation Annexes July 19, 2007

VILLAGE EIGHT, KLATEN JUNE 7 AND 8, 2007 Type of discussion

Gender

Number

Characteristics

FGDs

Men

Parents of child beneficiaries

Children

4 girls and 6 boys

SSI

Women Man

Women/man

Woman Total # interviewed

Man

Effects of the Earthquake

Parents of child beneficiaries Sub village, leader

Community Volunteers (SC prog.)

Beneficiary

• Out of a total of 75 houses in the village 60% were totally destroyed, the remainder were damaged. • 2 people were killed in Sub village (A total of 12 in village) • 2 people still suffer from injuries and cannot work

Support Received: GOI initially provided RP 90,000 per person for basic needs and more recently RP 9.6 million (2 instalments) for house reconstruction. CHF distributed transitional bamboo shelters to some families. JBS carried out 2 distributions of food items that included 5 boxes of bread and 5 boxes of noodles. Muhamadiyah assisted in permanent houses; Pertimina food and kitchen utensils. JRF permanent houses; CARE water purification blankets, jerry cans hygiene kits, CWS staple food, PMI clean up kits.

SC in partnership with LESTARI provided support to children through activities such as music, story reading and writing, drawing, and the distribution of toys, books, games and other items. SC also provided NFIs such as kitchen sets, hygiene kits

Recovery: All interviewees estimated that the children had recovered well, in the region of 80 to 90% of the pre earthquake situation. Housing and over all level of recovery is estimated in the region of 50%.

Pending: Further support to complete permanent shelter reconstruction and livelihood recovery (VL) What people liked about the work of the agency:

• Equal distribution of food and NFI’s meant that there was no conflict created in the community. • The distributions met the immediate needs of the community; the items were of good quality and were distributed soon after the earthquake. (FGD men) 88

Yogyakarta Joint Evaluation Annexes July 19, 2007

• All groups and individuals interviewed were very happy with the Child Support Activities and felt they had a positive effect on the recovery and wellbeing of the children. What difference they said it had made

• According to all interviews the children support activities were very successful and greatly helped the children recover from the trauma of the earthquake and return to school. • The children cited that they felt much happier now, have more friends, were more confident and doing better in their schoolwork. The adults’ interviewed also agreed with these outcomes. • Having organised activities for the children allowed the adults to concentrate on other tasks such as repairing shelters. How people were involved in the process:

The RT leader made beneficiary lists. The process was as follows; RW leaders collected data and passed it on to sub village leaders who passed it on to the village leader who shared it with SC. (FDG men/women).

Children were involved in selecting which activities they wanted in the program (FGD children). Both male and female FGDs felt that they were well informed about the program activities through monthly meetings. Concerns raised:

None of the interviewees raised concerns about the SC activities. While the community very much appreciated these activities there were some comments on how more important needs such as housing and livelihoods was not adequately addressed. (FDG men, SSI man, VL)

FGD women would like to see greater involvement of mothers in the program and felt that it was not always clear what was the aim of the project. What they want done differently next time:

• Make the program more sustainable through the training of youth (FDG children)

• Involve mothers more in the program and build their capacity so hat they can be more creative and have greater value to their communities (FGD women).

Yogyakarta Joint Evaluation Annexes July 19, 2007

Village Nine, Sleman June 8, 2007 Type of discussion

Gender

Number

Characteristics

FGDs

Men

Beneficiaries of NFI

Children

N/A

SSI

Women Man

Woman Woman Woman

Total # interviewed

Effects of the Earthquake:

2 housewives, trader, wife of RT

Sub Village leader

Grandmother (50 yrs), beneficiary

Housewife (33 yrs), beneficiary

Non Beneficiary of this program – lives in different district divided by pathway with Sleman

1 18

• Out of 3300 houses 2301 were totally destroyed. • 69 deaths.

Support Received: Oxfam provided shelter and money for livelihood assets. Through a voucher system CARE provided households with food non-food items. CARE, PMI, Cordaid, Kendaulatan Newspaper, and PELINDO distributed items such as blankets, tents, hygiene kits, kitchen sets, jerry cans and water purification tablets. The food voucher system operated for 3 months, the NFIs was a one-time distribution. The GOI provided grants of 15 million to 2,301 households for reconstruction. WANGO constructed 71 dome shelter units. ICRC provided blankets and shelter materials. CRS provided toilets, and 60 framed temporary shelter units. The GOI has provided more that 50% of total inputs. Recovery: Approximately 50% recovery

Pending: Livelihood support is still required. CRS collected livelihood data, but no follow up on what they are proposing to do. (SVL, FGD female, SSI female). Training for old women POSYANDU (FGD female). What people liked about the work of the agency:

• The voucher system was good, timely, appropriate, good quality products and helped local shops. (SVL, FGD male, SSI female, SSI male)

• Allowed people to focus on other activities (house reconstruction) as their food needs were being met. (SVL, SSI female) • Some people only had one house before the earthquake but now have 2 (FGD female)

Yogyakarta Joint Evaluation Annexes July 19, 2007

• Made the life easier for the leaders as they did not have to organise distributions. (SVL)

• Provision of shelter kits (cleaning kits) was very useful, raised community involvement and allowed the rubble to be cleared quickly. (FGD female).

What difference they said it had made:

• The basic household needs were met. (FGD female)

• Outside support raised the spirit of the community. (SVL, FGD male/female)

• The support decreased stress and reduced trauma and made people happy. (FGD male/ female) • The voucher system helped local shops (SVL How people were involved in the process:

• Vendors, community members and leaders consulted in the design and implementation of the CARE voucher scheme.(SVL, FGD male/female)

• While the community was happy with the design of CRS’ transitional structure, they were not involved in the planning of the project. • CARE had a clear plan of what they were proposing to do where as CRS did not.

• The community were kept informed of what was happening in the voucher program though meetings with village leaders and vendors. (SVL, FGD male/female) Concerns raised:

• Multiple collection of beneficiary data at the start of the emergency response. (SVL)

• The number of shelters provided by CRS was less than what was required and made it difficult for the community leaders to choose the recipients.

• 3 out of the 4 women at the FGD said that the water purification was not needed and some women used it to wash clothes.

• Voucher system did not include all members of the community. People modified the method and took the initiative to redistribute the vouchers equally between rich and poor. However rich people felt humiliated to know that they got the help from decreasing quota of the poor. They felt they were taking something not meant to be for their benefit. (FGD female).

What they want done differently next time: • Vouchers;

- Pay some of the money upfront to the vendors (FGD male, SVL) - Only 7,000 out of 11,000 people received vouchers, including full vouchers for babies. Better

if voucher allocation for babies when to other families not in the program. (SVL) - The voucher scheme should include all people including the more affluent (FGD female) • NGOs need to be more decisive when selecting beneficiaries and not have to keep repeating data collection. Yogyakarta Joint Evaluation Annexes July 19, 2007

â&#x20AC;˘

Initial distribution of food and tents could be quicker (SVL)

Note: The district boundary of Sleman and Bantul consists of a pathway which splits the community. According to some of the SSIs the people in Sleman received more and better assistance than those in Bantul i.e. people in Sleman could qualify for Oxfam transitional bamboo shelters and latrines while only a couple of meters the other side of the pathway people did not receive this type of support. The interviewees suggested that people just accepted this as the way NGOs operate and it did not create any animosity between the two communities. People in Bantul have only received the first two GOI house construction instalments, while people in Sleman have received all three.

Yogyakarta Joint Evaluation Annexes July 19, 2007

• The plan consists of an assessment or risk analysis – different risks are identified and their likelihood gauged. Planning revolves around these scenarios. • Sub-offices are organized into regions or zones within a country program. In the case of an emergency, the national office is always notified, but if the emergency is small, it is handled by the sub-office or region/zone. If the emergency is larger or outside support is needed, the national office will intervene or simply provide extra support.

YES; - a national IDPP (Initial Disaster Preparedness Plan) was available (including: identification of hazard, logistic plan, resources needed, etc).

The New Plan consists of;

Standards are the same as SC’s normal ones, but special conditions are outlined in the emergency plan. There are guidelines on how to set up offices and operations in new locations as well.

• Save prefers to enter into alliances with other organisations to respond, especially within the Save network.

• Each office has to have a staffing contingency plan – in case staff need to be deployed, so that other staff can cover for them.

• If a disaster results in 10,000 victims, the country program is obligated to intervene. If 100,000, there is global, agency-wide action taken. For smaller emergencies, global resources and staff can be called upon. For emergencies with less than 10,000 victims, the country program decides whether to take action or not.

• The local field offices must coordinate with the national office to assess the emergency, and determine the level of response

NO; - (SC recently developed new global emergency plan, for 2006-2010. This plan was not in place in Indonesia at the time of the earthquake.)

Annex Ten: Summary table on emergency preparedness of each agency

Planning Did a plan exist? (May 2006)

What does the plan consist of?

Standards

Operating procedures are the same as regular WV ones, but the NDPP plans specify modifications (as basis for regional IDPP) that may take place for emergencies.

CRS

NO; - CRS did not have a specific field office or national Indonesia emergency plan in place at the time of the earthquake. Agency and regional emergency strategies do exist. Prior to 2005, CRS in Indonesia had a separate emergency response team (ERT). This was phased out due to funding issues.

CRS Indonesia plan to re-establish a national ERT in the coming months. Once in place the team will lead the development of a national emergency preparedness plan.

As part of regional strategies, country programs respond to emergencies with local partners, and receive support at the regional level and from the agency-wide Emergency Response Team.

There is an agency-wide Emergency Handbook that provides guidelines and procedures for emergency response. Standard operating procedures in terms of finance, administration, HR and logistics are not yet adapted to emergencies. CRS at HQ level is in the process of adapting them for emergencies

CARE

NO; - At the time of the earthquake CARE had no plan in place however CARE are currently in the process of developing a Country wide EPP. They expect first drat to be ready by August 2007.

The New Plan is focused around CII key sectors and encompasses the following approaches:

• Safer water systems

• Immediate emergency shelter supply

• Market-based relief to recovery.

CARE has emergency finance, procurement and HR SOPS

Yogyakarta Joint Evaluation Annexes July 19, 2007

Resources Financial

Material

Human

A fund does exist in WV that allows for assessments and initial responses to emergencies before or while fund-raising efforts are conducted.

New Plan;

• International staff may stay longer

• Although skilled, they do require orientation upon their arrival, as many have not worked in Indonesia.

• SC has a roster of 100 staff for its global team. In reality, some are not always ready to be deployed.

• National staff are often deployed for a maximum of 2 weeks. Some are permanently assigned.

• Contingency plans at offices prevent gaps form emerging when staff are deployed.

• Staff in closest offices are responsible to respond to and often staff emergencies.

• There is a national level team of designated staff with experience also ready to be deployed

• There is an “office in a box kit” that is already pre-prepared that contains items, equipment, and forms needed to set up an office or operations for an emergency.

• SC maintains 2 warehouses in Indonesia with NFIs and water or water equipment as well.

If a disaster fits the profile of having 10,000 victims or more, it automatically qualifies for 200,000 pounds from an agency-wide emergency fund - approval from London for this. Some with less victims, such as Yogyakarta may qualify if justified in terms of scope of damage, etc.

• List of NFIs and suppliers are kept on-hand • WV has a Field Facilitator and Procurement Staff (generally in Jakarta) who have a network of contacts with suppliers. • The General Service Officer in Jakarta has contacts with different transporters as well as government offices so that items can be sent to different areas quickly.

• Staff from the closest WV office are responsible to react first - contacting national and other offices, compiling initial information, etc. • Staff from surrounding offices, in the sub-national region or zone can be mobilized

• If the emergency is large, staff from other zones will also be sent in, and quite often more senior or experienced staff from the national office. • For large emergencies a global team of international staff will be deployed

• WV Emergency Specialists in NO level are available (ready on call) and continuously monitor disaster in Indonesia

CRS

There are funds available for assessments and some initial responses but no set amount. CRS field staff interviewed were not clear about the availability of these funds and how to access them.

• CRS pre-positions NFIs, usually several hundred kits consisting of standard NFI items up to 1,500 in Java and Sumatra. Because CRS distributed to volcano victims before the earthquake its Java stock was already depleted for the earthquake. • The CRS Jakarta office has contacts with vendors and transporters, and thus the capability to procure needed items and deploy them to locations.

• Staff in closest offices are responsible to respond.

• Staff from other national offices can be mobilized to respond. • CRS has a global emergency response team, which can be called upon.

• There is a new Regional Emergency Technical Advisor.

CARE

Can access funds from CARE US ($50,000) and CARE Canada and CARE Australia ($20,000) for immediate response.

• No pre-positioning of material

• Because of existing SWS (water purification) project in Tangerang Java, CARE had immediate access to large quantities of chlorine.

• CARE has existing vendor contract with soap supplier for Aceh program.

• Also have in-country experience in implementing MBR (voucher) programs which relies on the use of local vendors for the supply of food and non-food items.

• Staff from other offices within the country are mobilized and seconded to emergency program for one-month (practice rather than policy).

• International technical staff may also be seconded or contracted depending on need.

• CARE have an emergency team based in Geneva.

Yogyakarta Joint Evaluation Annexes July 19, 2007

What went well?

What could have been done better?

- Combination of emergency specialists and development staffs contribute to the success of emergency response.

SC - Deployment of experienced CRS staff (national and worldwide).

CRS

- Establishment of a national emergency team/ unit ASAP in Indonesia.

- Completion of emergency preparedness and contingency plans for Indonesia.

- Already working on the ground on Mount Merapi and CRS was able to redeploy team and NFI kits and carry out distributions early and quickly.

- Good network of local partner organisations to work with on Yogyakarta emergency response.

Lessons learned from Aceh in terms of already having some type of national plan in place to know who or which offices would go in first, do assessments, and learning the importance of coordination.

- CFS is one of the strongest approach for WVI to response to the basic needs of the community through the children - Local staff are WVIâ&#x20AC;&#x2122;s biggest investment for direct monitoring in the field level. - Strong coordination with local government bodies. - Strong commitment of the staff: to be creative and efficient in managing resources - Large funding resources and efficiency in managing the available funds. - More preparation and capacity building for staff who did not have as much experience especially new staff - Continuous reflections from the lessons learned - Stronger coordination with other agencies involved in the response.

CARE

The rapid mobilization of donor funding by CARE US

Mobilization of experience staff from other field offices

Mobilizing stocks from other warehouses

SOP for procurement should be more flexible as it could be to ensure the rapid procurement of distribution items

Yogyakarta Joint Evaluation Annexes July 19, 2007