Terms of Reference
End of Project Evaluation Sri Lanka Youth Economic Leadership Project About Plan International We strive to advance children’s rights and equality for girls all over the world. As an independent development and humanitarian organisation, we work alongside children, young people, our supporters and partners to tackle the root causes of the challenges facing girls and all vulnerable children. We support children’s rights from birth until they reach adulthood and enable children to prepare for and respond to crises and adversity. We drive changes in practice and policy at local, national and global levels using our reach, experience and knowledge. For over 80 years we have been building powerful partnerships for children, and we are active in over 75 countries. Plan International Sri Lanka (PISL) is commissioning an end of project evaluation for the Sri Lanka Youth Economic Leadership Project (YEL). The project is scheduled to end in June 2019. PISL therefore intends to capture the main learnings from the YEL in order to design new and better initiatives for young people, especially young women in Sri Lanka. Plan, in Sri Lanka currently works in five districts. The districts are: Monaragala, Ampara, Anuradhapura, Batticaloaw and Mullative covering nearly 238 Grama Niladhari Divisions in the five Districts. At present Plan Sri Lanka works with a caseload of 20,227 sponsored families. For operational purposes the program areas are grouped into two main geographical zones as the North-central program area and Uva Programme area. Plan has also moved into the conflict affected Eastern Province and Northern Province as the organization continues to cover the most marginalized children in the country. In FY 19 Plan has planned to move to Mullative district as the new sponsorship programme area. Plan Sri Lanka’s five-year strategic plan that commenced on 1st July 2016, is planned to be concluded on 30th of June 2021. PISL will work on four Strategic Programmes which will interweave outcomes of vulnerable children, girls and women on thrive, learn, lead, and decide:
Girls and boys under 5 years, (especially with disability/development delays) achieve optimum growth and development; high prevalence of undernutrition among underfives in poor and post conflict areas will be addressed through holistic, streamlined programming for optimal growth and development.
Excluded girls and boys (especially Children with Disabilities/development delays) complete quality, inclusive compulsory education by addressing gender-based and other discriminations.
Girls and boys, including those with disabilities, are better protected from all forms of violence, particularly sexual abuse, corporal punishment and domestic violence.
Youth (female and male) and women excluded from the benefits of development have the skills and support networks to become socially and economically empowered citizens and gain access to responsive government services.
1
Project Background The YEL’s overall goal is to increase young people, especially young women’s, economic resilience by enhancing their skills and knowledge and offer alternative pathways to decent work through wage employment or entrepreneurship. YEL aims to change (outcomes) are that young people, especially young women, have acquired demand-driven skills and successfully transition to decent work and that the private sector increased its engagement in youth employment initiatives and or decision making processes. The YEL’s main activities are to provide demand-driven technical education and vocational trainings (TVET) through third parties to young people, especially young women in identified sectors. Respective TVET providers were identified, vocational courses modified and or adapted and young people enrolled. TVET was divided into three learning areas: (i) class room theory, (ii) classroom / workshop practical learning and (iii) employer workplace learning and training. Partnerships with local employers are forged to allow workplace learning and eventual work placements. TVET training mainly aimed at preparing young people for wage employment. Alternatively the YEL provided entrepreneurship training to young people to equip them with skills and knowledge to start their own business through self-employment. Young people were in addition linked up to relevance services such as mentoring and access to finance. Greater engagement of the private sector in youth employment initiatives is fostered through roundtable discussions, topic seminars, and network development bring together public and private actors to learn from each other, share experiences and develop strategies and adopt mechanisms to ensure youth-friendly and women-responsive work opportunities and placements. The YEL has had three phases: i. ii. iii.
January 2014 – June 2015 (evaluation available) July 2015 – June 2016 (mid-term evaluation available) July 2016 – June 2019 (to be undertaken)
The phase under investigation in for this end-of-project evaluation is July 2016 – June 2019. The geographical coverage of the project is the Eastern Province, Ampara and Batticaloa districts.
Evaluation Focus The purpose of this end-of-project evaluation is (i) learning and (ii) improvement by assessing the merit and value of the project, arrive at conclusions and provide recommendations for future programming and project design. Firstly, PISL intends to learn from the evaluation of what worked well in achieving outputs and outcomes with a specific emphasis on young women. Further, PISL is interested in learning of what did not go too well and what reasons can be identified. What and where are the shortcomings and what would have been possible alternative solutions? Are there any strategies, approaches, angels, and methodologies the project should have taken especially to the benefit of young women? Secondly, PISL is keen to take the learning to improve overall programming and project design. Here PISL is interested in which specific activities, strategies and approaches could be replicated in other interventions. 2
The evaluation is embedded in Plan International’s Research and Evaluation Agenda. The Agenda identifies areas of where to best focus efforts and resources in generating relevant evidence that will fill sectoral knowledge gaps and enhance influencing and programming. Within the thematic area of youth employment and entrepreneurship the YEL end-of-project evaluation should strive to make a contribution to start filling knowledge gaps in:
Plan International’s role in influencing private sector practice for equality and diversity: here PISL is keen to know how to overcome barriers for young women’s increased labour force participation in Sri Lanka. o What are the main barriers to young women to enter in to non-traditional TVET sectors? o What are the most effective methods to overcome poor social norms and attitudes that effect for young women as barriers to enter in to non-traditional TVET sector
Transition to employment for young people graduating from non-formal education: here PISL is interested in learning to what extent alternative trainings such as entrepreneurship and life skills training lead to a successful transition to work, can be made more relevant and which additional skills could be important to young women in Sri Lanka.
Barriers and enablers to young women’s asset creation and labour force engagement: here PISL would like to deepen its understanding of the driving social and gender norms that influence young women’s labor force engagement. Key issues are: women’s mobility as well as ascribed roles and status in the household and community. What can be learned about norms, practices, and behaviours in the Sri Lankan communities that either discriminate against or empower young women to improve their skills and engage in the labor market?
Evaluation Criteria The following DAC criteria and other criteria should be assessed as part of the evaluation: DAC criteria
Plan criteria
Relevance & Appropriateness Testing underlying assumptions Extent to which the project meets the needs of project beneficiaries, especially young women Effectiveness Extent to which project objectives were achieved, the reasons behind, Assessment of the quality and value of the project Unintended positive or negative outcomes/consequences for anybody involved or affected by the intervention, especially young women Sustainability Probability of continuation of long-term benefits to young people, especially young women Continuation of project structures, approaches, methods by project partners after the interventions phases out Impact: What are the long-term effects? Gender & Inclusion 3
The extent to which the project applied gender and inclusion sensitive approaches and explicitly aimed for results that improve gender equality
Key Evaluation Questions (KEQ) The consultant shall answer the following key evaluation questions (KEQ). However, the consultant is encouraged during the inception phase of this assignment to critically review the stipulated KEQ and make suggestions and provide feedback. It is expected that the consultant will finalize the KEQ together with the project team on the ground. Criteria Relevance Appropriateness
Effectiveness
KEQ & How well did the project outcomes and project strategies suit the needs and priorities of young people, especially young women? Any unmet needs the project failed to address? - Social norms and attitudes on girls engagement in TVET sector - Privet sector collaboration with TEVT sector to increase the demand driven skill development To what extent are young people, especially young women better: Skilled in Vocational training Able to pursue wage or self-employment opportunities What helped and what hindered them (consider both project strategies and contextual factors)? To what extend project able to change poor social norms and attitudes in society, effect for young women as barrier to TVET sector. To what extend project able to link the privet sector in to the TVET sector for increase the demand driven skill for young men and women.
To what degree did young women benefit from the project? What are some of the structural and or normative barriers or enablers that affect young women participating, engaging and befitting? What are the structural and or normative barriers tor enablers that influence young women’s participation in skills training and choice of employment pathway they want to pursue? Consider: the community; family; infrastructure; practices / environment of workplaces; practices / environment education institutions Impact Sustainability
Gender & Inclusion
To what extent do young people succeed in their jobs (wage employment or self-employment)? How project impact going to continue with partners/stakeholders - Gender sensitive carrier guidance - Addressing gender inequality issues with VT providers - Leadership training module What are some of the commonly held ideas, beliefs and practices at the individual and household level that affect young women to: 4
Access and participate in project activities, especially skills trainings Participate and make full use of work placements (on-thejob training) and Decide on their choice of employment pathway, e.g. wage or self-employment
Did the project address any structural and or normative barriers / harness enablers for young women to engage in the project? Were there any specific strategies developed to target young women? What do different stakeholders see as main barriers to young women’s participation in skills trainings, work-placed learning and choice of employment pathways? What do they identify as possible solutions?
Users of the Evaluation The findings of the evaluation will be shared with and used by a range of stakeholders, not only for reporting purposes but to inform future youth employment and entrepreneurship programming by Plan (in Sri Lanka and elsewhere) and potentially, for advocacy and policy influencing purposes. Audience
Funders of the current project (DFAT/ ANCP)
Use
Reporting and compliance requirements.
Guide assessment pieces to be completed as part of the start-up phase of the ADB project Guide the detailed design of the ADB project and future youth employment and entrepreneurship programming in Sri Lanka and beyond Accountability to current project stakeholders and participants. Reporting and compliance requirements Guide future youth employment and entrepreneurship in PIA’s portfolio Inform future research by PIA in the area of youth employment and entrepreneurship and women’s economic empowerment Contribute to knowledge and evidence related to key priorities in Plan Global’s Evaluation and Research Agenda. Inform future programming and research in Plan’s youth employment and entrepreneurship and women’s economic empowerment work globally.
Plan Sri Lanka Plan International Australia (PIA)
Plan Global
Stakeholders/ participants in current and future Plan YE programming
Inform the design of future policy and practice around youth employment and entrepreneurship and women’s economic empowerment in Sri Lanka
5
Methods for Data Collection and Analysis The detailed methodology for the evaluation/ research will be developed with the selected consultant, Plan staff and other relevant stakeholders. But some overarching parameters have been identified, that must be incorporated in the methodology:
Data collection must involve stakeholders who were involved in the current project and stakeholders who have not been involved. Stakeholders may include: Women who are not engaged in education/ employment Women who are engaged in education/ employment, including in nontraditional industries Women who experience multiple layers of disadvantage (e.g. women with a disability, women from rural/ remote areas) Private sector actors, including business or industry associations/ councils Relevant government representatives Educational institutions The evaluation and research will include primary data collection (largely qualitative) and take into account project monitoring data and secondary data sources. This will allow the consultant to triangulate multiple data sources, to draw robust conclusions about the current project’s effectiveness and learning for future programming. The evaluation and research should adopt a capacity-building approach for Plan Sri Lanka staff, ensuring that the project team is involved in key steps of the evaluation/ research, including methodology design, data collection and sense-making
Deliverables The following deliverables are expected from the consultant: Deliverable Preparation and facilitation of evaluation and research design workshop Inception Report including: Finalized KEQ Methodology and Analysis Agreed timeline of activities Overall activity plan Data collection tools Ethical considerations Consent forms for any primary data collection Draft methods for data analysis Presentation for plan staff Preparation and facilitation of “sense-making” evaluation draft findings workshop Draft evaluation report Final evaluation report with executive summary Final data collection tools Cleaned data Completed consent forms
Date Early March for 2 working days 7 days after the design workshop
End of April / early May (1 day) Mid May End of May End of May End of May End of May
Timeline Activity/ milestone Contract signing
Deadline 1 March
Lead PISL
Level of effort in days 0
6
Preparing design workshop Facilitating design workshop Drafting Inception Report including Finalized KEQ Methodology and Analysis Agreed timeline of activities Overall activity plan Data collection tools Ethical considerations Consent forms for any primary data collection Draft methods for data analysis Data collection Data cleaning Data analysis Sense-making workshop (including preparation) Draft report delivered Final report delivered
Consultant Consultant
1 2
Consultant
5
15 April 19 April 3 May 8 May
Consultant Consultant Consultant Consultant
10 2 5 2
17 May 31 May
Consultant Consultant Total consultant
5 3 35
Held by 15 March 29 March
Budget The consultant is expected to provide a financial proposal to Plan International Sri Lanka. The financial proposal should be inclusive of VAT and taxes, travel costs to Eastern Province, and other expenses. The payment schedule is as follows: Milestone
Amount to be paid (in %) Signing of contract 10 Design workshop and Inception Report 30 Data collection completed and “making 30 sense workshop” Final Evaluation Report 30
Expected timeframe 01 March End of March End of April Beginning of June
The duration of the assignment is March to May 2019 (three months). The estimated level of effort is expected to be 35 full working days spread over the three months.
Ethical and child protection statement Plan International is committed to ensuring that the rights of those participating in data collection or analysis are respected and protected, in accordance with Ethical MERL Framework and our Child and Youth Safeguarding Policy. All applicants should include details in their proposal on how they will ensure ethics and child protection in the data collection process. Specifically, the consultant(s) shall explain how appropriate, safe, non-discriminatory participation of all stakeholders will be ensured and how special attention will be paid to the needs of children and other vulnerable groups. The consultant(s) shall also explain how confidentiality and anonymity of participants will be guaranteed.
7
Permissions The consultant will be responsible for obtaining the necessary permissions to implement the research data collection at the local level. Ethics and child protection The consultant is obliged to respect the following ethical and child protection requirements:
Fair and inclusive: The consultant should seek the views of various stakeholders: CSO members, their families, community, partners and government actors and be able to identify and address potential conflict of interest and unequal power relationships. Special efforts should be made to make the evaluation process child-centred and sensitive to gender and inclusion.
Conflict of interest: The consultant must demonstrate the necessary independence and declare any conflict of interest and potential biases, including bias towards any of the stakeholders, target groups, types of research methodologies or approach, social, political or religious prejudice. Based on rights and ethics: The consultant must respect the rights and dignity of participants as well as comply with relevant ethical standards and Plan’s Child Protection Policy and Code of Conduct. The research must ensure: appropriate, safe, non-discriminatory participation; a process of free and un-coerced consent and withdrawal and confidentiality and anonymity of participants. The informed consent of each person (including children) participating in data collection should be documented.
Expected Qualifications The minimum qualifications for consultant are as follows:
Postgraduate degree in evaluation, sociology, psychology, pedagogy, social work, anthropology, international development or related field; 10 years working experience 7+ years of demonstrated working experience in leading and carrying out project evaluations; Proven project evaluations in Sri Lanka is a must; Strong command and skills on evaluation methods such as qualitative inquiry through interviews and focus group discussions Strong command and skills on evaluation methods such as quantitative inquiry through surveys paid with proven statistical skills Proficiency in data analysis, especially qualitative data analysis and analysis methods such as but not limited to grounded theory and qualitative content analysis Evaluations in the field of youth employment, skills development, women economic empowerment, youth empowerment, enterprise development and entrepreneurship a must Experience in working with youth, especially young women and communities is a strong advantage Fluency in English and Sinhala and Tamil is a must 8
Application The technical proposal and financial offer should be submitted as soft copies (PDF) for following email address. sunitha.siyambalagoda@plan-international.org
Interested consultants or agencies should submit their proposal in English on the basis of the TOR. Technical part: Detailed response to the ToR Proposal covering: (i) methodology, timelines, CV, example of previous work Financial part: Detailed budget, including daily rates, expenses, taxes, time line etc.
Contact details Plan International Sri Lanka (PISL) will form an internal steering committee led by DCDProgramme Director with the support of M&E Unit and Plan International Asia Hub Mr. Tillmann L. Guenther, YES Technical Specialist to manage and guide the review. The steering committee will provide operational and technical support to the review, monitor progress through discussions and presentations with the team of consultants and review the inception, draft and final reports and ensure quality throughout the process. The PISL staff will also provide support to the extent possible in accessing internal documents relevant to the review, access to beneficiaries and communities and projects that will be subject to the study. The PISL staff will be organized the orientation workshop with stakeholders to present the review methodology and inception report. Plan International Sri Lanka focal point for contacts: Name: Sundari Jayasuriya Title: Deputy Country Director Programmes E-mail: sundari.jayasuriya@plan-international.org For technical questions: Name: Shehan Fernando Title: MER Specialist E-mail: shehan.fernando@plan-international.org For Administrative questions: Name: Sunitha Siyambalagoda Title: Administrative Assistant E-mail: sunitha.siyambalagoda@plan-international.org
9