W+ Monitoring and Evaluation System 2017
The Purpose of the W+ Monitoring and Evaluation System The W+ Monitoring and Evaluation System is implemented to serve a range of both internal and external scheme performance feedback functions. Internally, collected data on performance indicators related to the Standard’s administration shed light on opportunities for improved efficiencies and effectiveness and help calibrate M&E and administrative resource needs over time. Externally, collected data on performance indicators reveal the scope and type of impact the W+ Standard is having in relation to its objectives; providing opportunity for refinements in communications, training, assurance, and administration. Another key external benefit of the W+ M&E system is to help WOCAN and the W+ Standard tell their story. Indictors and data are a powerful way to demonstrate the profound positive impacts the W+ Standard has on women, their families and communities, and confirms that applying the W+ is a useful tool and sound investment for project implementers.
The WOCAN Theory of Change WOCAN’s Theory of Change is to: “Advance women’s empowerment and collective action to tackle climate change, poverty and food insecurity within enabling environments.” In addition to the capacity-building, training and coaching WOCAN provides to women’s organizations, WOCAN has created the W+ Standard, with its six domains: health, time, leadership, income & assets, knowledge & education, and food security. The W+ Standard is applied by project implementers, where they implement specific activities, with measurable outcomes from a pre-project baseline, in one or more of the six domains. The application of the Standard is intended to have lasting, positive impacts, and result in a positive business case for the project implementer.
March 1, 2017
1
How the W+ Supports the WOCAN Theory of Change The W+ Standard supports the WOCAN Theory of Change in two ways: 1) By measurably addressing women’s empowerment opportunities, as well as leveraging projects which may be working directly, or indirectly, to address climate change, alleviate poverty or food insecurity, or provide many other benefits. 2) By returning assets from project revenues back to women in project communities, further empowering and supporting women and their groups.
The Elements of the W+ Monitoring and Evaluation System As an outcome of the way in which the W+ Standard is administered, projects applying the W+ must submit Project Idea Notes (PINs) and further expand upon those ideas in fully-developed Project Design Documents (PDDs). These documents provide many relevant data points from which indicators can be drawn. Additionally, projects undergo independent verification (auditing) by W+ accredited verifiers. These verification reports yield additional data, with project specific indicators which can easily be aggregated across all projects applying the W+ Standard for broader view of the impacts of the W+ Standard. Scale of Impact: The W+ Standard may be applied to large or small projects, within individual organizations, or may be incorporated into other standards or programs, where the W+ Standard and its six Domains may reach into diverse sectors and applications. Type of Impact: The W+ Standard may be applied in a diverse array of project types, settings and have diverse outcomes, as determined by the W+ Domains applied. W+ Organizational Effectiveness: The administration of the W+ Standard strives to be effective, efficient and fair. Through the interaction with project implementers and program partners or collaborators, the W+ Standard administration can gain insights about its systems and continuously refine and improve. Level 1 Indicators: These indicators are easily collected from existing reports from project implementers and project verifiers and communications between the W+ Standard Coordinator and project implementers or verifiers. These indicators typically measure short – to – mid-term performance, but are also useful for documenting a variety of trends. The following data are collected on an ongoing basis, and compiled for regular “Impact Reports” (See Appendix 1):
Scale of Impact:
March 1, 2017
2
o o o o o o o o o
Number of W+ PINs submitted Number of W+ PDDs submitted Number of W+ Projects Number of collaborating/partnering standards and certification schemes Number of collaborating/partnering organizations and investors Number of W+ units generated Number of W+ labeled carbon units generated Number of women/households benefitted Project locations
Type of Impact: o How many W+ Domains have been applied Examples of W+ Project Activities for each o How many women receiving direct payments o The total amount of direct payments o The average amount of direct payment per woman or per group
Organizational Effectiveness: o Number of media interactions o Number of stakeholder interactions o Number of stakeholder contacts for outreach o Number of newsletter subscribers, website ‘hits’, twitter followers o Number of W+ accredited verifiers o PIN, PDD and Verification Report review turn-around times o Number of presentations, lectures, courses, webinars or other W+-related instruction provided
Level 2 Indicators: These indicators are to be collected on approximately an annual basis. These indicators help WOCAN and the W+ Standard understand longer-term trends and the issues or concerns of project implementers and auditors. Surveys of project implementers to collect data about: o Cost and time needed to design and implement W+ project activities and to verify project outcomes o Estimated value or benefit of the application of the W+ Standard to their project: monetary (quantitative) or performance (qualitative) Surveys of W+ accredited verifiers to collect data about: o Adequacy of verifier training o Clarity of verification guidance o Challenges encountered Level 3 Indicators: These larger-scale indicators are based on analyses which may look at data and level 1 or level 2 indicators covering longer periods of time to evaluate trends, or which may take retrospective looks at community-scale impacts. These studies may
March 1, 2017
3
take place every 4 or 5 years, and are predicated on having enough study candidates or a long enough data set to make trend analysis meaningful. The following are examples of possible future Level 3 M&E studies:
Evaluating the longevity of beneficial impacts of W+ Project activities Evaluating the long-term economic and social impacts of direct payments to women and their groups Evaluating the marginal additional value of W+ labeled carbon units across different carbon registries and project types vs. the performance of the carbon market overall Evaluating the value of guidance provided by the W+ Standard as a framework and set of indicators for project planning
W+ Monitoring and Evaluation System Stakeholders The following groups represent the main W+ M&E system stakeholders: Women beneficiaries Project implementers Buyers of W+ Units or W+ Labeled Carbon units W+ Accredited Verifiers International development organizations and governments Philanthropies and funders Project Investors Companies and value chain ‘actors’ In the course of utilizing the W+ M&E system, each category of stakeholder may be asked to provide, or may receive, different kinds of feedback or information. Data collected to demonstrate program performance (indicators) may be more (or less) relevant to each stakeholder group. Future M&E summary reports will highlight issue areas by relevant stakeholder group.
Activities, Roles and Responsibilities The W+ Impact Report: Performance indicators are collected by the W+ Standard Coordinator and reported to the WOCAN Executive Director and the W+ Advisory Council on a quarterly basis1 using the “Impact Report” spreadsheet. An example is provided in Appendix 1. The W+ Standard Coordinator also surveys all W+ project implementers and accredited W+ verifiers in the last quarter of each calendar year. Summaries of the prior 1
This reporting will begin with the May, 2017 quarterly meeting of the Advisory Council. The Impact Report information is maintained in a shared document system and so is always accessible to the WOCAN Executive Director. March 1, 2017
4
year’s report will be posted on the W+ website in the first quarter of the year, beginning in 2018. Other WOCAN staff or consultants contribute to the quarterly impact report as it applies to their area of responsibility. The WOCAN Executive Director reviews the Impacts Report with the W+ Standard Coordinator and other staff or consultants as needed and determines if changes to the indicators, or resource changes to the M&E program are necessary. The W+ Advisory Council receives the Impact Report during regular quarterly calls. The Advisory Council may provide recommendations for additions or modifications to the M&E indicators, or overall system.
Resources Dedicated to the W+ Monitoring and Evaluation System The W+ Standard is young in its implementation. At the start, the W+ Standard Coordinator has primary responsibility for collecting and reporting on selected indicators to the W+ Advisory Council and the WOCAN Executive Director. The time required for this role at this time, is approximately 10% of the W+ Standard Coordinator’s time. The time demands of the M&E system will expand over time, as the up-take of the Standard expands. Resources will be allocated to the expanding program as necessary. The W+ Standard Coordinator’s required competencies2 ensure the M&E system has continuity and accuracy. Competencies for the M&E system administration include: strategic planning, data collection for reporting and analysis, data analysis and presentation, client and stakeholder communications, survey design, relational database design. At the start, the W+ M&E system will record indicator data in spreadsheets. These are also useful for translating the data to graphs, tables and charts. In the future, a more complex relational database may be needed to manage the information from larger numbers of more diverse projects. By keeping the indicator data in an easily exported format, WOCAN ensures that upgrading M&E data infrastructure can be done efficiently when it becomes necessary.
2
The W+ Standard Coordinator’s Term of Reference (TOR) lists required competencies March 1, 2017
5
Appendix 1 – W+ Standard Impacts Report
March 1, 2017
6