GPSA Workshop Report 24-26 June 2013, Marseille, France
Report prepared for the GPSA Secretariat by Russell Wildeman, Independent Consultant
July 2013
1
1. Introduction This workshop presented the first opportunity for the World Bank to convene and meet with the 12 finalists or project teams, which will become part of the Global Partnership for Social Accountability (GPSA). A total of 17 individuals from 12 different organizations1 attended this event in addition to a group of finalist partners, global partners and resource persons who were tasked to assist the GPSA team in executing parts of the program with a total of 29 participants. This group of finalists emerged from an initial pool of 216 applications. The workshop was conducted at CMI in Marseille (France) over three days starting on Monday 24 June 2013 and concluding on Wednesday 26 June 2013. The workshop aimed:
To explain the overall vision, purpose and expected outcomes of the GPSA;
To discuss the political economy of GPSA Projects’ implementation, and share the grantees’ strategic approaches;
To discuss key challenges for GPSA Projects’ implementation, including programmatic and social accountability approaches;
To align GPSA’s Program-level Results Framework with GPSA’s Project-level Results Frameworks;
To identify priority knowledge and learning activities that may be facilitated through the GPSA’s Knowledge Platform; and
To explain fiduciary requirements and procedures that GPSA Projects must follow and comply with in accordance to World Bank’s operational procedures.
Structured facilitation was done for all the main sessions and the resource persons actively participated, especially in the various group work activities that project teams had to perform. This report has been organized in four sections. Section 2 summarizes key points on thematic issues that were the main focus for this gathering; namely a focus on the political economy and how this is likely to affect project implementation, an in-depth discussion and analysis of the M&E aspects of the program, the introduction of the Knowledge Portal and the GPSA learning component, as well as a session focusing on the fiduciary requirements and other operational matters that define the working relationship between the Bank and grantees (this report will not delve into the content of this session). Section 3 provides a brief assessment of the extent to which this workshop met its objectives based on the key goals for the workshop and a summary of the participant’s feedback. Finally Section 4 summarizes some of the next steps that were agreed to at the workshop,
1
See the full list of attendees and organisations in the Appendix of this report.
2
while the Appendix provides a full listing of organizations and individuals who had attended the workshop.
2. Project implementation challenges This theme was discussed and reflected upon under two session’s formats: a plenary session on the political economy challenges that are likely to affect the implementation of projects, which was followed by more focused working group discussions on implementation challenges likely to be encountered around key cross-cutting areas (ICT, budget analysis, multi-stakeholder coalition building, engagement with state institutions etc.). The plenary session was led by Roby Senderowitsch and Maria Poli from the GPSA Secretariat, and Russell Wildeman (the lead facilitator). This session also convened a panel discussion with resource persons on the broader political economy challenges facing social accountability projects today. Roby Senderowitsch and Maria Poli shared some of their experiences working on social accountability projects and used the opportunity to challenge project teams on the impact of such interventions, and whether CSOs collect the right kind of evidence to demonstrate their claimed successes. Some of the key issues that emerged from the discussion included the following concerns and key points:
The GPSA team presented a simple analytical tool that could be used to identify stakeholders that have influence and interest. The idea behind the use of this tool is to maximize the involvement of stakeholders that can help bring about the change expected from social accountability projects. The tool demonstrates how projects could map key stakeholders that may positively or negatively influence project implementation; this was widely appreciated by the participants. Some participants noted that real life is often more complex and that the tool would only be the start of more indepth analyses. The GPSA team stressed out that politics and political power are at the heart of the work to be undertaken by project teams, further noting that teams should not make the mistake of thinking that project implementation is a purely technical exercise. Power and power relations must be understood if projects are to effectively traverse the social, political and economic (country) contexts that define each participating project. The GPSA team also noted that the GPSA does not follow a rights-based approach to social accountability instead its emphasis is on solving problems. This approach underlines the importance of relationships and collaboration with people in government who have the ability to change service delivery realities, reinforcing the point that effective project implementation requires both technical and political resources in order to ensure that the goals of the GPSA projects are met. During the facilitated panel discussion that zoomed-in on the notion of a theory of change, most panelists noted that CSOs tend to get this right 3
through an iterative process of trial and error. However, not enough CSOs do it explicitly. Courtney Tolmie from Development for Results Institute (a GPSA Global Parnter) indicated that it is better for CSOs to think about a theory of change in terms of a decision tree -in other words, CSOs have to carefully consider how certain decisions/interventions affect the next set of activities and influence the likelihood of meeting set targets in a project’s lifespan. Another prominent idea that emerged from this discussion was the notion that CSOs should not subscribe to the view that the mere presence of ICT tools will solve the problems they would want to address through their technological and communication platforms and tools. On the issue of whether the institutionalization of social accountability in some governments could present a challenge or an opportunity, panelists argued that governments use the same tools as CSOs and have become adept at understanding how to use social accountability tools in governance and service delivery debates. Linked to this view was the suggestion that CSOs should take time to learn about their environments so that they are able to decode political messages correctly. Projects were encouraged to invest considerable time in better understanding their environments, mapping the key stakeholders and develop realistic assumptions that would help implementation of their respective projects. One of the key challenges as far as understanding the overall effectiveness of social accountability interventions is that no systematic study exists on the impact of social accountability interventions and yet there is important disperse information on what works and does not. In the interactions between the audience and the panelists, the following questions were raised: o The importance of distinguishing between organizations that test a theory of change and those that are still developing a theory of change. Often, CSOs are not clear as to where they fit into this representation of the theory of change which may negatively influence project implementation. o Another issue that came to the fore was the importance of the leader in ensuring consistent implementation of the project. There were differing views on this matter as some felt that this was very much a question that could only be resolved at the organizational level.
These broad ranging discussions were followed by group-work and plenary presentations that focused on cross-cutting areas and the kinds of implementation challenges that GPSA projects are likely to encounter. Below are some of the key findings pertaining to each of the areas are reported. A. Multi-stakeholder engagement and coalition building Discussions centered on four elements, namely the identification of the core coalition partners, what role these coalitions play, assumptions the lead 4
partner makes about the coalition and some of the difficulties facing the coalition building process. Coalition partners ranged from Consumer Unions, local government authorities, village health committees, the local World Bank representatives, national ministries etc. Many of these partners play a capacity building role. In terms of the key assumptions made about the coalition partners, project members mentioned the following factors: o Willingness of core partners to see the approach taken by the GPSA project as a viable way to improve local outcomes; o That the space for dialogue exists to enable proper communication between project leaders and the core coalition; o That the data collected by different coalition partners is unbiased; o That all the core partners are committed to change and act in an impartial manner. In terms of the difficulties that coalition building processes may encounter in GPSA projects, it became clear the need for the GPSA projects to manage relationships so they don’t lead to competition among coalition members; It is important to clarify how various partners play a distinct and defined role in bringing about change according to a theory of change (ToC). This could reduce potential intra-coalition conflicts. In the plenary discussion where groups were required to reflect on challenges, the following transpired: o A mapping of key stakeholders is needed; o The buying-in of key stakeholders is vital for the success of the coalition building process; o Partnership roles must be clarified at the start of implementation; o There is a need to build relationships with political leaders; o Managing expectations at the start of projects is vital to avoid competition or partners’ perception of project gains; and o Projects were encouraged to rely on the networks and coalitions to scale up the project’s operational model.
B. Constructive engagement with the government This discussion required participants to identify entry points in their work with the government; and how these translated into action consistent with the change their projects envisaged; as well as key actions for engaging the government in an effective manner. Potential entry points to meet with the government ranged from sectorial ministries, steering committees where key domestic and international stakeholders were represented, the finance ministry, local government officials, local World Bank staff who have direct contact with the government and Members of Parliament etc. In the plenary session, participants highlighted some of the key implementation challenges they have encountered or are likely to encounter
5
based on their local contexts, as well as possible options in order to address them. These included: o Not having the same counterparts to interact with and talk to has been identified as a serious constraint in CSOs engagement with the government; o CSOs must do their homework and identify key stakeholders (agencies and individuals) to target within government; in doing so, GPSA projects must identify users of information from the government. In other words, who are the officials who would benefit the most from the work done by CSOs? - which speaks directly to the value added by CSOs; o Effectively using the World Bank Country Office to strengthen relationships between the government and CSOs; o The same World Bank Office could become an obstacle in the quest to promote and nurture CSO and government relationships and hence it is crucial that CSOs develop relationships with the designated Task Team leader in-country; o Internal dissent in government is another factor to consider because alignment to one group in government could affect the CSO access to information and contact to key officials; o CSOs are required to produce information that is useful for and to the government, thus establishing possible long-term relationships; C. The use of ICT for social accountability This discussion focused on the types of ICTs that each of the projects was planning to use, the kinds of constraints that limit the use of these technologies and some of the key challenges to be considered when thinking about the uses of ICT in GPSA projects. Prominent among the tools were mobile phones (due to its widespread penetration), the use of online polling, interactive maps, the use of iPads, and the use of websites for feedback on project implementation; Some of the constraints of using ICT included the following: o Gender gap. Fewer women than men are using mobile phones in certain developing country contexts; o Connectivity was flagged as a major issue because it is easier to connect in some places; o Illiteracy is a challenge and could limit the use of ICT in project implementation; The plenary session highlighted some implementation issues regarding the use of ICTs: o Context will determine the technology to be used; o Projects must learn to tailor their messages for specific audiences; o Many governments do not have the capacity to use ICTs, thereby limiting opportunities for ICT use; o Projects were encouraged to look for open source solutions;
6
o Projects must not replicate what already exists because it would be inefficient; and o Projects need to engage ICT experts inside and outside of government to increase their chances of finding relevant data and understanding the various data protocols. D. Budget analysis and monitoring It was clear from the various group discussions that a range of budget related tools will be employed by GPSA projects. These include independent budget analysis (at national and local level), sector budget analysis (education, water, sanitation), participatory budgeting, budget tracking, social audits with a focus on public budgets, bottom up planning and budgeting, municipal budget analysis, using the ward public meeting to talk about budgets etc. Some of the key assumptions that drive projects’ considerations about budget engagement include: o Communities and individuals are interested in engaging with the budget at the appropriate levels of government; o Governments are committed to open budgets; o An active mass media disseminates information; o There are credible budget processes that produce relevant information on a timely basis; and o The willingness of citizens to participate as well as the willingness of the government to respond. Participants also shared some thoughts on how their budget tools fit into the government decision-making processes and some of the issues raised were: o There was agreement that the production of budget analyses will always have a receptive audience and space; o Most projects indicated that two routes are open to them: namely the direct route (at ward meetings, through interactions with government officials on the outcome of a report card process etc.) and an indirect route (where they make submissions to the representatives of the citizens, i.e., national Parliaments or through Planning Committees); o Some projects indicated that a further direct route is available, namely by working directly with the relevant ministries (education or health national ministries regarding budget execution Reports). In the plenary session, the following challenges were noted: o Lack of budget data; o Non-cooperative officials; o Making sure that the level of analysis is appropriate for the audience and not to always assume that citizens want to hear about budget analysis; and o Use tools that are appropriate for the tasks at hand.
7
3. M&E The M&E session consisted of three parts. First, Professor Lily Tsai from MIT made a presentation on the fundamentals of the M&E process. Second, a fishbowl activity was conducted where Prof. Tsai and selected participants reflected on the M&E framework of Care Malawi’s project. The fishbowl exercise was intended to be instructive and so the third part followed this approach as four working groups debated the M&E frameworks of particular GPSA projects. Participants were able to understand and practice how to examine and improve a project’s results framework. Prof. Tsai noted that monitoring is actually a harder task than evaluation because of the different types of demands on organizations (methodological, understanding the measurement burden, and charting a defensible theory of change). Broadly speaking her comments could be divided into two main sets of issues, namely on methodological issues and on design issues. On the issue of methodology, Prof. Tsai indicated that: o A good results framework should not lead one to confirming what one already expects, but should be focused on finding results that disprove the expectations. This is the opposite of confirmation bias and is likely to lead to stronger and more rigorous evidence; o While expensive randomized control groups is out of the question for most projects, she urged projects to adopt at least some form of the control group methodology so as to make their evidence more defensible; o Projects should commence with properly designed baseline studies; and o Projects need to be aware of the fact that social accountability concepts are hard to measure; previous studies and work have not yet established clear benchmarks or templates, and there would be something experimental about the implementation of social accountability results frameworks. On the issue of project design, Prof. Tsai indicated that: o It is better that projects fail as early as possible so as to be afforded the opportunity to learn from errors; o Simpler is better –rather to select fewer indicators that are good and in doing so you reduce the measurement burden and increase the quality evidence at your disposal; The second part of the M&E session consisted of a fishbowl conversation about the relative strengths and weaknesses of the Care Malawi project results framework. Prof. Tsai modeled the process for reflecting whether project actions taken will lead to the desired outcomes or changes in behavior as per the project outcomes.
8
Some of the key issues that emerged from the fishbowl conversation about the results framework of the Care Malawi project included: Generally, project indicators have been well defined, proposed data collection is frequent and realistic, the framework demonstrates a good mix of tools and indicators, and the role of partners and the lead agency is clearly defined in the monitoring process; Questions were raised, however, about how this project would measure change, the arbitrariness of some of the targets in the absence of a baseline study, the large number of direct beneficiaries (cost and measurement issues), the level at which this project is going to be implemented etc. In the plenary session, after the groups’ work on 4 particular GPSA projects’ results frameworks, very similar conclusions emerged about their strengths and weaknesses. Some of the strengths of the results frameworks included: o There appears to be a good mix of quantitative and qualitative indicators; o Some of the frameworks propose a good mix of data collection methods suggesting that the designers were acutely aware of the challenges and costs in collecting data; and o The presence of baseline studies has been welcomed as it provides more rigor to the proposed projects’ interventions. Among the weaknesses highlighted were: o Frameworks where the measurement burden is heavy (too many indicators) there is need to reduce some of the indicators and keep the simplicity principle that Prof. Tsai noted; o Indicators that are hard to measure; o Efforts made by CSOs in the GPSA projects that might have already been done by other CSOs. For example, many CSOs would do citizen report cards and use the evidence in the results framework and there was a plea that CSOs scan work that was done already and use that in their own results framework. In her closing remarks, Prof. Tsai emphasized the importance of simplicity and asked projects not to impose measurement burdens that are impossible to carry out. She also reiterated that projects need to be careful in the way they put together indicators.
4. Knowledge and learning as vital to the success of the GPSA The knowledge and learning component -including the Knowledge Portal- was done in three segments. Firstly, Marcos Mendiburu from the GPSA Secretariat and Janet Oropeza from FUNDAR Mexico made presentations on the GPSA Knowledge component and on the Knowledge Portal respectively. Marcos explained the importance of the GPSA knowledge component and the linkages among the various 9
K sub-components whilst Janet focused more specifically on the Knowledge Portal. The second segment required project- teams to develop their draft Knowledge and Learning Plans (K&L) and respond to the questions from the K&L questionnaire. The third segment required projects to focus on their expected use of and contributions to the Knowledge Portal (KP). In his presentation, Marcos Mendiburu made the following salient points: Knowledge and learning is an integral part of the GPSA and should be considered at the very start of project implementation; In order to promote the use of knowledge and learning, he indicated that GPSA projects should pay attention to existing partnerships and how different partners contribute to knowledge and learning; projects were urged to build upon what already exists and to not duplicate, and projects were asked to seek collaboration with academics, research institutions, thematic networks; Towards the end of his presentation, Marcos Mendiburu stressed out the importance of measurable knowledge indicators and provided a list of possible indicators that would measure “intermediate capacity building outcomes” and indicators that measure the effectiveness of the Knowledge Platform. Projects were actively encouraged to consider some of these indicators and other ones that make sense within their various projects’ contexts. Janet Oropeza focused on the Knowledge Portal (KP) and emphasized the following issues: The ultimate goal of the KP was to enhance CSOs access to knowledge, learning and networking to strengthen the effectiveness and impact of social accountability initiatives; She noted that the KP will include four functional modules: a knowledge repository, learning, networking and knowledge exchange. The first of the four modules, the repository, will provide practitioners with access to relevant knowledge on social accountability and could also be used to store and disseminate relevant GPSA grantees’ knowledge materials; The KP implementation will be undertaken in an incremental manner; and Towards the end of her presentation, she shared possible measurable indicators that would indicate the global success of the KP and that could also be used and adapted for use by the GPSA project teams. In the plenary session that followed the presentations, project teams were required to present their draft annual K&L plans, the idea being that team members would go back to their respective organizations, consult with partners and project staff, and then submit a revised final version to the GPSA secretariat. The plenary sessions focused on the feedback by the project teams based upon the K&L questionnaire and the drafting of the K&L annual plans. Among others, the following points were highlighted:
10
The success of the KP is dependent on everyone’s participation. Incentives need to be created for stakeholders to engage on the KP; The GPSA secretariat noted that although the 12 finalists attending the meeting are for now the primary users of the KP, because access is open and free, there will likely be many other users of the KP; The language could be a constraint for projects that work in non-English speaking countries; Some participants noted that access to the KP might be limited in some countries as a result of connectivity problems; Participants suggested the production of a newsletter, which would serve as a tool to enhance communication within the GPSA ‘community’ and keep colleagues abreast of developments in the various topics of social accountability; Participants felt that one learns as much from the projects’ successes as from the failures and the KP should not only promote so-called ‘success’ stories; Security concerns were also raised and some participants felt that writing about sensitive issues, their impact on policies and service delivery can be quite risky in some political and country contexts. Everyone agreed that there is a need to shelter project staff from such dangers by for example making use of fictional names and fictional countries, but this is clearly an important issue that should be thought more thoroughly; Participants were encouraged to take note of existing initiatives such as the Open Government Partnership (OGP), as well as metrics such as the Open Budget Index (OBI), and Global Integrity reports, etc; Maria Poli urged projects to think outside the box in terms of the resources and people needed in order to make their projects a success and also noted that the processes connected to capacity building need to also be shared and not only the capacity building materials; Marcos Mendiburu reminded participants that the essence of the KP is to facilitate partners’ knowledge sharing and in that way, the KP should be thought about more as a networking space than a knowledge hub–by connecting people and organizations. In terms of projects’ K&L draft activities, the following issues came up in the group discussions: o Some partner organizations were quite explicit about their needs to team up with international NGOs that have done this kind of work and could help with the production of relevant K&L activities; o Some organizations acknowledged limited experience in K&L and the need to be supported more and then the question remains of how best to do this kind of capacity support; o Some project teams have extensive experience with K&L and so a differentiated base exists among the GPSA project teams as far as their capacity to develop and implement K&L plans that complement and support their broader project goals.
11
5. Assessing whether the workshop met its key objectives To evaluate if the workshop met its objectives the GPSA team took into account the issues raised during the discussions and the input from the participants’ feedback. This section offers an overview on how the activities were carried out and their key results, in addition to a summary that highlights the main points from the participants’ feedback. At the start of the workshop, Roby Senderowitsch and Maria Poli stressed the problem solving approach taken by the GPSA and listed three key issues for discussion at the workshop, namely: 1. What type of feedback will your Project generate? 2. Who in the government is expected to use it? 3. What incentives do they have for acting upon this information? The workshop enabled this particular issue to be dissected in three ways. Firstly, there was an introductory (and frank) discussion by Roby and Maria about the political and power dynamics that projects are likely to face. The key point that emerged from this was that project implementation should not be seen as a technical activity only and that due recognition must be taken of the socioeconomic and political environments in which projects operate. Secondly, the panel discussion succeeded in clarifying some of the key implementation issues, including the need of a theory of change. Thirdly, participants were given extensive time and opportunities to think through broader cross-cutting implementation challenges (multi-stakeholder coalition building, engagement with state institutions, ICT, budget monitoring etc.). Participants were required to complete formal evaluation forms at the end of the workshop. Below, we provide a summary of the key findings that emerged from this feedback. From the 22 evaluation forms that were completed (see Annex 7.7), participants identified the knowledge and learning, M&E and result framework discussions as the most valuable for them while also highlighting the unique opportunity that the workshop created for the participants to share and network with peers. For future workshops or activities participants recommended longer time for discussion as well as sessions for a deeper understanding of other grantees’ projects. Finally participants suggested that the GPSA should support groups to create a space for further information sharing and exchange and also the need to conduct a mid-term evaluation of the projects implemented.
6. A summary of key next steps agreed upon at the workshop At the end of the workshop, the following key steps had been agreed to between project partners and the GPSA team, ought to be submitted by July 22. Revised results frameworks; Knowledge and Learning Annual Plans; Project implementation plans; 12
ďƒź Procurement plans
7. Appendix 7.1 List of Attendees, organizational affiliation and country. 7.2 Final Agenda 7.3 ICT Handout 7.4 M&E Handout 7.5 Knowledge and Learning Plan and Questionnaire 7.6 Knowledge Portal Questionnaire 7.7 Participants’ Feedback from GPSA Workshop Survey
13
7.1 List of Attendees, organizational affiliation and country LAST NAME
1 2 3 4 5 6 7 8 9 10 11 12 13 14
15 16 17
18 19 20 21
22 23 24 25 26 27 28 29
FIRST Name
Origin country
Organization
Workshop
Anowarul Mohammad Arif
Bangladesh
Finalist
Anam
Shaheen
Bangladesh
Canete
Rosa
Dominican Republic
Aritonang
Asteria
Indonesia
Sinaga
Masrawati
Indonesia
Gradwal
Sabina
Kumpolota
Anderson
Malawi
Kubalasa
Dalitso
Malawi
Lupusor Vlad
Adrian Victoria
Moldova Moldova
Vidal
Domingos
Mozambique
Cigarro
Agostinho
Mozambique
Lalรก
Aly
Mozambique
Sumangil
Pura
Kharrat Puddifoot
Selim Jonathan
Tunisia United Kingdom
Aidaraliev
Rakhatbek
Kyrgyz Republic
Kondowe Monidwa
Benedicto Kalako
Malawi Malawi
De los Angeles
Dante
Philippines
Care Bangladesh Manusher Jonno Foundation Manusher Jonno Foundation Fundacion Intermon Oxfam World Vision Indonesia World Vision Indonesia Development Policy Institute Care Malawi Malawi Economic Justice Network Expert Grup Expert Grup Concern Universal Mozambique Concern Universal Mozambique Concern Universal Mozambique Concerned Citizens of Abra for Good Government Al Bawsala Oxfam Tajikistan Association of Village Health Committees of Kyrgyz Republic CSEC CSEC Partnership for Transparency Fund
Oropeza Eng
Janet
Mexico
Fundar
Kode
David
South Africa
Tolmie
Courtney
Bhargava
Vinay
CIVICUS Results 4 Development Partnership for Transparency Fund
Wildeman Stickel Tsai Wasikhongo
Russell Cathryn Lily Joseph
Haq Khan
Bangladesh
Kyrgyz Republic
Philippines
Uganda USA South Africa USA USA Kenya
SMS Frontline MIT Elimu Yetu Coalition
Finalist Finalist Finalist Finalist Finalist Finalist Finalist Finalist Finalist Finalist Finalist Finalist Finalist Finalist Finalist Finalist Finalist Partner Finalist Partner Finalist Partner Finalist Partner Resource, Global Partner and Finalist KP Global Partner Global Partner Global Partner and Finalist Partner Resource Person Resource Person Resource Person CSF Grantee
14
30 31 32 33 34 35 36
Thomas Poli Trommer Senderowitsch Mendiburu Fokkelman Grandvoinnet
Seema Maria Josef (Stig) Roby Marcos Emilie Helene Marie
USA USA USA USA USA USA USA
World Bank World Bank World Bank World Bank World Bank World Bank World Bank
GPSA (WB) GPSA (WB) GPSA (WB) GPSA (WB) GPSA (WB) GPSA (WB) SDV (WB)
15
7.2 Final Agenda Monday, June 24, 2013 8:00am – 2:00pm
2:30pm – 4:00pm
4:00pm – 6:00pm
Registration Introductions and Gallery Tour Welcoming Remarks: Presentation of GPSA and Workshop’s Objectives (Roby Senderowitsch, Program Manager) Presentation of Participants and their Expectations Plenary Session: How social accountability contributes to governance reforms in order to achieve development results? Understanding the political economy of countries and sectors for projects implementation. Roby Senderowitsch & Maria Poli (With contributions from Courtney Tolmie, Cathryn Stickel, Russell Wildeman, Helene Grandvoinnet, Joseph Wasikhongo) Reception
6:00pm – 7:00pm
Tuesday, June 25, 2013 9:00am – 9:15am
Introduction to Roundtable Discussions: Addressing key challenges in project implementation (Russell Wildeman) Concurrent Roundtable Discussions (Participants will be asked to join 4 working groups – 3 projects/each group ) | Each resource person leads discussion for 45 minutes before moving to next table| Discussions on key approaches for effective implementation of projects Multi-stakeholder Engagement (Courtney Tolmie) Constructive Engagement with the Government (Helene Grandvoinnet) Use ICT for Social Accountability (Cathryn Stickel – Frontline SMS) Budget Analysis & Monitoring (Russell Wildeman)
9:15am – 10:00am
Concurrent Discussion- Rotation 1
10:00am – 10:45am
Concurrent Discussion- Rotation 2
10:45am – 11:00am
Coffee Break
11:00am – 11:45am
Concurrent Discussion- Rotation 3
11:45am – 12:30pm
Concurrent Discussion- Rotation 4
12:30pm - 1:00pm
Plenary Session: Recap and Lessons Learned Roby Senderowitsch and Maria Poli
1:00pm –
2:00pm
Lunch Break
16
2:00pm – 3:00pm
Plenary Session: Aligning M&E Plans with GPSA Results Framework Lily Tsai
3:00pm – 4:30pm
Reviewing Our M&E Plans and Refining Outcomes (Lily Tsai, Courtney Tolmie)
4:30pm – 4:45pm
Coffee Break
4:45pm – 6:00pm
Plenary Session: Summary of Working Groups’ Discussions and Ways to Improve M&E Plans
8:00pm – 10:00pm
Dinner
Wednesday, June 26, 2013 9:00am – 9:30am
Plenary Session: GPSA’s Knowledge Platform, Objectives Led by Marcos Mendiburu and GPSA KP grantee
Working Groups of 3 projects + others | Each with 1 resource person (TBD) | Discussions based on key K&L challenges and strategies to address them 9:30am- 11:00am
11:00am – 11:15am
11:15am – 12:00pm
12:00pm-12:30pm
12:30pm – 1:30pm 1:30pm – 2:00pm 2:00pm – 2:30pm 2:30pm – 3:00pm 3:00pm – 3:15pm 3:15pm – 3:45pm 3:45pm – 4:15pm 4:15pm – 5:00pm 5:00pm – 6:00pm 6:00pm – 6:15pm 8:00pm – 10:00pm
Sharing and reviewing K&L Plans; discussing strategies and goals, and identifying ways to develop the plans and implement them effectively. Coffee Break Working Groups (cont.): making the KP work for Grantees and taking advantage of its potential; collaborating and working together; contributions to KP; how to maximize the projects’ impact trough dissemination, networking and knowledge sharing. Plenary Session: Summary of Working Groups’ discussions. Ways to improve K&L Plans and Agreeing on Key Priorities of the KP and Grantees’ Contributions. Marcos Mendiburu Lunch Break Plenary Session: Working with the World Bank as Grantees In GLDN section of Villa Introduction on Grants Cycle (Stig) Legal WB Legal Specialist Coffee Break FM Disbursements and Fiduciary Requirements WB FM Specialist Procurement WB Procurement Specialist Reporting WB Reporting Specialist Plenary Session: Next steps Closing Remarks: Roby Sesnderowitsch Dinner
17
7.3 ICT Handout Global Partnership for Social Accountability Focus on ICT When designing a project, ICT integration is best done after careful consideration and in an agile manner. It is tempting to assume that, because information and communications technologies have dramatically altered life in the developed world, these effects will be duplicative everywhere. However, ill-considered ICT use can be useless at best and more likely an expensive handicap. Design Considerations *What if any ICT tools do your target participants already possess? Products should be designed around the digital landscape AS IT EXISTS. Device familiarity will dramatically improve potential outcomes, and device availability will dramatically improve potential sustainability. Introducing new and/or unfamiliar technologies at the beginning of a project is not recommended. *What costs will your target participants necessarily incur when utilizing your ICT tools? ICT cost money. The tools themselves and their chargers, cords, and repairs; the costs of sending and receiving SMS, connecting to the Internet, software subscription fees, and making voice calls. Will your participants run into prohibitively high expenses? Past the extent of your involvement, will local organizations be able to afford these expenses, in the interest of sustainability? *Does your ICT require access to the Internet at any point? Assume the absence of Internet connection. The best estimates of developing world connectivity hover around 33%, and even that access if often infrequent and costly. *What are the literacy levels amongst your target participants? Illiteracy remains an intractable problem in the developing world. If your participants are unable to read, it may pose problems for the introduction of ICT. However we have found that many individuals are incentivized to develop what we call ‘functional literacy’ in which they learn to read the words necessary to participate. Pragmatism is well advised when considering the likelihood of this occurring. *What are the cultural mores around ICT ownership within the communities of your target participants? In many communities around the world, phone ownership is an emotionally charged issue. Unfortunately, it’s a largely gendered issue, with women less likely to own a phone than a man, often at the behest of her husband. Often women are 18
sensitive to possessing a phone because other women will suggest it is for purposes of romantic engagement outside her marriage. It is recommended that planners consider cultural conditions relative to ICT when designing a project. Additional Resources The FrontlineSMS User Guide on Data Integrity is intended for FrontlineSMS users designing, implementing, and monitoring their programs with data integrity concerns in mind. An increasing number of users transmit sensitive information using SMS, which makes questions of data quality and integrity more important than ever. This guide addresses the resulting data integrity considerations of confidentiality, authenticity, availability, and usability of information transmitted using FrontlineSMS. Drawing directly on the experiences of those who have used FrontlineSMS in vulnerable contexts, this guide provides a data integrity framework that can be used to understand the level of risk inherent in SMS-based activity. Our approach is based on identifying ways to mitigate the risks identified from the perspective of the user, in combination with a focus on the quality and usability of information exchanged through a FrontlineSMS hub. FrontlineSMS does not define the exact details of how users should address issues of data integrity when deploying our software, but instead provides suggested guidelines on how users can reduce and mitigate the risks by taking a responsible and informed approach program design and management. CASE STUDY - The Oro Verde Program, a social enterprise committed to supporting mining communities in Colombia shares a case study on the impact market prices via SMS have had on ensuring miners have access to current Gold prices. This pilot has demonstrated the potential of SMS in improving the way market prices are communicated to miners; both in terms of efficiency, and the utility of the information shared. This case study details how the Oro Verde Program set their Price SMS service up, and shares the key learning from the experience. http://www.frontlinesms.com/wpCASE STUDY - This case study shows how Georgetown University’s Institute for Reproductive Health (IRH), a global organization dedicated to improving reproductive health worldwide, has used FrontlineSMS to provide a rapid prototype of a new mHealth service. This service — called CycleTel™ — empowers women by providing them with accessible reproductive health information through SMS. The ICT is just a tool; a means to an end rather than the end in itself.
19
7.4 M&E Handout
Participant Guidelines for Reviewing and Improving M&E Frameworks 1. Understanding the proposal and M&E framework
Identify all the “treatments” in the project. What actions are being taken, or what products are being created by the organization or initiative?
Diagram the theories of change or causal processes. Create “arrow diagrams” for each “treatment.” These arrow diagrams should show how the intervention or each component of the project leads to the final outcome(s). For example: Projects distributes report cards with information about the percentage of discretionary funds an MP has disbursed to public projects (100% is ideal) citizens learn new information about the performance of their MP increases citizen dissatisfaction with MP performance citizens less likely to vote for MP in next election. Are there any holes or gaps in the diagram that the proposal and M&E framework don’t provide enough information on?
2. Evaluating the M&E framework: Strengths, challenges, and suggestions
Assess the scope of the M&E effort required
How many arrow diagrams are needed to represent the project? How many steps are in the arrow diagrams? On which diagrams and which steps should M&E effort be concentrated?
Identify which steps in the arrow diagrams are being measured as outputs and outcomes in the M&E framework.
Do the last steps correspond to the indicators measuring the final outcomes? How many of the steps in the middle are being measured by indicators for program outputs and intermediate outcomes? Are enough of the steps in the middle being measured so that we can figure out whether the treatment is working the way it is supposed to in a timely fashion?
Does the M&E framework collect data that contributes to the measurement of key indicators in the GPSA’s Results Framework?
How do the indicators in the M&E framework help the GPSA collect data and put together country case studies on the following indicators? (1) Increase in beneficiary participation and feedback on service delivery; (2) Improvement in service delivery due to beneficary feedback and participation; (3) Increase in the capability of CSOs to facilitate beneficiary participation and feedback.
20
Evaluate the indicators and the plans for data collection. Do the indicators measure what they’re supposed to? Are the data likely to be accurate and believable? Are the plans for data collection feasible?
Do the proposed indicators of outputs and outcomes actually measure what they say they are supposed to measure? Do they really correspond to the step in the causal arrow diagram they are supposed to be measuring? Are the proposed indicators concrete and observable? Are the data being collected in ways that are likely to be “true”? Is there any reason to think the data collected will not be believable?
Are plans for monitoring and collecting data on program outputs and intermediate outcomes as developed and rigorous as plans for evaluation?
Will we know whether the program is working in the way that we expect it to, based on the arrow diagram – but in time to make improvement during the middle of implementation?
21
7.5 Knowledge and Learning Questionnaire and Matrix
K&L Plans 1. Please describe the Knowledge and Learning (K&L) needs that your staff and partners have to effectively implement your project.
2. What are the main K&L challenges you perceive for effectively developing your project? What about the main challenges for K&L within your organization?
3. If you know, please list the possible providers for your K&L needs
22
4. What is your organization’s experience working with individual researchers or academic institutions?
5. How do you plan to involve individual researchers and/or academic institutions into your project? Please be specific.
6. Is your organization part of any thematic networks or other networks? Of which one(s)? How do you expect these networks to contribute to the K&L Plan? Please be specific.
23
K&L activities
Outputs/ products
Whom will develop it?
Where is the demand coming from?
Partners
Other comments (benchmarks)
* Add as many rows as needed to include all your K&L activities.
24
7.6 Knowledge Portal Questionnaire Knowledge Platform (KP) 1. Are you aware or have you used other knowledge management platforms (? If so, which ones? Please elaborate about it. 2. How many persons in your staff will be in charge or involved in interacting with the KP? Please provide details and in which capacity (full time/part time; participant in webinars, discussion forums, etc.; communications; uploading documents? 3. Please provide details for the contact person/s responsible for handling KP-related issues and communications in your project. 4. How could you connect the KP with other offline activities developed as part of your project? Please be specific 5. What can you do to create incentives for other organizations in your country to participate in the KP? 6. Which issues/topics/skills you think would be useful to address through webinars? 7. Which knowledge products can your project provide or develop for the knowledge repository?
25
8. Which knowledge products from your partner organizations can be provided for the KP? 9. How can your organization help disseminate the KP among interested actors in your country? 10. Are there any actors/resource people/stakeholders you think should be invited to participate in or contribute to the KP in your country or elsewhere? 11. What are the main challenges to using and contributing to the KP? 12. Which indicators included in your K&L plan are used to capture the interaction with the KP? 13. How could you include additional targets related to the KP in your M&E plans? Which would those be? 14. What other technical or substantive issues should be addressed by the KP (language, bandwidth, computer skills, management skills, etc.)? 15. What opportunities for networking do you think that could be addressed through the KP?
26
7.7 Participants’ Feedback from GPSA Workshop Survey
Global Partnership for Social Accountability South-South Workshop for GPSA finalist, Marseille July 24-26, 2013
Participants’ Feedback Description: On June 24-26, 2013, the GPSA organized the first South-South workshop for the 12 finalist civil society organizations that the GPSA Steering Committee has pre-selected for the first grants. The workshop marked the kick-off of the GPSA Knowledge Platform activities. The goal of the workshop: to prepare the finalists for project implementation, once their GPSA grant funding would be approved. To evaluate the success of the workshop, participants completed an evaluation form with three questions. This feedback reports provides the answers from a total of 22 evaluations completed by the participants.
1. What have you found to be most valuable about the workshop? 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17.
Discussion with resource persons. Knowledge Portal. Meeting people, learning, sharing. Knowledge Portal, M&E facilitation was great. Almost everything. Reflecting on our M&E frameworks and K&L plans. The financial management bit was really great. Networking with other countries; learning starts. Sharing with peers. Passing by all the important steps we have to take care off. Interaction with fellow practitioners from all over the world and listen to their experiences. K&L Prospect for further networking. Learn more about K&L and results framework. Networking and knowing more about other projects. Reflections about results framework and the knowledge and learning questionnaire. Experiences from other countries, strategic advice from GPSA team. 27
18. 19. 20. 21. 22. 23. 24. 25. 26. 27. 28. 29. 30. 31. 32. 33. 34. 35. 36. 37. 38. 39. 40. 41. 42.
43.
Gallery walk presentation. Result framework discussion. Knowledge and learning discussion. Working group on four topics. Networking. M&E framework. K&L planning. Practical advices regarding project implementation. Networking with peers. Friendly environment. Commitment from the GPSA team to contribute in social transformation process for influencing politics. Great sessions on SA and governance. Result framework discussion is useful. M&E and Results framework discussion Knowledge and Learning Rigor needed to do stakeholders analysis (Theory of Change). Sharing and learning experience. Facilitation process. Support from resource people. Explanation of the GPSA instrument objectives Knowledge and learning, M&E, ICT. Networking, sharing, learning, getting great feedbacks. Space to reflect on our proposals from other peoples’ components. Space to interact with other CSOs doing similar work, learning what works. The degree of interaction and participation between participants. All projects were quite interesting, but have similarities though they are implemented in different contexts. Overall it was well organized. The sessions on results framework.
2. Do you have any specific feedback about aspects that need to be improved or changed? 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12.
More time for discussion based in similarity of objectives from the projects. More time for feedback. Should keep some free time, e.g. one half day. Schedule was very tight, give more days for the workshop next time. We should make all the improvements after the grant is approved. Drafting results framework and indicators. More group discussions and visual illustrations. Some sessions were too busy; more time for informal moments. Longer time for Q&A after gallery walk presentation. Seating arrangement (with table is better than without table) More time to get to know all projects. Gallery show was too quick/short. More attention and time for financial management. 28
13. More active participation of resource persons. 14. No, I think we should continue on the same line. 15. The format of the contract with World Bank needs to encourage the delivery of those things the program is interested for, i.e. results and outcomes. 16. Schedule some time to get to know the place, the event is being conducted for people to explore socially. 17. Avoid making programs too tight for example partners arriving at 11am and starting workshops 2 hours later. 18. It was well organized and the facilitation was great, so the organizers just need to maintain the same standards. 19. Participation of good experts. 20. I love everything.
3. Do you have any other suggestions or expectations looking forward that you’d like to share? 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. 19.
The notes or documentation from this process sent to participants also. No, but lets make serious effort to share expertise, knowledge challenges. Organize this type of workshop at least once a year. Organize orientation for grantees’ financial staff World Bank should initiate and support Regional level networking and learning forum. I hope we set further continuous support on project implementation issues (including at this stage). GPSA should integrate realistic reviews and partners reflections to sustain shared learning. More such interactions beefing up on shared learning and drawing lessons/ energy from what’s working to deal with what’s not in Social Accountability. Keep up the project (GPSA)! Minutes of meeting to be shared daily through email. Find a way to help the groups share information among themselves in between workshops. Create a Google group as equivalent. More discussion around knowledge management and learning component and how will it be linked with knowledge portal. Learning online social accountability from WB perspective. Continue engaging with the stakeholders present. More explanation and setting of each of the sessions so that it is clear what WB expects participants to do with the session process or results later on. The GPSA team is great! I ‘ll share this! Looking forward to more interaction and learning. More interactions between the bank and civil society at the local levels including marginalized groups. A mid-term evaluation to find out which works and which doesn’t.
29