Cost Application Annex 6: Evaluation by Vazgen

E VA LUATION Evaluation of USAID Support to the Eurasia Foundation under Core Grant III (2001-2013)

March 2013 This publication was produced for review by the United States Agency for International Development. It was prepared by Social Impact, Inc. under Task Order No. AID-OAA-TO-12-00028 under the Evaluation Services IQC.

PERFORMANCE EVALUATION OF THE EURASIA FOUNDATION

March 2013 This publication was produced for the United States Agency for International Development. It was prepared by James Fremming, Lyubov Palyvoda, and Mathias Kjaer of Social Impact, and David Cowles, Sharon Valentine, and Taryn Lovelace of USAID. The authorsâ&#x20AC;&#x2122; views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development nor the United States Government.

This document (Report No. AID-OAA-TO-12-00028) is available through the Development Experience Clearing House (http://dec.usaid.gov). Additional information can be obtained from:

Social Impact, Inc. 2300 Clarendon Boulevard, Suite 1000 Arlington, VA, 22201 Tel: (703) 465-1884 Fax: (703) 465-1888 info@socialimpact.com

ACKNOWLEDGEMENTS The evaluation team would like to thank all the Eurasia Foundation staff in both Washington, D.C. and the eight Eurasian countries visited that provided such valuable support throughout our effort. EF staff consistently took time out of their busy schedules to help find project documentation, arrange and participate in interviews, answer follow up questions, and review our draft report. Without exception, EF staff showed themselves to be helpful, honest, and gracious hosts. The team is especially grateful to Lisa Coll, Rob O’Donovan, and Horton BeebeCenter for the assistance and insight they have provided over the past several months. The team would also like to thank our Advisory Group (AG) members, Vivian Derryck, Bill Fuller, and Larry Garber, for their interest and participation in our evaluation. The insight and understanding that came out of our AG meetings provided the basis for our understanding of a more general “foundation model” to providing development assistance and helped elucidate how well EF has and has not followed that model. The AG’s comments on the draft report were also helpful for strengthening our final recommendations. We also wish to acknowledge and thank our USAID Europe and Eurasia Bureau (USAID/E&E) colleagues for their guidance throughout the evaluation. Specifically, the team wishes to recognize Jerome Gallagher for his detailed review and methodological suggestions for strengthening our report and our COR, Erin McCarthy, for her patience, responsiveness, and overall willingness to aid our effort from the very beginning of our contract. Erin’s collegial and supportive approach to overseeing this contract was a significant factor in our ability to revise and strengthen the report to what it is today. Lastly, we wish to express our gratitude to all the EF beneficiaries that spoke with us or answered our online survey as part of this effort. This report would not have been possible without their detailed and honest answers.

ACRONOYMS AG AOR BSEU BSI BSTU BSU BTEUCC BUEE CANS CAR CMI CRRC CS CSD CSO DOS DRL EC EEF EERC EF EFCA EF-DC EPF EUR-ACE FAPS FCO FGD FNE FY GMS GoB IDF IMP KII KSE LOE M&E MBA MoE MVF NEE

Advisory Group Agreement Officerâ&#x20AC;&#x2122;s Technical Representative Belarus State Economic University Business Support Institutions Belarus State Technological University Belarus State University Belarusian Trade and Economics University of Consumer Cooperation Business Union of Entrepreneurs and Employers Central Asia News Service Central Asian Republics Capacity Mapping Initiative Caucasus Regional Resource Centers Civil Society Civil Society Development Civil Society Organization US Department of State Bureau of Democracy, Human Rights, and Labor (DOS) European Commission East Europe Foundation (Ukraine, Moldova) Economics Education and Research Consortium Eurasia Foundation Eurasia Foundation of Central Asia (Kazakhstan, Kyrgyzstan, Tajikistan, Turkmenistan, Uzbekistan) Eurasia Foundation, Washington DC Eurasia Partnership Foundation (Armenia, Azerbaijan, Georgia) Office of the Coordinator of U.S. Assistance to Europe and Eurasia Foundation Administered Projects UK Foreign Commonwealth Office Focus Group Discussion New Eurasia Foundation (Russia) Fiscal Year Grant Management System Government of Belarus Institutional Development Framework Institute for Privatization and Management Key Informant Interview Kiev School of Economics Level of Effort Monitoring and Evaluation Master of Business Administration Ministry of Economy Media Viability Fund New Eurasia Establishment (Belarus)

NGO NICRA OCA OCAT PE PA PF PSDE RIPMP SATR SBLP SI SKII SO SOW UNDP USAID USG

Non-Governmental Organization Negotiated Indirect Cost Rate Agreement Organizational Capacity Assessment [Tool] (USAID) Organizational Capacity Assessment Tool (McKinsey) Private Enterprise Public Administration Partner Foundation Private Sector Development and Entrepreneurship Russia Independent Print Media Program Support for Armenia-Turkey rapprochement Small Business Lending Program Social Impact Structured Key Informant Interview Strategic Objective Scope of Work United Nations Development Program United States Agency for International Development United States Government

TABLE OF CONTENTS COMPONENT I: EVALUATION OF EF AND THE EF NETWORK Exeuctive Summary ....................................................................................................................... i Background on Eurasia Foundation ............................................................................................. i Evaluation Purpose and Methods ................................................................................................ i Key Findings and Conclusions ................................................................................................... ii Recommendations ....................................................................................................................... x Introduction ................................................................................................................................... 1 Purpose of the Evaluation ........................................................................................................... 1 Background, Context, and Evolution of the Eurasia Foundation ............................................... 1 Evaluation Design and Methodology ......................................................................................... 4 Findings and Conclusions ............................................................................................................ 6 Issue I: Has EF Contributed to the Achievement of the Three SOs Identified in the Original Grant Agreement? ....................................................................................................................... 6 Issue II: Has EF-DC succeeded its efforts to establish and build sustainable foundations through its Network prior to the end of core grant funding? .................................................... 21 Issue III: What are the strengths and weaknesses of EF as a model for advancing US development objectives? ........................................................................................................... 35 Recommendations ....................................................................................................................... 44 Annexes ........................................................................................................................................... I Annex A – Scope of Work .......................................................................................................... I Annex B – Detailed Methodology, Limitations, and Threats to Validity ............................. VIII Annex C – List of People Contacted ..................................................................................... XIV Annex D – List of EF’s Gender Indicators ..........................................................................XVIII Annex E – Grant Agreement SOW ....................................................................................... XIX Annex F – Spending Across SOs by Country ...................................................................... XXV Annex G – Interview Guides ............................................................................................. XXVII Annex H – Electronic Survey.......................................................................................... XXXVI Annex I – Individual PF Case Studies .............................................................................. XLVII List of Figures Figure 1: EF and PF Organizational Chart .................................................................................... 3 Figure 2: Grant Analysis of Grant Amounts over Time ................................................................ 6 Figure 3: Grantee Survey Response............................................................................................... 7 Figure 4: Senior Management Self-assessment of Impact and Effectiveness ............................. 14 Figure 5: Performance Against Financial Sustainability Targets ................................................ 28 Figure 6: History of implementation for each PF ........................................................................ 32

EXEUCTIVE SUMMARY BACKGROUND ON EURASIA FOUNDATION The Eurasia Foundation (EF) is a publicly funded, privately managed foundation working to foster open, just, and progressive societies in the former Soviet Union. A joint United States Agency for International Development (USAID) and Department of State’s (DOS) Office of the Coordinator of US Assistance to Europe and Eurasia (EUR/ACE) initiative, EF began operating in May 1993 with an initial $75 million grant. It received a second $104 million grant in 1997 and a third $148 grant, its current, in 2001, which is due to end in December 2013. EF’s grant agreement states that EF “was created explicitly to deliver a bottom-up, grassroots grants program in Eurasia.”1 It began as a traditional grantmaker providing both open-door and targeted grants to the most promising and sustainable Eurasian institutions carrying out activities consistent with three broad United States Government (USG) strategic objectives (SOs): Private Enterprise (PE) Development; Civil Society (CS) Strengthening; and Public Administration (PA) and Policy Reform. However, in the early 2000s, EUR/ACE requested EF shift away from its reliance on core funding to a model of financial sustainability. EF responded in 2004 by beginning to establish the EF Network, “a constellation of affiliated, locally-registered foundations in Russia, Central Asia, the South Caucasus, Ukraine, and Moldova,”2 as well as a locally registered representative office in Belarus. In 2010, EF produced a Sustainability Plan, which established the conditional release of additional core funding upon EF-Washington, D.C. (EF-DC) and its five partner foundations (PFs) meeting specific fundraising targets. Following nearly 20 years of operation, $320 million in USG core funding, and activities spanning 12 countries, EF finds itself at a crossroads. The prospect of additional USG core funding looks unlikely, and EF-DC and its PFs face significant challenges to their future sustainability and are increasingly shifting from a foundation to a service provider model in search of new donor funding. EF’s experience presents a unique evaluation opportunity to not only take stock of past performance and organizational capacity internally but also for the USG to assess the comparative advantages and disadvantages of the “foundation model” in achieving high-level foreign policy objectives externally.

EVALUATION PURPOSE AND METHODS USAID/E&E contracted Social Impact (SI) to conduct a summative performance evaluation of its assistance to EF under its current grant. A similar evaluation was conducted in 2001 toward the end of the previous grant.3 The evaluation is divided into two parts. Component I focuses on evaluating EF’s success in meeting the grant’s three SOs, assessing EF’s organizational capacity, and commenting on the comparative advantages of the foundation model. Component II, which follows in a separate evaluation report, focuses on EF’s representative office in Belarus, the New Eurasia Establishment, assessing its organizational capacity, overall performance, and alignment 1

“The Eurasia Foundation Grant Agreement,” Grant No. EMT-G-00-02-00008-00. Pg. 10. Eurasia Foundation website. “About Us.” http://www.eurasia.org/about-us. Accessed on January 15, 2013. 3 See Blue, Richard, et al. “Final Report: Eurasia Foundation Evaluation.” Nathan Associates. January, 2001. 2

to USAID/Belarus’s SOs. A team of SI and USAID/E&E evaluation specialists, five from SI and three from USAID/E&E, conducted the evaluation between September 2012 and March 2013. They utilized a qualitatively focused but mixed-methods approach involving a desk review of primary and secondary documents; key informant and focus group interviews; an electronic survey of EF grantees across all 12 countries; and a Delphi Advisory Group of former foundation executives. Field data collection involved over 130 interviews with EF staff, donors, partners, and beneficiaries in eight Eurasia countries (Russia, Ukraine, Moldova, Armenia, Azerbaijan, Georgia, Kyrgyzstan, Tajikistan, and Belarus) between September and October 2012.

KEY FINDINGS AND CONCLUSIONS The findings and conclusions presented below represent the evaluation team’s selection of those believed to be of most importance and relevance to the reader. A complete listing of findings and conclusions can be found in the main body of the report.

EF’S CONTRIBUTION TO ACHIEVEMENT OF THREE OVERARCHING SOS LEVEL OF INVESTMENT WITHIN EACH SO Overall, the evaluators found that there is a distinct emphasis on CS programs over the life of the grant. Data provided by EF shows that 46% of total grant amounts fall under CS, followed by PE with 33%, and PA with 21%. A trend analysis illustrates that while EF placed an early emphasis in PE, CS overtook PE in 2004 and continued to be the top-funded SO until 2011 (see page 6). Investment across the SOs in EF’s flagship programs was more equally distributed; however, this investment more accurately reflects areas where EF could play a role in promoting economic and democratic reform rather than an overt attempt to invest in specific SOs. Regarding the PFs’ individual portfolios, the team found that current PF programs are strongly oriented toward CS. Interviews with PF staff indicated that this was due in large part to their core competencies, country contexts, and respective donor interests.

LEVEL OF SUCCESS WITHIN EACH SO The current grant agreement explicitly states that EF was created to deliver a bottom-up approach to support small grassroots initiatives in strengthening local civil society, which was expected to result in increased demand for national level political reform. Although the 2001 evaluation of EF’s core grant (the latest external evaluation available) recommended that EF shift its focus to working at the national level, the drafters of the current grant agreement chose to continue the grassroots-level focus. Consistent with the grassroots focus, the EF Network has awarded close to 3,000 grants to date and implemented numerous direct programs in three broad SOs in 12 countries. As EF did not collect data on national level impact and the evaluation did not allot the resources needed for the evaluators to collect the data themselves, the evaluators were unable to test the grant agreement’s development hypothesis that grassroots activity would spur national-level reform. However, the evaluators relied on qualitative interviews with EF donors, staff, and beneficiaries to get a sense of where interviewees thought EF had the greatest level of impact. They found that donors and PF staff consistently replied that the PFs’ core competencies, and thereby implicitly

their largest level of perceived impact, fell in the CS area. The evaluators also noted that even when PFs were engaged in PE and PA activities, their focus was often on working with CS partners to bring about changes within these SOs.

FACTORS THAT FACILITATED OR HINDERED SUCCESS In terms of general competencies, interview responses consistently indicated that the PFs’ key strengths were in building both the organizational and technical capacity of its local partners; their emphasis on treating local entities as partners rather than solely as beneficiaries; and their extensive local knowledge and networks. In some cases, PF leadership also described community-based projects as a niche, noting that there are fewer competitors doing this work compared to national-level projects. As for country context, regional democratic backsliding presented both a challenge and opportunity for the PFs, influencing the nature and success of their programs. PFs in all regions noted that corruption and the degree of democracy had implications for both CS and PA efforts. The variation in local partner capacity was also another key factor influencing PF success. Organizational factors such as the PFs’ reputation as international organizations or the PFs’ staff continuity and skill areas were also significant determinants of success. While the PFs’ USaffiliation has in some cases hindered their ability to work with national governments, this affiliation has led to PFs being widely recognized as having strong financial and program management and grantmaking capabilities due to their international background, making them appealing to donors and local partners. However, as the PFs broadened their donor base to focus on sustainability, staff at times lacked the required technical skills in new program areas.

STEPS TAKEN TO ENSURE GENDER BALANCE EF-DC continues to track gender-related metrics for all programs and some PFs have programs that specifically address gender issues, which is particularly relevant in Central Asia. One PF has a written gender mainstreaming policy. However, as the PFs have evolved into independent organizations, there does not appear to be a consolidated approach to gender.

CONCLUSIONS: EF’S LEVEL OF ACHIEVEMENT IN EACH OF THE THREE SOS •

There is a clear upward trend in investment in CS programming over the life of the current grant. In addition, EF programs appear to have been more successful in CS, relative to the other SOs, followed by PE and PA respectively, according to donor and beneficiary interviews that consistently described EF as having had an impact in developing civil society and numerous local CSOs.

•

The grant agreement emphasized working at the grassroots level. While some programs did in fact focus on national-level reform issues, it was not possible to demonstrate that the grassroots interventions led to improvement at the national level in any of the SO areas.4 Furthermore, the grassroots approach makes it almost impossible to measure

It should be noted that the evaluators did not rigorously test for national level impact; they did not explicitly ask about national level impact in their interview protocol, trace causal logic chains, or run an impact evaluation. However, in nearly all interviews, if the KI spoke to national level issues while being interviewed about a particular project, the team would follow up by asking whether the project had any national level impact. This was particularly

iii

impact at the national level given that the EF network awarded close to 3,000 grants and implemented numerous direct programs in the broad SO areas in 12 countries with a combined population of 255.5 million. While the total size of the third core grant seems large, $148 million over 10 years, it actually represents about five cents per person per year. Several sources noted that successes occurred at the local level but that given the size and scale of several of the countries, particularly Russia and Ukraine, it was unreasonable to expect national level impact. •

Country context, particularly democratic conditions and civil society landscape, had implications for success as well as the overall ability to work in CS and PA areas. In some countries, undemocratic conditions precluded PFs from being able to work at the national level in the PA space effectively. Issues such as conflict, corruption, and political instability also created challenges for working in PA at the national level. Furthermore, suspicion of the PFs as US organizations hindered work in both CS and PA areas. Logically, the availability of strong NGOs or other partners in the CS area was cited as a determinant of success. In countries with a richer NGO environment, the PFs felt they had more success than in countries where NGOs were weaker.

•

The EF Network’s reputation as independent and transparent organizations contributes to their ability to build trust with local partners and donors. Staff longevity can affect the performance of the PF overall, and high turnover clearly has negative consequences on the success of specific programs regardless of the SO alignment. There are also cases in which staff did not possess the appropriate technical skills for implementing donor programs regardless of their longevity with the organization.

LESSONS LEARNED •

If you mandate that a foundation operate with a bottom-up, grassroots-level approach, it will be exceedingly difficult to produce convincing evidence to link project level activities to observable changes in national-level indicators, particularly within fixed budget and LOE constraints of a typical evaluation.

EF’S SUCCESS IN ESTABLISHING AND SUPPORTING SUSTAINABLE PARTNER FOUNDATIONS EF has faced the sizeable task of overseeing its PF’s transformations while at the same time managing its own in the face of declining resources. This section analyzes the extent of EF-DC’s success in building its PFs’ managerial, operational, and financial sustainability.

ESTABLISHMENT AND EARLY SUPPORT Before commenting on EF-DC’s steps to strengthen the managerial and operational capacity of its PFs, the evaluators thought it important to underscore the significant amount of work, and relative success, that EF-DC had in merely establishing these PFs in the first place. EF-DC true with issues surrounding public administration. Furthermore, and more importantly, the evaluators did consistently ask whether the project was successful and in what areas the PF was most effective, which oftentimes led to provision of information on impact. The evaluators also reviewed PF documentation on impact. Following these methods, the team did not identify impacts at the national level.

worked with PF staff and local lawyers to navigate the various host country laws associated with establishing independent foundations. This had the complicating effect of requiring different operational structures to respond to the various host country regulations. Each structure has its own comparative advantages and disadvantages along with unique challenges to organizational sustainability.

LEADERSHIP AND MANAGERIAL CAPACITY Within the PFs and even EF-DC, different functions such as strategy identification, control, and execution are distributed respectively among the Boards of Directors, Advisory Boards, and the PF Executive Directors/Presidents. The team found that EF has placed high emphasis on the development of the Board of Directors as an oversight institution of the PFs but that these boards vary in how actively engaged they are programmatically and operationally with their respective PFs. However, in general the boards do carry out oversight, governance, advisory, and fundraising functions in close collaboration with the respective PF Presidents. The team found that PF leadership was a significant determinant of future sustainability. Russia provided a particularly apt example of how strong and resourceful PF Presidents can help their foundations navigate significant threats to their operation through thoughtful adjustments to their funding sources and operational model. In terms of strategic planning, the team found that all PFs indicated that they had developed at least a three-year strategic plan with regional and local partners utilizing different strategic planning approaches. The strategies were found to be adequately done and provided rationales for strategic objectives and periodic updates with board input; however, the team noted that foundation staff were more familiar with their annual operational plans than their strategic plans.

OPERATIONAL CAPACITY Interviews with PF staff indicated that a significant step for ensuring their sustainability is the building of in-house technical expertise, as staff acknowledged their limited technical knowledge and expertise in specific areas of their work. However, the evaluators found that PFs generally do not have a formal plan for staff development and limited and diminishing core funding will force the PFs to reconsider the staff policies, trainings, and structure. Interview responses overwhelmingly emphasized that the strong program management procedures inherited by the PFs are a key operational strength for future sustainability. However, over 70% of interviewees also indicated that these procedures needed to be further refined to respond to their specific host-country needs. EFâ&#x20AC;&#x2122;s establishment and support of the EF Network as a means of leveraging economies of scale and knowledge sharing within the Network is another key step taken to encourage sustainability. However, the evaluators found that opinions on the utility of the Network varied significantly among the PFs, particularly according to their various stages of development. They also found minimal evidence to suggest that there were significant benefits or economies of scale for individual PFs. While some respondents commented that the Network was useful for identifying new donors and funding opportunities, few of these respondents were able to provide specific examples. There was similarly little evidence of efficiencies in the provision of specialized services such as technology, communications, and evaluation. As for EF-DCâ&#x20AC;&#x2122;s role in supporting the PFsâ&#x20AC;&#x2122; long-term institutional sustainability and program success, much remains to be seen. The evaluators did not find substantial evidence that EF-DC improved the identification,

implementation, or monitoring and evaluation (M&E) of PF projects.

FINANCIAL OPERATION AND SUSTAINABILITY OF PFS The grant agreement stipulated that EF would be sustained through an endowment. However, such an endowment was never established during the life of the grant.5 Instead, the decision was made to shift EF’s operational model toward building sustainable legacy foundations given specific fundraising targets that it needed to achieve. Incremental funding was tied to achieving those objectives – at least informally – through the Sustainability Plan. While the EF Network has broadly met these targets to date, there is significant variation among the PFs based on country context and available sources of funds, and some might not successfully transition to long-term sustainability. EF-DC is also struggling to adjust to new realities without core funds. It has attracted some corporate money and has won competitive bids for other USAID and State Department funded programs, but without core funds EF-DC increasingly finds itself moving from a foundation to a service provider model.

DEVELOPMENT AND IMPLEMENTATION OF THE CMI TOOL One of the most significant steps EF took to support the sustainability of its PFs was the development and implementation of its Capacity Mapping Initiative (CMI) tool. However, the evaluation team found that CMI was unevenly applied and that the capacities mapped and people interviewed to gauge those capacities appeared to be determined by a few select employees, usually senior EF and PF staff, thereby introducing a significant risk of selection bias. The evaluation design does not include an assessment of the accuracy of the CMI tool itself. However, it does allow the evaluators to present a summary of the opinions of PF staff on CMI’s effectiveness in building organizational capacity. The majority of interviewed PF staff answered that CMI was generally a good tool to paint an overall picture of an organization; to detect problem areas, bottlenecks, or areas for improvement; and to organize their PF’s approach to capacity building.

CONCLUSIONS: EF’S SUCCESS IN ESTABLISHING AND SUPPORTING SUSTAINABLE PARTNER FOUNDATIONS 

The adoption of a new business model of sustainability following the end of core funding requires significant transitions in a range of organizational capacity areas beyond a sole focus on fundraising. The process can entail significant downsizing and staffing changes and requires significant lead-time. The evaluators found that FNE has undertaken the most significant transition and adopted a business model distinct from the other PFs, but most other PFs have not really undertaken a major organizational transformation to date. EF-DC does not appear to have fully recognized the need for a transformational change from a grantmaking to operational foundation and has had limited capacity to effectively manage its PFs’ transformations, which is in large part due to reduced core funding.

N.B.: The grant agreement stated that EF should rely on USG funding until the endowment was established; however, it did not specify whether the endowment would be to maintain EF indefinitely or for a fixed period of time (say 10 or 15 years) with the expectation that EF would disappear at that time as the mission would be broadly accomplished. The implicit assumption was that EF would have a stream of funds from the USG until the endowment was established and that they would then continue their work, indefinitely or for a fixed period of time, using the revenue from the endowment.



Many respondents, however, mentioned that PFs inherited strong operational procedures will need to be further adapted to individual country contexts. PFs also face a dual challenge of needing to build technical expertise while at the same time responding to the additional burdens of declining human resources and support from EF-DC caused by decreases in core funding. Finally, the EF Network does not appear to be providing the economies of scale and expected benefits envisioned in EF’s Sustainability Plan. It is of greatest benefit to the least developed PFs within the Network; however, the importance of EF-DC decreases as PFs mature and smaller, more regionally focused networks provide greater benefit.



With the end of core funding, from a financial perspective, some PFs are at risk. In some cases, PFs have to undertake a significant program realignment to meet the needs of new donors. This is already happening in Russia and to a lesser extent Central Asia. Organizations may find themselves in very different lines of business than those undertaken when funding and program direction came from the USG. The provision of key EF-DC functions to the PFs, such as fundraising and M&E, have been lost due to a reduction in core funding. It is critical that the PFs have sufficient resources to cover fundraising/proposal development, which traditionally had been covered by core funds.



Financial sustainability is compromised in some cases in which the USG has decided to decrease funding to these countries or determined that the locally registered PFs are not eligible to compete for funding reserved for local organizations either because of their board structure, which includes internationals, or their perceived alignment to EF-DC.6



The CMI tool was not applied consistently and was not used to its full advantage.7 However, the majority of interviewed PF staff believe that CMI produces scores that accurately reflect their individual capacity areas, but most also commented that there has been little follow-up by EF-DC and that some, but not all, of the recommendations produced by the exercise have been implemented by the PFs.

LESSONS LEARNED 

Once core funding ends and a foundation needs to compete for funds in order to survive, its willingness and ability to continue to support specific US foreign policy objectives will be diminished. The EF experience conveys the lesson that when and if core funding ends, the mission of the foundation will likely start to move away from supporting the original donor objectives. US policymakers need to be aware of the implications of the decision to end core funding and the potential reduced focus on US priority objectives.

ADVANTAGES AND DISADVANTAGES OF THE FOUNDATION MODEL FOR ADVANCING USG SOS Aiming to increase the utility of their findings, the evaluation team modified the original 6

The evaluators encountered several examples, the most striking of which occurred in Armenia where the PF was excluded from a competition because its board structure (which does not include any native Azeris or Armenians) does not meet USAID Forward principles. The issue is that Azeris and Armenians will not sit on the same board, so a predominantly local board is not possible. 7 It should be noted that the CMI process was phased out due to limited core funding.

vii

evaluation question to focus on an analysis of the strengths and weakness of foundations in general (i.e., the “foundation model”) for advancing US development objectives and also to provide a commentary on how well EF was able to conform to this model. Given EF’s experience, in which a dedicated funding stream was discussed but never implemented, the team felt that there was limited value in commenting on the usefulness of replicating this model, which they would not recommend. However, the team felt that a more macro-level discussion on the comparative advantages and disadvantages of foundations in general, and the extent to which EF was able to capture these, would prove more useful to future programming.

DEFINING THE TERM “FOUNDATION” The team found surprisingly few commonly accepted definitions of foundations, perhaps due to the varied structures, activities, revenue sources, and country laws of foundations. However, one defining characteristic appeared in most definitions – namely, the availability of a steady funding stream usually in the form of an endowment or trust. It is on this criterion that EF appears to have deviated most significantly from the traditional foundation model as the transition to sustainability was undertaken.

COMPARATIVE ADVANTAGES AND DISADVANTAGES OF THE FOUNDATION MODEL IN TERMS OF SPEED, FLEXIBILITY, AND INNOVATIONS COMPARED TO USAID CONTRACTING Based on secondary data, interview responses, and their Advisory Group discussions, the team found that foundations offer comparative advantages in terms of flexibility in programming, speed of start-up of projects, and an ability to pursue long-term objectives compared to traditional direct contracting models. Foundations benefit from their longer-term institutional presence (compared to contractors), which leads to deeper knowledge and relationships with local partners fostering greater acceptance, access, and trust. However, it should be noted that one of the defining characteristics of foundations, and the basis of the foundation model’s most significant and tangible comparative advantages, is the availability of a reliable source of longterm funding which was not present in the EF example. On the other hand, the team found that contractors operating under USAID projects often have a defined period of performance and defined scope of work. These attributes mean that contractors may be better suited to work on issues that can be addressed within a defined life of project and which require specific technical expertise. Contracts by their nature are not quick or flexible and may be more suitable for addressing a specific set of well-defined issues. A complete listing of the individual comparative advantages and disadvantages of the “foundation model” can be found on pages 35-38.

EF’S CONFORMITY WITH THE FOUNDATION MODEL The team found that EF’s performance in terms of speed, flexibility, and innovation in comparison to USAID contracting was largely dependent on the availability of core funding. When core funding was available, EF was able to capitalize on many of the advantages of the “foundation model” and generally outperformed other contractors in terms of supporting local ownership of programs, fostering sustainability, and supporting the social capital needed to enhance democratic processes. However, as core funding declined, EF was no longer able to capitalize on these advantages and did not perform as well as contractors, particularly in terms of programmatic focus and depth of technical expertise. In terms of sector-specific comparative advantages, the evaluation team concluded that EF’s viii

work has generally been most successful in the civil society arena. This result is in accord with the trends in PF portfolio investments observed earlier in this report. Given the predominance of civil society activity in the overall program of EF, it is not possible to determine the effectiveness of EF’s model in other areas from its work in civil society. While the evidence here is clear that the model “performs well” in supporting civil society assistance, this does not mean that the model would not work for other sectors as well. Even though the evidence available in this evaluation was limited, several foundations, such as The Asia Foundation, have demonstrated success in these sectors.

BENEFICIARIES’ EXPERIENCE WORKING WITH EF Beneficiaries often found PF’s operational expectations, such as quality of proposals, reporting, and financial transparency, to be more rigorous than other grantmakers, and in the end, PFs were usually seen to be supportive, competent partners. The more experienced grantees were able to compare PF expectations with those of other donors. In some instances, the level of difficulty was about the same as with other grantmakers, but more often PF processes were considered more challenging. However, some interviewees stated that high expectations by the PF helped their organizations to build capability and find subsequent sources of new funding. Most interviewees stated that they would work with the PF again.

EF COORDINATION WITH USAID MISSIONS AND OTHER USAID PROGRAMS The EF regional grant is overseen by a USAID Agreement Officer’s Technical Representative (AOR) based with the Bureau for Europe and Eurasia in Washington, D.C. At the country level there are no formalized coordination points or mechanisms except on occasions (such as in Moldova, Kyrgyzstan, Armenia, and Russia) when the USAID Mission has awarded one of its own programs to a PF. In Russia, FNE is widely known and FNE leadership and staff talk regularly with donors – although of course with the closing of the USAID program in Russia such frequent communications are not expected in the near future. The working relationship in Moldova is not close and USAID personnel noted a lack of coordination with EEF-M, although we note that the Country Director was invited to visit the most recent meeting of the EEF board and other events. Further, donor coordination meetings convened by USAID/Embassy, which had been effective in earlier years for understanding and directing the work of EEF-M, are no longer held regularly. On the Washington side, EF-DC historically has looked to State and EUR/ACE for oversight and coordination rather than to USAID. For its part, USAID’s historical pattern has therefore been to exert little oversight through the AOR.

CONCLUSIONS: ADVANTAGES AND DISADVANTAGES OF THE FOUNDATION MODEL 

Foundations offer comparative advantages in terms of flexibility, speed, and ability to pursing long-term objectives compared to traditional direct contracting models. Contracts by their nature are not quick or flexible and may be more suitable for addressing a specific set of well-defined issues. These models can thus complement each other by capturing advantages and benefits not captured by the other.



There was a marked difference in EF-DC and the PFs’ ability to conform to the comparative advantages of the foundation model as it became clear that core funding would end. As articulated above, having a reliable funding stream is a defining

characteristic of foundations and basis for key comparative advantages. 

While the application, monitoring, financial management, and reporting requirements of EF grantmaking can be challenging at times for grantees, these operational expectations appear not to be overly burdensome to beneficiaries and indeed prove to be opportunities for valuable capacity strengthening for many.



Despite a strong recommendation from the first evaluation, the level of coordination between PFs and USAID Missions is inconsistent. When the PF has received money from the bilateral mission, they become an implementing partner with strong communications linkages, whereas in other countries the USAID mission was not even aware the PF existed.

RECOMMENDATIONS RECOMMENDATION FOR USG AUDIENCE

USAID/PPL SHOULD CREATE AN AGENCY-WIDE POLICY ON FOUNDATIONS USAID has and will continue to support foundations in addition to EF in the Europe and Eurasia region and elsewhere. As a result, USAID has continued to gain experience working with foundations in support of critical foreign policy objectives and developed a deeper understanding of the strengths and weakness of the model. USAID’s Bureau for Policy, Planning, and Learning (USAID/PPL) should consider developing a policy for establishing, supporting, and working with foundations. The policy should define what constitutes a foundation; describe the different kinds of foundation models (e.g., endowed vs. non-endowed) and outline their comparative strengths and weaknesses; differentiate when USAID should consider creating its own foundations versus collaborating with existing foundations; and clearly articulate how to support and sustain foundations advancing desired USG foreign policy interests. This policy could be informed by this evaluation but should draw from a variety of sources in order to properly capture the richness and diversity of the foundation world.

DESIGNERS AND POLICYMAKERS SHOULD ESTABLISH THE FUNDING AND EXPECTATIONS OF A FOUNDATION UPFRONT It is vital that the conditions for funding foundations are explicitly stated and followed upfront. A foundation can be designed to exist indefinitely based on an endowment, exist for an unspecified period of time based on a “sinking fund,” or operate for a fixed period with a dedicated source of funding. As witnessed in the EF example, the implication of switching towards a sustainability model, compared to the traditional model of having a sustained source of funding, limited EF’s ability to capitalize on the comparative advantages of the foundation model. It also demonstrates the consequences of designers and policymakers failing to have a clear idea up front on the type of foundation they are establishing. Policymakers need to understand that changes in operating assumptions made midstream, such as switching its operational model to one focusing on the sustainability of local partner foundations, can be very disruptive to the effectiveness of the foundation and can significantly impact a foundation’s ability to capitalize on the advantages of the foundation model. To best capture these comparative advantages, particularly in terms of speed, flexibility, and long-term view, a foundation must have access to a reliable source of long-term funding. x

POLICYMAKERS NEED TO RECOGNIZE THAT A TRANSFORMATION IN A FOUNDATION’S OPERATIONAL MODEL REQUIRES SUBSTANTIAL INVESTMENTS OF TIME, RESOURCES, AND ORGANIZATIONAL CHANGE EXPERTISE. Should policymakers find themselves in the suboptimal position of needing to mandate a change in a foundation’s operational model, policymakers need to allocate sufficient time, resources, and expertise to allow for a full transformation to take place. Policymakers need to realize that if requiring a transition to an operational program, a foundation will need significant organizational restructuring and that additional resources and support may be required in the short term to make this transition possible. Policymakers should realize that the foundation’s leadership may not have the right skills or capabilities to lead this transition. Organizational change expertise is required, which the foundation headquarters is not likely to have. Also, if the headquarters entity is undergoing a similar transition, they may have fewer resources to devote to the transition of local “affiliates.” In the EF example, both EF and the USG did not appear to fully recognize the difficulty of this transformation and the level of time and investment needed to do so. From the foundation perspective, the organization’s leadership: (1) must recognize the need for an organizational restructuring; (2) develop a plan to significantly realign staff and identify the needed resources; and (3) implement concrete steps to overhaul its staffing composition. This should include investing heavily in training current staff in and/or hiring new staff that possess the required competencies.

POLICYMAKERS SHOULD EXPECT THAT THE END OF CORE FUNDING WILL LEAD TO PROGRAMMING LESS DIRECTLY ALIGNED WITH USG FOREIGN POLICY INTERESTS Once core funding ends and a foundation needs to compete for funds in order to survive, its willingness and ability to continue to support specific USG foreign policy objectives will be diminished. Foundations will have the incentive to pursue other donor funding which may or may not align with USG interests.

RECOMMENDATIONS FOR EF AUDIENCE

USAID MISSIONS SHOULD SEEK MORE ACTIVE ENGAGEMENT WITH PFS REGARDLESS OF WHETHER THEY ARE IMPLEMENTING PARTNERS OR NOT

There are benefits to having a local foundation on the ground and USAID should leverage that as a means to build understanding and networks as well as consider areas of collaboration. There are specific actions that could be taken, such as including EF PFs in regular meetings of implementing partners regardless of whether the PF is a direct recipient of USAID funding at that time. USAID AORs could do more to foster the relationship between the field Missions and the PFs by increasing communication to the Missions regarding EF activities. The AORs should continue to visit PF offices while in country and share reports and other relevant information.

EF-DC AND PF SHOULD UNDERTAKE SIMILAR EFFORTS TO ENGAGE WITH USAID EF should increase its outreach to USAID with information on its ongoing activities. EF should seek to invite USAID to public events, share annual reports, and convene their own donor coordination meetings in areas where the PF has clear expertise.

INTRODUCTION PURPOSE OF THE EVALUATION This evaluation report provides the United States Agency for International Development (USAID) with a high-level assessment of its support to the Eurasia Foundation (EF) under its current grant agreement, the third awarded since 1993. The $148m grant (2001-present) provides core funding to EF and its EF Network – a constellation of affiliated but locally registered foundations in Russia, Central Asia, the South Caucasus, Ukraine and Moldova – to promote “open, just, and progressive societies” in the former Soviet Union.8 9 With the grant due to end on December 31, 2013, USAID procured this evaluation to: 1. Assess how well EF has met the grant’s three broad Strategic Objectives (SOs): promoting private enterprise (PE), civil society (CS), and public administration and policy reform (PA); 2. Analyze the success of EF’s strategy for building sustainable partner foundations; and 3. Identify the advantages and disadvantages of the “foundation model” in delivering USAID assistance as opposed to contracts and/or “project-based” assistance. The report aims to build a better understanding of the strengths and weaknesses of the foundation model to inform various audiences within USAID and the broader United States Government (USG) on future program design. It provides findings that may be useful to the EF home office (EF-DC) on how to improve their operating model, sustainability efforts, and support to PFs. These recommendations will also be useful for other foundations and non-governmental organizations (NGOs) providing similar development assistance.

BACKGROUND, CONTEXT, AND EVOLUTION OF THE EURASIA FOUNDATION The “Stremlau Memo” of January 1992, named after the then Deputy Director for Policy Planning at the US Department of State (DOS), called for the establishment of EF as a “cost effective and political way to accelerate and broaden US USAID Grants to EF: engagement in the development of civil societies in the 1. Core Grant I - 1993-1997 ($75m) region.” It explained that EF’s goal was to provide “many 2. Core Grant II - 1997-2001 ($105m) small grants to assist grass-roots organizations” across the region, where non-existence of civil society created 3. Core Grant III – 2001-2012 ($148m) particular challenges. The foundation model was cited as

“Eurasia Foundation.” EF website, http://www.eurasia.org/about-us. Accessed on November 12, 2012. N.B.: Part of the core grant funding also goes to supporting EF’s programs in China but these programs are outside of the scope of this evaluation. 9

having a comparative advantage in reaching civil society,10 and the work of EF was to be complementary to the activities of larger bilateral and multilateral agencies. Subsequent grant agreements further clarified that EF should fund the most promising and sustainable Eurasian institutions and carry out programs consistent with USG regional policy objectives. The focus and activities of EF have consequently evolved over time. EF initially awarded small grants for projects proposed through unsolicited applications known as the “open door” mechanism. This grantmaking approach allowed EF to respond to needs and solutions identified by local institutions. In 1999, “EF began to get rid of open door grant making as it is expensive to administer and difficult to fundraise,” switching instead to a “grants plus” approach which provided larger but more targeted grants that were often accompanied by technical assistance, training, and networking opportunities.11 Grant competitions also concentrated EF’s resources in particular fields of interest, such as supporting local government councils, watchdog NGOs, or regional media outlets. In 1995, in consultation with USG and other partners, EF began implementing “Foundation-Administered Projects” (FAPS) to extend the reach and impact of its core grant programs. EF’s three FAPS were: •

The Economics Education and Research Consortium (EERC), which supported economics education in Ukraine and economic research and policy analysis in Russia. In 2003, the EERC emerged as an independent institution – the Kiev School of Economics (KSE) – that has awarded more than 500 master’s degrees since its inception.

•

The Small Business Lending Program (SBLP), which disbursed about $6.8 million to 132 borrowers over ten years of operations in Ukraine, resulting in 828 new jobs and loan losses equal to just 1.04% of total disbursals. In Armenia, SBLP disbursed about $10.5 million to borrowers over ten years resulting in 2,219 new jobs. Both programs were terminated by EF when local banks met the market demand.

•

The Media Viability Fund (MVF), which provided technical consultations and access to loans to independent media outlets in Russia and Ukraine. In 2004, the MVF evolved into the Russia Independent Print Media Program (RIPMP), which still operates today serving more than 50 clients and directly assisted with the establishment of the Alliance of Independent Regional Publishers in Russia.

An evaluation of the second EF grant found that EF’s grantmaking program had largely met its objectives and “had significant impact, especially at the micro-level of NGO development.”12 However, the evaluation also noted that EF needed to focus on steps to translate grassroots action into lasting change at the regional and national level. The evaluation also noted that EF has the infrastructure and leadership to become a USG “legacy” organization in the region and made a recommendation that EF seek to diversify its funding base and that USAID consider offering matching funds for a public-private EF endowment. 10

“The Eurasia Foundation would emulate the success of the Asia and other foundations in filling vital grass-roots needs not covered or easily accessible to larger official bi-lateral and multilateral donor agencies…experience in other regions has also shown that congressionally funded foundations are able to work more effectively in local communities because they are perceived to be independent and, in any event, less interventionist than official donors. Moreover, they can support private initiatives in ways that are difficult if not impossible for bi-lateral and multilateral donors.” 11 Quote from email communication with EF-DC staff. 12 Blue, Richard et al. “Final Report: Evaluation of the Eurasia Foundation.” Nathan Associates. January 2001.

However, the stipulation of the current grant agreement did not follow the recommendations of the previous 2001 evaluation but instead continued to emphasize bottom-up, grassroots activity as a means to advance USG objectives. It focused on similar areas as previous grants and referenced a plan to leverage USG resources through the establishment of an endowment or trust with the understanding that EF would remain primarily a grantmaking institution. During its early phases of implementation (April 2003), the focus of the grant shifted at the direction of the USG from grants management and program implementation to establishing and building PFs in the region that would extend beyond the life of core USG funding.13 Consequently, since 2004, the Eurasia Foundation has evolved from a US-based foundation with multiple field offices into the Eurasia Foundation Network that works in partnership with Eurasia Foundation DC (EF-DC). To date five partner foundations have been established as follows: 

The New Eurasia Foundation (FNE) launched in Moscow in 2004.



The Eurasia Foundation of Central Asia (EFCA) launched in 2005.



Eurasia Partnership Foundation (EFP), launched in 2007, serving Armenia, Azerbaijan and Georgia.



East Europe Foundation of Ukraine (EEF-Ukraine), launched in 2007 and previously included Moldova.



East Europe Foundation of Moldova (EEF-Moldova), launched in 2010 as an independent entity.

Figure 1: EF and PF Organizational Chart

Eurasia Foundation (EF-DC) Washington, DC

New Eurasia Foundation (FNE) Russia

Eurasia Foundation of Central Asia (EFCA)

Eurasia Partnership Foundation (EPF)

Kazakhstan Tajikistan Kyrgyzstan Turkmenistan

Armenia Azerbaijan Georgia

East Europe Foundation (EEF) Ukraine Moldova

and futures….Unle ss the situation changes, the Network is not useful or necessary.” ~FNE New Eurasia

This change was made in “Amendment 3,” which added a section on grants to indigenous organizations to enable EF-DC to use core resources to establish these entities.

In addition, EF-DC has established the New Eurasia Establishment (NEE), which is a Belarusian legal entity, non-profit organization that acts in the capacity of a representative office of the EFDC in Belarus. It was founded in 2005 after the closure of the EFâ&#x20AC;&#x2122;s representative office by the Government of Belarus. The NEE is not structured as a partner foundation and is discussed separately in Component II. As a result, since 2004, funding for direct grants and operating programs declined as core funding was redirected to support the establishment of indigenous PFs. As part of this transition process, in June of 2010, a Sustainability Plan was agreed upon by DOS, USAID, and EF-DC that outlined a path to end the reliance on ongoing USG core funding. Thus, the vision of establishing a traditional endowment, defined as a dedicated pool of money transferred to EF, was unfulfilled and EF-DC and PFs were told to raise their own funds from US and non-US donor and private sources. The plan envisioned full financial sustainability for EF-DC without the need for further core USG resources by the end of 2012 with specific benchmarks to measure progress in meeting this goal. While the EF Network met many of these fundraising goals in the last two years, future funding streams remain uncertain and will remain a constant challenge.

EVALUATION DESIGN AND METHODOLOGY Evaluation Questions The evaluation Scope of Work (SOW) focused on three major evaluation questions, each with a set of specific sub-questions. It also specified how much to focus on each question. 1. Has EF contributed to the achievement of the three SOs identified in the original grant agreement (20% of LOE)? a) Was EF successful in advancing the three SOs identified in the original grant agreement and were EF and its partners better able to carry out programs in some SOs than others? b) Were there any contextual factors that facilitated or hindered the achievement of each SO? What steps did EF take to ensure a gender balance in each of the SOs? 2. Has EF-DC succeeded in its efforts to establish sustainable foundations through its partnership network prior to the end of core grant funding (40% of LOE)? a) What steps has EF-DC taken to ensure the financial sustainability of its partner network beyond the end of core grant funding? b) Does each of the partner foundations have the financial, managerial, and operational capacity to ensure ongoing program growth and sophistication? c) Is EF-DCâ&#x20AC;&#x2122;s Capacity Mapping Initiative (CMI) successful in accurately measuring the organizational capacity of partner foundations and addressing the key areas that allow for this determination? 3. What are the strengths and weaknesses of EF as a model for advancing US development objectives (40% of LOE)? a) Were there sector or country areas where the EF model speed, flexibility, and innovation had a comparative advantage in delivering development assistance compared to other forms of USAID assistance? 4

b) How do grantees and other beneficiaries report on their experience working with EF and its partners and how did this compare to working with other donor programs, i.e., were procedures open and transparent, how difficult was it to meet reporting, financial and other requirements of USAID compared to others, etc.? d) How did EF coordinate with USAID Missions and other USAID programs? Design and Methodology Social Impact (SI) employed a mixed-method evaluation design involving desk review, key informant and focus group interviews, a grant beneficiary survey, and a Delphi Advisory Group comprised of former foundation executives. The evaluation took place between September and December 2012 and involved six weeks of fieldwork conducting more than 100 interviews with USAID staff and other donors, EF-DC leadership and Board members, EF Partner Foundation staff, implementing partners, and beneficiaries. The beneficiary survey was distributed to 677 organizations that had received a grant(s) from EF (across partner countries) and 148 individuals responded. The evaluation team travelled to seven countries to conduct in-depth interviews including Russia, Ukraine, Moldova, Armenia, Azerbaijan, Kyrgyzstan, and Tajikistan. Qualitative and quantitative field data was subsequently synthesized and discussed in numerous sessions by the evaluation team to reduce bias and subjective judgments. Preliminary findings were then presented to the Advisory Group (AG) to provide a different perspective and layer of analysis before finalizing conclusions; however, the focus of the AG was on the question of the comparative advantage and disadvantage of the foundation model and the sustainability of EF’s PFs. A more detailed discussion on the methodology employed can be found in Annex B. Threats to Validity The selection of PFs, programs, and beneficiaries was the most significant limitation of the study. Given resource and time constraints, the evaluation team could not travel to all countries where EF PFs operate and thus selected a sample of which PFs they would visit and which of their respective programs they would review. This purposeful convenience sampling of countries was based on a matrix comparison of both the overall percentage of core grant funding received as well as the percentage of funding received in each of the three strategic objectives. The purposeful sample was intended not to be completely representative of all PFs but instead to sample PFs receiving the largest amount of funding. The evaluation team also intentionally sampled larger and more recent programs in the PFs’ portfolios to more accurately reflect and comment on how the majority of the USAID core grant was being spent. Programs that were reviewed included initiatives that are funded by other donors – both USG and non-USG, including competitive bids for USAID Mission-funded programs and DRL. Another significant threat to validity was the selection of grant beneficiaries interviewed following program selection. Given the limited time and logistical challenges of arranging focus group interviews during the evaluation team’s visit, the evaluation team sought the support of the PFs in arranging focus group interviews with project beneficiaries. PFs arranged interviews based on availability and possibly selected beneficiaries more inclined to give a favorable account of their interaction with the partner foundation. The evaluation team attempted to mitigate this bias through the use of probing questions during their interviews as well as distributing an electronic survey to a larger group of beneficiaries (1,800) in order to triangulate

and validate statements made during the focus group interviews. The evaluators also attempted to mitigate this risk by meeting with donors and implementing partners for these initiatives and not just beneficiaries. Please see Annex B for further details.

FINDINGS AND CONCLUSIONS ISSUE I: HAS EF CONTRIBUTED TO THE ACHIEVEMENT OF THE THREE SOS IDENTIFIED IN THE ORIGINAL GRANT AGREEMENT? Before determining the level of EF’s success in achieving the three SOs identified in the grant agreement, the evaluation team thought that it important to explore the level of investment made in each of the SOs. The findings below summarize the team’s research on the allocation of financial resources among the SOs and the specific trends of investment in EF grants, flagship programs, and the current portfolio.14 Overall, the evaluators found that there is a distinct emphasis on CS programs in both the grants and current program portfolio over the grant life. Grant Portfolio Data provided by EF shows that most of the grant activities over the life of the grant fall under the CS SO, accounting for 46% of total grant amounts.15 PE followed with 33% of total funding and PA with 21%. A trend analysis illustrates that while early emphasis was in PE, CS overtook PE in 2004 and continued to be the top-funded SO until 2011 (see Figure 1 below). Figure 2: Grant Analysis of Grant Amounts over Time

$7,000,000

Grant Amounts by EF Over Time

$6,000,000

All Countries, 2001-2011

$5,000,000 $4,000,000 $3,000,000 $2,000,000 $1,000,000 $0 2001

2002

Civil Society

2003

2004

2005

2006

Private Enterprise Development

2007

2008

2009

2010

2011

Public Administration and Policy

When the data is disaggregated by PF, this pattern is consistent across countries.16 For New Eurasia Foundation (FNE), Eurasia Foundation of Central Asia (EFCA), and Eurasia Partnership 14

See Annex E for how the SOs are defined in the Grant Agreement. The Methodology section notes data limitations on how programs are aligned to specific SOs. 15 Includes grant funding from USAID core funding to any country within EF’s purview during the 2001-2011 period regardless of whether that country is currently supported by an EF partner foundation. 16 See Annex F for additional charts.

Foundation (EPF), PE received the greatest funding early on but was overtaken by CS in 2003 and 2004. East Europe Foundation-Ukraine (EEF-Ukraine) and East Europe FoundationMoldova (EEF-Moldova) grants have been heavily focused on CS and neither country has made much investment in PE. This shift to CS was further evidenced by the evaluators’ grantee survey in which nearly 60% of respondents considered themselves to be engaged in CS activities (see Figure 3 below).17 The EF Fiscal Year (FY) 2003 Report to USAID helps to explain this shift, noting that, “[F]ield offices continued to identify a number of fields of interest within the Foundation’s mandate that resulted in guided grant-making for more focused results…field staff applied knowledge of country trends, needs and experience to both the development of the Foundation's grant portfolio and to programs funded by other donors.”18 As one EF-DC key informant clarified, “As core funding went down, EF and the partner foundations began to get rid of open door grant making as it is expensive to administer and difficult to fundraise for…The field offices, and later the partner foundations, were responding to needs they see in their communities and investing resources in those areas where they thought they had value or expertise to add. Later, when the partner foundations were largely using core funds to leverage other donor funds, the priorities of those donors also played a role in where the resources were directed.”19 Thus, the increased CS focus represents a series of conscious decisions based on EF’s understanding of country needs and opportunities. Figure 3: Grantee Survey Response

Flagship Programs In the third and current grant agreement, EF invested $3.75 million into the EERC, which was established in 1996 together with other donors. This program is best categorized as PA since its objective was to “create sustainable local capacity in economic research and analysis to support improved decision-making and policy in both private and public sectors at both the federal and regional levels” in Ukraine, Russia, and other countries.20

Given the following choices, which area of activity best captures your activities? Answer Options Private Enterprise Development Civil Society Public Administration Other (please specify)

Response Percent

Response Count

17.3%

59.6% 13.5% 9.6% answered question

31 7 5 52

In the current grant agreement, EF invested approximately $850,000 in SBLP, which was launched in Armenia and Ukraine in 1995 and is best categorized as a PE program. Its goal was to provide small business loans unavailable through commercial banks. In the current grant, EF invested $568,922 in MVF, which was designed to help cultivate a free and independent press in Russia21 and fits best within the CS SO. MVF provided technical consultations and access to loans to independent media outlets. In 2004, it evolved into the 17

However, it should be noted that the survey’s low response rate makes any definitive conclusions tenuous. EF FY 2003 Report to USAID. 19 Key informant interview with EF-DC staff. August 2012. 20 EERC Program Evaluation 2003. 21 FY 1996 USAID Annual Report. 18

Russia IPM Program. It still operates today and serves more than 50 clients. It also directly assisted with the establishment of the Alliance of Independent Regional Publishers in Russia, the leading professional group for independent media companies in Russia today. USAID continued the investment in the three FAPs in the grant agreement.22 However, the design of those programs during the previous grant was driven by EFâ&#x20AC;&#x2122;s understanding of where they could play a role in economic and democratic reform by implementing grants directly, as opposed to simply providing grants, and not as an attempt to invest in specific SOs. PF Portfolios Similar to the trend in grant funding discussed above, the team found that current PF programs are strongly oriented toward CS.23 Based on interviews with PF staff, donors, and in some cases beneficiaries, the evaluation team found several reasons why the current portfolios, which are funded by several donors and private sector sources, are more oriented toward CS. Staff at three of five PFs indicated staff and organizational core competencies drive programmatic focus. As a result, the PFs have focused on areas where they have a comparative advantage, which is more often noted in CS and, to a lesser extent, PA.24 Conversely, PFs also recognized areas where they did not have the technical expertise to engage, even if it was a priority issue in the country. For example, FNE noted that they never made a foray into human rights or rule of law because there were others better equipped to tackle these issues. Similarly, three PFs noted the lack of expertise in PE compared to other organizations.25 Country context also influenced programmatic decisions.26 For example, given political context, some PFs cited plenty of room for growth and demand in CS programs, thus shaping their priorities.27 In other instances, where the national government is not reform-oriented or is plagued by instability, PFs decided it was not possible to focus on PA.28 Programmatic focus was also driven by donor interests, but this was perhaps not always due to country context. Staff at 22

See Annex E for Current Grant Agreement, pg. 17. EPF and EFCA both described their portfolios as heavily focused in CS, including programmatic priorities in youth, media, support to vulnerable populations, community development, and cross-border engagement. They noted that they are least active in PE, and although PA is an area of focus for some priority programs in Moldova, Ukraine, and Central Asia, it tends to be at the local level. PF portfolios in Russia and Ukraine vary considerably, but both include a large CS component. FNE identifies four broad areas of focus: education, territorial development, social projects, and independent media. Territorial development, which is defined as PE by FNE, originally focused on small businesses but has expanded to include housing, urban planning, and vocational training, which may not be as specific to PE and rather be more related to CS. FNE also manages social projects for people with special needs as well as public health and energy. In Ukraine, the four areas of focus are local economic development, local social development, governance and civil society, and energy efficiency and environment. Thus, though in both Ukraine and Russia there are areas that do not clearly align to the SOs of the grant agreement, CS is still a clear focus area. EEF-Moldova is an outlier, as it has a greater focus on PA than the other PFs in areas such as civic and voter education. EEF-Moldova is also unique in that its work in PA includes several national level initiatives, while other PFs are focused more on the local level. 24 EFCA, EPF both said CS, EEF-Moldova in PA 25 EEF-Moldova, EPF, and EFCA 26 Please see Annex I, which provide individual PF case studies and richer country context. 27 EPF, EFCA, FNE 28 EPF and EFCA 23

three of the PFs felt that programmatic focus was at least some extent driven by donor priorities.29 PFs noted that they are “flexible” in responding to donor priorities and sometimes take on projects even if they would not consider it a country priority.30 Furthermore, in some countries it was noted that the donor interests are shifting, which will reduce their focus in some areas.31 Finally, in the case of EEF-Moldova, where the exception has been noted regarding the focus on PA, this was also donor driven in the sense that the US Embassy directed the PF’s focus in PA.

CONCLUSIONS: WHERE EF INVESTED AND WHY •

There is a clear upward trend in investment in CS programming compared to the other two SO areas in EF’s grants, flagship programs, and PFs portfolios over the life of the current grant.32 o While the grants program was initially “open door” and thus applicant driven, it evolved into a more targeted grantmaking approach favoring CS programs based on specific country needs and PF’s comparative advantages. o EF’s flagship programs demonstrated a more equitable level of investment among the three SOs; however, this appears to be more because of EF’s understanding of where they could play a role in economic and democratic reform than a conscious attempt to invest equally in each SO. o PFs’ current portfolios demonstrate a clear focus in CS programming. PF staff indicated that this focus was a result of the in-house technical expertise, comparative advantage in CS arena versus other organizations that were more experienced in PE and PA, and donor interest most often favoring CS programs.

WAS EF SUCCESSFUL IN ADVANCING THE THREE SOS AND WAS IT EASIER TO CARRY OUT PROGRAMS IN SOME SOS OVER OTHERS? EF’s current grant agreement emphasizes working at the grassroots level and did not adopt a recommendation of the previous grant agreement that the organization work more at the national level. Although the agreement also called for working in three broad SO areas, the development hypothesis was that small grassroots programs with intimate knowledge of local context would be instrumental to accomplishing USG foreign policy objectives in the region. The current grant agreement explicitly states that EF was created to deliver this bottom-up approach, which would complement the work of other bilateral and multilateral organizations.33,34

EPF, EEF-Ukraine, EFCA FNE, EEF-Ukraine, EFCA 31 FNE and Azerbaijan for CS and “democracy” programs 32 Although it should be noted that all three SOs show a general downward trend in grant investment overall. 33 “Experience of the past ten years strongly suggests that the West should increase its engagement in the region— although not necessarily the size of its financial commitment. And, most importantly, it should re-orient its support to focus on bottom-up, grassroots activity (emphasis original).” 34 Blue, Richard, et al. “Final Report: Eurasia Foundation Evaluation.” Nathan Associates. January 2001. 30

However, the grassroots approach creates challenges in measuring success and impact, particularly at the national level. Given that the EF network awarded close to 3,000 grants and implemented numerous direct programs in three broad SO areas in 12 countries, demonstrating a linkage between the activities of EF and the SO areas on a national scale is particularly problematic. There are examples of individual programs that sought to achieve a national level goal, particularly in the PA reform area, but the evaluation did not have the resources to collect data that would link numerous grants and programs to a specific national level indicator. Thus, in the context of such an expansive project focusing on broad SOs and emphasizing a grassroots-level approach, the evaluation team sought to more manageably determine criteria for measuring the advancement of the SOs by grants, flagship programs, and current programs separately: 

With respect to the grant portfolio, the team decided to make a judgment about SO advancement by first assessing whether EF did what was mandated by the grant agreement: investing at a grassroots level. Second, the team considered whether, according to key informants’ own understanding of the success of activities funded by these grants, a preponderance of evidence suggested that grant-funded activities had been successful. At the individual grant level, evidence suggests that grants have largely been useful in advancing the goals of grantee organizations.



With respect to the flagship programs, the team-based judgments on whether the programs achieved the purpose or goals prescribed for each one in relevant documentation. The purpose of each program and to which SO it aligns are listed and described in more depth in the findings below. A secondary consideration in judging success is whether the program has achieved or is expected to achieve sustainability.



For current programs of each PF, similar to the measurement used for grants, the team again considered whether, according to key informants’ own understanding of the success of activities funded by these PFs, a preponderance of evidence suggested that programfunded activities had been successful in terms of the relevance of programs to at least one USAID SO and with regard to identifiable program outcomes. In this case, the team also asked PF leadership to quantitatively rate the impact and effectiveness (taken together) of programs on a scale of 1-5 so that the team could compare the ratings across SOs and PFs.

To collect evidence of success, the team interviewed PF staff, donors and beneficiaries; read evaluations of flagship programs and programs; reviewed progress in meeting indicators established by projects; studied PF presentations and documentation on how they assess success; and conducted an electronic survey of grant beneficiaries. There are also important data limitations to this effort. The team only reviewed a fraction of programs, including primarily current or recent activities, and EFs and PFs suggested many of the programs that were assessed and key informants that were interviewed. As noted, the team also allowed interviewees to use their own understanding of the terms “success,” “impact,” and “effective” when responding to all questions regarding advancement of the SOs. Grant Portfolio

The grants program disbursed a large number of small grants to hundreds of organizations in the region, as envisioned by the grant agreement, with the goal of building the most promising institutions.35 Between 2001 and 2011, 2,456 grants were made, totaling a little over $61 million (or 41% of the total amount of the $147 million agreement) with an average grant size of about $25,000. Based on EF self-reporting, since 2002, the grants program has cumulatively reached more than 1.5 million individual beneficiaries and 113,000 organizational beneficiaries. It has trained over 322,000 people, leveraged more than $26 million in contributions (much of this inkind) from counterparts, and created more than 11,000 new jobs.36 The grant agreement instructed EF to make many small grants; they did so and have an extensive reach based on the number of people and organizations that were beneficiaries, as noted above. The evaluators’ beneficiary survey revealed that 80% of grantees that responded (40 of 50 respondents) felt that they had been either “successful” or “very successful” in working toward their mission; 69.8% (37 of 53 respondents) felt that EF had helped them build their organizational capacity; and 79% (42 of 53 respondents) would reapply for additional grants from EF. Grantee responses here may of course be biased toward positive comments regarding a funding source, but since (a) most grantees have derived funding from other sources in addition to the EF and (b) grants have been quite consistently awarded and monitored with high levels of attention and professionalism, this pattern of positive responses is indicative of authentic, substantive contributions to the effectiveness of grantee programs. Again, we cannot fully test the development hypothesis that this would translate into national level impact on reform. However, as noted in the previous section, the focus of the grants program evolved over time to focus on CS areas, and PFs noted that they felt they had greater success in CS than in the other SO areas. In triangulated interviews with PFs, donors, and beneficiaries, a consistent finding is that the PFs contributed to CS development in their countries; PFs in the Caucasus and Central Asia, as well as the primary donor in Moldova, said that their organizations have been instrumental in establishing or building the capacity of the strongest NGOs in the country.37 For example, in Armenia, the PF stated that of 150 active NGOs in the country, they have supported the establishment of 25-30 strong and sustainable NGOs, and Azerbaijan said there is hardly an active NGO in the country they have not supported. Flagship Programs As noted above, there are three flagship programs aligned to the three SO areas: the EERC to PA, the SBLP to PE, and the MVF to CS. EERC: After a $1.2 million investment by EF-DC, in 2003 the program spun off into three components: KSE, Kiev Research Center, and Moscow-based Research Network. Bannock Consulting conducted an assessment of the program’s performance in 2003. The overarching 35

As described on page 10 of the grant agreement, “[EF’s] purpose is to support programs at the grassroots level designed to help the citizens of Eurasia build local institutions that will serve their own best political, social, and economic interests….the majority of the foundation’s grants are awarded in response to needs identified and solutions presented by local institutions.” 36 Information taken from Eurasia Foundation Annual Reports. 37 Interviews with PF management in Azerbaijan, Armenia, Central Asia and SIDA in Moldova.

conclusion of the assessment was that EERC Ukraine had been very successful as a “delivery mechanism” for masters-level economists trained in Western techniques. EERC succeeded to some degree in influencing policy in as much as many of the participants engaged in policyrelevant work. Initial expectations that many of its graduates would find employment in government have not been met but largely for reasons outside of the control of EERC.38 With respect to sustainability, it was noted that EERC faced challenges. These admittedly dated findings were echoed in interviews with current staff. In Ukraine, staff hoped the program would influence the way economics was taught in other universities and provide a stream of highly qualified economists. The aim of the program was in effect to support public policy. Today the school has turned out over 500 graduates and 20% have gone on to complete their Ph.D in a Western university. However, staff stated that the school has had no discernible impact on policy and no graduates are currently in key positions in government.39 With respect to sustainability, KSE is donor-dependent and donors are leaving. Staff interviewed at the school are fairly confident about funding for the next two years but expressed doubts after that. They also lamented the fact that they missed earlier opportunities to work on sustainability (e.g., buying a building in the early years when money was plentiful). All interviewed staff characterized the school as somewhat successful. In Russia, the Bannock Consultants concluded, “[F]ew, if any, research grant competitions anywhere in the world approach EERC’s level of efficiency and transparency. The online presence at www.eerc.ru is world class, and is as imaginative and well executed as that of any other research network or policy research institute in the world.” However, EERC could not be viewed as a success unless it was able to demonstrate that it could contribute to the policy process. They concluded that it is too soon to judge whether these policy initiatives will be successful. SBLP: In Armenia, the program received co-funding from the Izmirlian Foundation in 1999 and ceased to be an EF program in 2004; the Izmirlian-Eurasia Universal Credit Company continued to operate the program until Araratbank purchased it in 2009. The program in Ukraine also ended in 2004. According to the EF fact sheet, the SBLP accomplished the following: In Armenia 

Loan amounts reached up to $125,000 at a rate of 15% and the loan period could be up to four years.



296 loans were made totaling $10,475,749 with an average loan size of $35,400.



Recipient businesses created 2,219 new jobs and loan losses equaled less than 2% of total disbursals.

In Ukraine 

132 loans were made totaling $6,803,137 with an average loan size of $51,500.

Government employment was found to be unattractive to EERC graduates for reasons such as, for example: minimal salaries, outdated management styles, and lack for innovation in decision making, and the absence of a policy for hiring graduates with Western education. 39 Although it should be noted that there have been KSE graduates that have served in government and policy positions in the past.



Recipient businesses created 828 new jobs and loan losses equaled 1% of total disbursals.

The EF-DC manager characterizes both programs as “highly successful,” emphasizing that they worked through local banks and provided training in small business loan management. The loans also had a longer maturity than those available from other sources because EF provided longterm capital to the banks. The programs were successful in disbursing funds and had very low loss rates. Both proved to be sustainable: In Ukraine the three banks continued to make similar loans after the end of the program and, in Armenia, the Izmirlian-Eurasia Universal Credit Company continued lending. A weakness of the SBLP is that the loans were made at below-market rates and denominated in hard currency. Subsidized interest lending is discouraged by USAID policy as is hard currency lending where borrowers assume the foreign exchange risk. While the programs were successful in helping clients get access to much-needed capital, the aggregate amounts were miniscule given the size of the economies concerned. In Ukraine, for example, total loans made under the program were less than $1 million a year. Total domestic credit to the private sector was $7.8 billion in 2002 and increased to $47.5 billion by 2006. MVF: An FNE key informant notes that the program is scheduled to run until 2015 but has already accomplished a great deal. USAID/Russia DG office views the current programs as more effective than the predecessor program, stating that the program’s major accomplishments have been supporting the Alliance of Regional Independent Publishers (ARIP) and establishing professional standards of journalism/newspapers. The national government has tried to suppress their work through claims of libel, defamation, etc., which USAID believes may be considered a sign of its success and impact. USAID is pessimistic that the individuals currently benefitting from the project will be able to continue their collaboration with EF as an implementing partner. AIRP staff believes that their members are the most developed in Russia, with high quality content and solid design, but they feel uncertainty about their operations over the next three to five years. No Russian funders want to support this type of controversial program in independent media. In summary, EF, USAID, and beneficiaries regard this CS program as highly successful but believe its continuation is uncertain due to the current hostile political environment. PF Portfolios In almost all cases the PFs felt they had the greatest impact and were most effective in addressing CS issues. PF management was asked to evaluate their impact and effectiveness in each of the three SO areas on a 1-5 scale, with 5 being the highest. While there was some variation among the countries that responded to the question, CS was most consistently rated highest.

Figure 4: Senior management self-assessment of impact and effectiveness of program activities in USAID SO areas (1-5 scale with 5 highest)40

Country Armenia Azerbaijan41 Georgia Kyrgyzstan Moldova Russia* Tajikistan Ukraine

Private Enterprise 4 N/A 2 4 N/A

Civil Society Development 5 N/A 4 5 5

Public Administration 5 N/A 2 3 N/A

3 4

3.5 5

4 4

* Question not covered during this interview as discussion of unexpected events took precedence and the interviewers ran out of time.

Furthermore, during interviews the PFs indicated more limited success in the two SO areas other than CS; while there are success stories in the PE and PA areas, the PFs felt they were having greater impact in the CS area. The PFs also consistently described their missions in CS terms and feel they have core competencies in this area. The evaluators noted during their interviews that even when PFs were engaged in PE and PA activities, their focus was often on working with CS partners to bring about changes in these SOs. In other words, their “tool of choice” was to work with CS to achieve results in PE or PA. The fact that many PFs see themselves as CS oriented further reinforces that this is where they feel they have greater impact. Again, there are some exceptions, such as PA work in Moldova and a clear evolution away from CS work in Russia, but most PFs believe they are strongest in CS. Another common theme from many of those interviewed was that programs were successful in the locations where they were active but that resources were not available to scale up and replicate at a national level. In the larger countries, such as Ukraine and Russia, this was particularly true.42 Moldova is an exception in this area and has a significant working relationship with the national government, including work on elections and establishing the National Participatory Council, which engages civil society in the review of laws. EEF-Moldova may be the leading example of a PF’s ability to work in PA at the national level, as many other PFs noted specific difficulties in accomplishing this.43

CONCLUSIONS: EF’S LEVEL OF SUCCESS WITHIN EACH SO

For Central Asia, the team covered Kyrgyzstan and Tajikistan as part of the evaluation. Thus, Uzbekistan, Turkmenistan, and Kazakhstan were not part of the evaluation. 41 Baku staff did not provide numerical ratings but indicated that CS was strongest followed by PA and PE. 42 Ukraine: Senior staff noted the need to consider the size of the country; energy efficiency program highly successful but no funds to go to scale. In Russia both vocational education and a small business program are regarded as successful but lack resources to have impact at country level. 43 Russia, EFCA, EPF

â&#x20AC;˘

EF programs were more successful in CS, followed by PE, with PA as the least successful across the three SO areas.44 o Although difficult to quantify, the majority of interviewees, including PFs, beneficiaries, and donors, felt that the grant program managed by EF had an impact, meaning an implicit indicator of success, in developing CS and supported numerous organizations and institutions. There was a clear shift to focus on CS over time, as noted above, and PFs noted they felt this was an area where they could have a definitive impact, particularly in relation to PE and PA, for which some PFs indicated that they were less effective due to a range of reasons discussed in the next section. o Of the flagship programs, the EERC was aligned to PA, the SBLP to PE, and the MVF to CS. All were seen as successful in achieving specific objectives, but only the SBLP achieved its main goal and can be considered sustainable. The EERC has continued to contribute to improved research and education standards and training graduate level economists in Western techniques, its main goal, but did not affect national level policy, and sustainability is unlikely as donors disappear. The SBLP was successful in achieving its main purpose and in subsidiary objectives. The SBLP can be considered sustainable since loans continued to be made after program funding ceased. The MVF program accomplished significant results and continues to support its main goal of bolstering independent media in the country and cultivating free press. However, given that the situation in Russia is problematic for the programâ&#x20AC;&#x2122;s future, its sustainability is not ensured. o With regard to current portfolios of programs, the PFs, donors, and beneficiaries consistently cited a greater level of success in the CS area. This included areas of local organizational capacity building, in particular, and some PFs could specifically note that they were instrumental in building capacity of local organizations and developing civil society as a concept, which was new in many of these countries. They most often describe current missions and programmatic priorities in CS terms. This is not to say that there are not examples of strong work in PA and PE, but these are more the exception than the rule. The PFs felt they were less successful in PA and PE.

WERE THERE ANY CONTEXTUAL FACTORS THAT FACILITATED OR HINDERED THE ACHIEVEMENT OF EACH SO?

As previously noted, the evaluators did not rigorously test for national level impact; they did not explicitly ask about national level impact in their interview protocol, trace causal logic chains, or run an impact evaluation. However, in nearly all interviews, if the KI spoke to national level issues while being interviewed about a particular project, the team would follow up by asking whether the project had any national level impact. This was particularly true with issues surrounding PA. Furthermore, and more importantly, the evaluators did consistently ask whether the project was successful and in what areas the PF was most effective, which oftentimes led to provision of information on impact. The evaluators also reviewed PF documentation on impact. Following these methods, the team did not identify impacts at the national level.

The factors that appeared to encourage or inhibit success of EF grants, EF FAPs, and PF programs can be divided into three main categories: (1) General Competencies; (2) Country Context; and (3) Organizational Capacity. In some cases, these are aligned to specific SOs but for others the findings are more general. General Competencies Capacity Building (CS): PF staff consistently identified their strength in focusing on capacity building of local organizations as a key success factor in CS programs. Many PFs felt they had contributed significantly to developing civil society in their countries, and staff frequently stated that their expertise in capacity building gives the PF a comparative advantage over competitors in the CS space. Donors that selected PFs in competitive awards reiterated this theme, noting that PFs were selected due to their strength in capacity building and working with local partners.45 The strengths in capacity building were evident in several programs reviewed by the evaluation team, in which beneficiaries and donors frequently emphasized that PFs integrated capacitybuilding efforts into implementation (Kyrgyzstan, Tajikistan, and Ukraine).46 Beneficiaries also mentioned receiving technical assistance in a range of areas, including technical writing (e.g., journalism standards), organizational procedures (e.g., accounting) and fundraising (e.g., working with a PF to prepare a proposal). Conversely, in programs that were described as less successful by donors and partners, interviewees noted that EF did not sufficiently incorporate capacity-building elements, even if other program output indicators were reached.47 Partnership Approach (CS): Another success factor frequently cited by PF staff, donors, and beneficiaries in CAR, Ukraine, and Moldova is the emphasis on treating local entities as partners rather than solely as beneficiaries. In CAR and Ukraine, many of the successful programs reviewed by the team involved collaboration and coalitions with numerous stakeholders. This partnership approach included conducting local needs assessments and facilitating sessions to jointly design projects. In Youth Bank and vulnerable population and conflict resolution programs, small-scale efforts were often designed to address a specific need identified by the local community. In three projects in Moldova (anti-corruption, elections, and National Participation Council), this was institutionalized through a Secretary for Coalitions that was created to build and leverage partnerships. Local Knowledge and Networks (CS and PA): Another factor that has contributed to the success of PF programs is their extensive knowledge and networks at the local level, including the local NGO landscape and community needs. PF leadership consistently identified this as a reason for success and as a competitive advantage in winning other donor projects in both the CS and PA areas.48 In some cases, PF leadership also described community-based projects as a niche, noting that there are fewer competitors doing this work compared to national-level projects. This community-focused approach allowed PFs to build strong relationships with local NGOs and organizations in CS as well as with local government officials in PA programs. EFCA PFs 45

USAID Kyrgyzstan for Cross Border and Educational Loans, USAID Ukraine, USAID Moldova EFCA â&#x20AC;&#x201C; Professional Youth Journalism Development Program, CANS (although late), AmCham Tajik, UNITER 47 EFCA â&#x20AC;&#x201C; Education Loan Program and Cross Border Youth; FNE donor for N Caucasus Conflict Mitigation, Donor in Moldova said EEF does too much themselves rather than teaching NGOs how to do it . 48 PF leadership interviews in CAR, Caucasus, Ukraine and Russia. 46

noted, for example, that support from local officials was important to success in several programs, including a capacity-building effort with local service providers and a youth outreach program facilitated by local schools. In Ukraine, donors cited the EEF’s relationships with local officials as critical to the success of an energy efficiency and e-Governance project. Both Armenia and Moldova staff noted the importance of support from municipal officials in their Youth Bank project. Country Context The composite picture for the region shows that PE, CS, and PA indicators in EF countries are well below those of the countries of Central and Eastern Europe that have graduated from the USG assistance program. In all three areas, there has been significant backsliding during the grant period. While there are important differences in these indicators among the countries in which EF operated, overall EF was fighting an uphill battle in the three SO areas.49 Democratic and Political Conditions (PA and CS): Conditions for democracy and civil society presented both challenges and opportunities for PF activities and influenced the nature and success of PF programs. PFs in all regions noted that corruption and the degree of democracy had implications for both CS and PA efforts. Several PFs cited challenges to working in PA due to suspicion of international organizations as well as local NGOs. In Tajikistan, for example, EFCA meets quarterly, on average, with the Ministries of Foreign Affairs’ Department of International Organizations to explain their activities. They noted that the government is suspicious of all NGOs, particularly those with US funding, and that following the Arab Spring conditions seemed to worsen. Azerbaijan noted similar issues although they believe their transition to a local organization has facilitated their ability to work with the government since they are no longer perceived as a US organization. While Kyrgyzstan is a far more democratic environment, the PF emphasized that political instability hinders them from working with the national government on a sustained basis. In Ukraine beneficiaries of the e-Governance project said that turnover at the local government level has also created challenges. Media programs are also affected by country conditions. EFCA staff stated that the Central Asia News Service (CANS), which sought to increase the quality and quantity of news on the region, was successful in Kyrgyzstan (and to a lesser extent Tajikistan) but encountered significant challenges in covering Uzbekistan and Kazakhstan due to the more restrictive media environments in those countries.50 In Russia, the independent media program faces pressure from the government, including claims of libel. FNE staff, USAID, and beneficiaries regard the program as very successful but are concerned with its sustainability given the need to have private sector funding, which is increasingly unlikely due to political factors. Furthermore, in the CS space, PFs have taken on extremely difficult issues and programs such as cross border and conflict resolution efforts that are clearly driven by the political environment. While donors praised PFs for not shying away from tough issues, they also said that these programs – specifically CAR Cross Border Youth and the Caucasus Media Program – did not 49

Annex F contains specific data regarding country performance with respect to private enterprise development, civil society, and public administration based on the E&E Bureau’s MCP reporting. 50 EFCA staff in both countries and beneficiaries.

meet expectations. EFP staff noted that external factors could very quickly destroy the work undertaken by their cross border projects. NGO Landscape (CS): One issue that clearly influenced the success of programs was the collective capacity of local NGOs at the onset and partners and the operating environment for NGOs. In this regard, the evaluation found significant diversity across the region. In CAR, for example, the PF staff in Kyrgyzstan noted benefits from a rich NGO environment, while in neighboring Tajikistan, NGO capacity is very low and this creates numerous challenges for finding local partners. Despite the more open society, EEF-Moldova also cited challenges with partners lacking good governance and management practices. Staff in Armenia and Tajikistan also noted that local NGOs engage in non-transparent behavior and nepotism, further frustrating success in reaching objectives. Organizational Factors Reputation as an International Organization: While the PFsâ&#x20AC;&#x2122; US-affiliation has in some cases hindered their ability to work with national governments, the PFs are widely recognized as having strong financial, program management, and grantmaking capabilities that make them appealing to donors and local partners. PF leadership also noted that competitor organizations do not have the procedures and policies in place that PFs have from EF-DC.51 In addition, PF staff and donors noted that they are perceived as an independent, non-political organization. Thus, there is a general reputational element that benefits them not only in winning work but also in being seen as a trusted partner by local entities.52 This is not directly aligned to an SO area but would clearly benefit work in the CS area as well as PA at the local level. Donors and PF leadership also emphasized that having a governance structure with independent boards is a differentiator and enhances their programmatic capabilities.53 During the transition to sustainability, PF Boards continue to provide guidance to ensure that the organizations focus on core competencies and remain mission driven.54 However, a main concern during some PF interviews was the ability of PFs to do this as they compete for donor funds.55 Staff Continuity and Skill Areas: One area that clearly influences program success relates to staff continuity and skills. In Central Asia, turnover is a major issue where the donor-rich environment creates higher-paying opportunities. In Kyrgyzstan, there has been significant staff turnover, including the recent change in Executive Director. In Bishkek, one program had two different program managers and three different IT staff tasked to create a website. In the Osh office, the three primary staff members have been there less than one year and one program had three project managers in a two-year period. Country programs suffered from delays in delivery, lack of responsiveness to USAID and partners, and some issues complying with USAID regulations.56 Tajikistan also suffered from a fraud situation that led to the dismissal of five staff

Other donors in Azerbaijan cited their trust of financial procedures as a reason to select them â&#x20AC;&#x201C; although this is not about programmatic success. 52 CAR, Azerbaijan, Ukraine, Moldova staff. 53 Ehrlich and Tim in CAR, Ukraine. 54 Jeff and Tim in CAR. 55 EFCA, EEF-Moldova. 56 According to an interviewed Agreement Officerâ&#x20AC;&#x2122;s Representative.

members and resulted in delays in implementation and a loss of technical capability (although EFCA was praised for the transparent way in which this matter was handled).57 However, in Ukraine, Russia, Moldova, and the Caucasus there is greater longevity in staff and various staff members commented that they have stayed because they are committed to the mission of their PF.58 Furthermore, with regard to the longer running flagship programs, EF staff for both SBLP and MVF cited staff skill and longevity as factors for success in those programs.59 Beyond the issue of turnover, the technical skills of staff are cited as both strengths and weaknesses in specific programs. There are clear areas where the PFs have developed technical skills and have demonstrated core competencies throughout the region. In Central Asia and the Caucasus, partners and donors stated that the PF staff had developed strong skills in youth, media, cross border dialogue, and conflict resolution. In Russia, PF staff, donors, and beneficiaries commented that the success of education and media programs, in particular, are due to having very strong technical expertise in these areas.60 Donors also commented on the strengths of the staff in implementing a range of projects in Ukraine, Azerbaijan, Armenia, and Tajikistan.61 However, in programs that were seen as less successful, the lack of technical knowledge by PF staff was sometimes noted. Interviewees said that PF staff have appropriate technical skills in grant management and a good understanding of important issues in SOs areas but lack deep knowledge in some technical areas. This included the Education Loan Program and some Youth Banks in Central Asia in which donors noted that the PF staff did not possess the right technical skills. In Armenia, one grantee working in media noted that EPF lacked the ability to understand and assist with challenges unique to that field.

CONCLUSIONS: CONTEXTUAL FACTORS THAT FACILITATED OR HINDERED SUCCESS •

The organizations’ focus on capacity building of local NGOs/organizations, emphasis on working in partnership with beneficiaries, and extensive local networks and knowledge contributed to success in CS and, to a lesser extent, PA. This was recognized by the PFs as well as donors and in some cases beneficiaries. PFs have developed a niche for working at the local level in some countries.

•

Country context, particularly democratic conditions and civil society landscape, had implications for success as well as the overall ability to work in CS and PA areas. In some countries, undemocratic conditions precluded PFs from being able to work effectively in PA at the national level. Issues such as conflict, corruption, and political instability also created challenges for working in PA at the national level. Furthermore, suspicion of the PFs as US organizations hindered work in both CS and PA areas. Logically, the availability of strong NGOs or other partners in the CS area was cited as a

Meeting with Nigina and Ravshan and PM (Tajikistan); Meeting with Erkin, Osh Office, Bishkek office. Interviews with staff in Ukraine, Armenia, and Azerbaijan. 59 Interviews with EF-DC staff managing flagship programs. 60 Interviews with FNE staff, USAID, Mott (on education), and beneficiaries (on media). 61 Interviews with former USAID staff in Tajikistan; OSCE, USAID and UK Embassy in Ukraine; BP and UK Embassy in Azerbaijan; and USAID, Mott and British Embassy in Russia. 58

determinant of success. In countries with a richer NGO environment, the PFs felt they had more success than in countries where NGOs were weaker. •

From an organizational perspective, the EF network’s reputation as an independent and transparent organization contributes to its ability to build trust with local partners and donors. Their positive reputation allows them to be seen as a trusted partner by local entities, which benefits work in CS area and, to a lesser extent, the PA area.

•

Staff longevity versus turnover can affect both the performance of the PF overall and high turnover clearly has negative consequences on the success of specific programs regardless of the SO alignment. There are cases in which staff turnover created skill deficits that negatively impacted program delivery. There are also cases in which staff did not possess the appropriate technical skills for implementing donor programs regardless of their longevity with the organization.

WHAT STEPS DID EF TAKE TO ENSURE A GENDER BALANCE IN EACH OF THE SOS? The original grant agreement identified seven primary indicators to be tracked by EF, with two gender specific indicators: the number of identifiable participant trainees by gender and number of identifiable beneficiaries by gender. This shows gender balance was largely achieved with females representing 47% of total beneficiaries and 48% of training participants. This data only relates to grants, so as the grantmaking activities declined, this becomes a less relevant indicator for gender balance. However, the indicators are included in Annex E. In addition, specific programs address gender-based issues, and EF continues to track genderrelated metrics beyond grants. An EF-DC document on Women’s Empowerment notes that since 1992, EF has had 115 programs that promote the roles of women, including training and support to women and women-led NGOs. Some particularly successful examples highlighted entrepreneurship training for women in rural Georgia, economic literacy for women in Uzbekistan, and efforts in Azerbaijan and Armenia to increase the role of women in politics. During fieldwork, the evaluation team did not ask questions specific to gender but addressed this when a PF staff member identified this area as a priority. There were clear examples in which gender issues influenced program design and implementation. This is particularly the case in Central Asia where women face several cultural and social challenges. For example, programs with vulnerable populations and involving income generation addressed the needs of women who are able to work only in culturally-accepted areas such as hairdressing and sewing. Training programs and grants have been used to develop vocational capacity for women and girls. The Access to Justice Program in Central Asia provides legal support to women who are dealing with such issues as divorce and domestic violence. This is especially an issue in Tajikistan, where the migration of men to Russia has resulted in severe hardships on women who are left with no sources of income when divorced or abandoned. In addition, the Osh office manages a Women's Peace Bank project that seeks to integrate women's groups into conflict resolution issues. Also in Central Asia, several EFCA staff noted the challenges of securing and retaining female participation in specific programs due to strong societal pressures not to participate.62 For example, in the Youth Journalism and Youth Bank programs, recruiting and retaining ethnic 62

Ravshan Interview on Youth Radicalization program.

Uzbek girls has been an issue.63 In Tajikistan, of the five communities targeted for a youth radicalization program, four of the five are completely male and even if girls begin to participate, they are often encouraged to exit the program. Other PFs also have existing programs that focus on gender-based issues. For example, in the joint EPF-Armenia and EPF-Azerbaijan “Unbiased Media” project, journalists interviewed women to produce series of “Women’s Narratives” in an effort to have women’s voices heard. The Support for Armenia–Turkey Rapprochement (SATR) also engaged women in activities. EPF has developed gender-mainstreaming strategies to incorporate gender equality into all activities, rather than as a separate programmatic area, and conducted gender-mainstreaming training for its staff.

CONCLUSION: STEPS TAKEN TO ENSURE GENDER BALANCE •

The EF Grant Agreement includes indicators that track participants/beneficiaries by gender, but this is less relevant now as a reflection of gender balance since the overall grants program has declined. The evaluators found that EF-DC continues to track genderrelated metrics for all programs. Furthermore, some PFs have programs specific to addressing gender-based issues, particularly relevant in Central Asia, and one PF has a gender mainstreaming policy. As the PFs have evolved into independent organizations, there does not appear to be a consolidated approach to gender.

ISSUE II: HAS EF-DC SUCCEEDED IN ITS EFFORTS TO ESTABLISH AND BUILD SUSTAINABLE FOUNDATIONS THROUGH ITS NETWORK PRIOR TO THE END OF CORE GRANT FUNDING? In early 2003, Amendment 3 to the Core Grant Agreement required EF to establish a network of indigenous entities (i.e., PFs) to carry out programs following the end of USG core funding. EF complied and launched FNE the following year, followed by EFCA in 2005, EPF and EEFUkraine in 2007, and EEF-Moldova in 2010. EF’s 2010 Sustainability Plan added a further layer of complexity to EF’s new operational model, requiring EF-DC to lead the transition of its PFs from grantmaking to operational foundations while at the same time undergoing a similar process requiring internal restructuring, expansion into new program areas, and identifying new sources of funding. This has created a challenging situation in which EF-DC has declining resources to support the PFs’ transitions while also managing its own. This section will analyze the steps taken by EF-DC to build the managerial, operational, and financial sustainability of its PFs under the current grant and provide an assessment of EF-DC’s success in this effort. The evaluators thus defined success as EF-DC fully recognizing the level of resources needed to ensure that the PFs had the human, organizational, and financial resources needed to fully transform their business models to a sustainable approach of achieving their mission and continuing operations beyond the end of USAID direct support.

DOES EACH OF THE PFS HAVE THE MANAGERIAL AND OPERATIONAL CAPACITY 63

Professional Youth Journalism Development Program – Uzbek girls leave the program in Osh but in Tajikistan and Bishkek 90% of participants are girls. In the Youth Banks (Cross Border?) program, the ability to retain girls is difficult due to societal pressures.

TO ENSURE ONGOING PROGRAM GROWTH AND SOPHISTICATION?

Establishment and Early Support Before commenting on EF-DC’s steps to strengthen the managerial and operational capacity of its PFs, the evaluators thought it important to underscore the significant amount of work, and relative success, that EF-DC had in merely establishing these PFs in the first place. EF-DC worked with PF staff and local lawyers to navigate the various host country laws associated with establishing independent foundations. This had the complicating effect of requiring different operational structures to respond to the various host country regulations. For example, in Russia, Ukraine, and Moldova, the PFs are each single registered entities, while the Central Asian and Caucasus PFs are both regional in scope with one overarching name, President, and Board but separate legal registrations in each country. Each structure has its own comparative advantages and disadvantages along with unique challenges to organizational sustainability. For example, while the regional model entails additional bureaucratic and political challenges (such as having to sign official documentation three times or considering the political sensitivities of, for example, having Azeri or Armenians working together), it also offers unique advantages such as strengthening regional knowledge sharing and collaboration, management and operational unification, and programming flexibility. EFCA and EPF noted in interviews that the regional model gives them a competitive advantage in bidding on regional projects (i.e., EFCA’s Equal Before the Law or EPF’s CRRC programs). Managerial Capacity EF has placed high emphasis on the development of the Board of Directors as an oversight institution in the PFs, a US-based approach to foundation governance and management. The host-countries’ NGO and foundation legislation seeks to balance powers among different structures of organization such as the Board of Directors, Advisory Board, and Executive Director. Different functions, such as strategy identification, control, and execution, are distributed respectively among the Board of Directors, Advisory Board, and Executive Director. Governance. While the PFs are under the executive leadership of their respective Presidents, PF governance also features Boards of Trustees made up of high-profile individuals that are often of international stature. These boards vary in how actively engaged they are programmatically and operationally with their respective PFs, but in general the boards do carry out oversight, governance, advisory and fundraising functions in close collaboration with the respective PF Presidents. The EEF board, for example, holds quarterly committee meetings and semi-annual full board meetings. Board committees include Executive (which evaluates the President’s performance), Finance, Nominating and Governance, and Communications committees. Representation on the boards tends to be derived from the local private sector as well as international donor organizations. This can be advantageous for gaining new financial support for programs, but the relative lack of representation from technical specialists or representatives of grantee groups on some boards (such as EEF’s) arguably stands as a limitation to board effectiveness. Strategic Planning. All interviewed PFs indicated that they had developed at least a three-year strategic plan with regional and local partners utilizing different strategic planning approaches. As a rule, a unified strategic plan is developed for a regional office that encompasses three countries (as in Central Asia and Caucasus) and individual strategic plans are developed for 22

organizations that work only in one country (as in Moldova, Ukraine, and Russia). For the most part, the strategic plans cover functional areas such as Program/Financial Management, Board Governance, HR, and Fund Development and Communication. They do not include institutional capacity building as a separate area of the strategy. The strategies are adequately done with rationales provided for strategic objectives and periodic updates with board input. The strategies tend to have been developed in-house, however, with limited contribution or participation by grantees or other local-level partners. Annual operational and financial plans are prepared according to the strategic plans. The evaluation team noted that foundation staff were often more familiar with the annual operational activity plan than with the strategic plan. Operational Capacity Staff Capacity. A majority of the PF staff identified the need to build sufficient in-house technical expertise as a priority for strengthening future operations. EF fashioned its PFs based on its experience as a grantmaker, providing a strong Grants Management System and training a small but dedicated internal staff to oversee the grantmaking process. However, as acknowledged by PF staff, the skillset of a grantmaker and implementer differ: the former needs broader operational knowledge and the latter deeper technical expertise. All together PFs have around 250 staff members, the majority of whom are administrative (office managers, accountants, administrative assistants, logisticians, drivers, etc.). The staff of PFs have a clear assignment of responsibility and tasks, which were developed specifically for grantmaking foundations. Interviewed staff members generally recognized their limited technical knowledge and expertise in specific areas of their work. However, the evaluators found that PFs generally do not have a formal plan for staff development and limited and diminishing core funding will force the PFs to reconsider the staff policy and structure. This will most likely result in overall staff reduction particularly among administrative/support personnel. For instance, in Azerbaijan, most of the 13-14 staff members will have to leave by the end of December 2012 due to project closeouts. Reduced core funding will also lead to the need for local staff training in skills specialization and professional standards and the review of the remuneration policy and types of employment. Some functions “Even though we’re registered locally we’re seen as American(e.g., M&E) may not be affordable based...we’re perceived to have systems and procedures, after core funding ends, and the handbooks, policies, charters, GMS, etc.” ~ EPF staff reduction of core funding has already “Major benefit of EF was that we inherited solid business resulted in less investment in processes and procedures. Seen as having international standards communications, M&E, and the loss – DC is very open and transparent.” ~ FNE staff of the DC-based grants manager. “EF provided EFCA with internal regulations, rules, procedures, and processes…Procedures remain in place and we now have Thus while the PFs are strong in their strong monitoring and evaluation systems. Our GMS is also core competencies – grantmaking, strong. We can stay competitive because of this.” ~ EFCA staff relationship-building with local civil “[EEF-M are] extremely qualified and extremely professional. society partners, and ability to engage We undertook system based audit of them and our auditor said with local governments and that it was one of the best system based audits that they have businesses – there is a need to build ever carried out. I would give them a very, very good grade.” their in-house technical expertise in ~ Sida (donor) representative on EEF-M. order for them to grow as implementing organizations. The PFs appear to be cognizant of this need as evidenced in many

of the slides addressing sustainability in the PF presentations during the latest EF Network meeting in Chisinau, in which PFs suggest a need to “reposition” or “restructure.”64 Program Management Procedures. Interview responses with PF donors and staff consistently emphasized the strong internal systems and procedures inherited from EF as a key operational strength (see textbox right). However, while EF has clearly been successful in transferring sound grants-based procedures and quality standards, feedback from staff convey that further action is needed to support the PFs’ transitions to becoming operational foundations. Twenty-one of 29 PF staff respondents (72%) indicated that current internal systems and procedures will need to be further modified to respond to their specific host-country needs and that their success in doing so will be a major determinant of their future sustainability. Some PFs have already succeeded in adapting their inherited procedures – i.e., “all [procedures] have been adapted to local circumstances. Russian rules and procedures are complex” (FNE) – while other PFs are still in earlier stages – i.e., “our procedures need to be localized because we have things like labor regulations” (EPF). Leveraging the EF Network Another significant step in EF’s efforts to support the sustainability of its PFs was the establishment of the EF Network, described by EF as a “constellation of locally chartered institutions organically linked to the international foundation community.”65 EF-DC is the selfdescribed “hub of the Network, leveraging interaction and cooperation among member foundations, overseeing successful institutional development as well as joint ventures and economies of scale in administrative, training, fundraising, communications, and other specialized services.”66 The evaluators noted that similar programs among the PFs indicate some of the influence of the EF Network but found that opinions on the utility varied significantly among the PFs, particularly according to their various stages of development (see textbox below). Opinions on the Utility of the EF Network Strong Utility: “Network has very important benefits…for example, we leveraged past program experience—as an example, we modeled our Ferghana Valley program after a similar program in the Caucasus.” ~ EFCA-Tajikistan “EF is very helpful. It will be difficult if EF decides to close its offices in DC.” ~ EPF-Georgia Mixed Utility: “We have infrequent interaction with DC—mostly help with the narrative part of proposals—but frequent with regional office.” ~ EPF-Armenia “Finance network has finance network and programs has programs. These mini-networks work better than overall Network.” ~ EEF-Ukraine Little Utility: “We have not been successful in transferring skills and knowledge in Network.” ~ EPF-Azerbaijan “[PFs] have a network, a common past, but very different structures, funding sources, governance principles

EF Network Meeting in Chisinau, Moldova. September 28-29, 2012. “Eurasia Foundation Sustainability Plan: 2010-2014.” Pg. 2. 66 Ibid. Pg. 5. 65

This is perhaps not surprising given the diversity of backgrounds, personalities, and positions of respondents within the PFs. While the opinions on the utility of the overall Network were too varied to draw aggregate conclusions, the evaluators noted that there seemed to be common agreement that the PFs’ relationships with EF-DC were most useful during their earlier stages of development and that their regional networks were more important for identifying present funding opportunities. In describing the advantages of their relationship with EF-DC, most PF staff would point to the establishment of sound policies, procedures, and a functional GMS, and they also mentioned that their current interaction related mostly to reviewing funding proposals. EF’s 2010 Sustainability Plan stated that “the network structure offers economies of scale and supports institutional development and specialized services, such as fundraising, new technology, communications, and evaluation.”67 EF’s 2010-2012 Strategic Plan further detailed that “in the new architecture of the Network, EF-DC’s central purpose is to support the long-term institutional growth and program success” of the PFs.68 Based on the evidence available to the evaluators, it is not clear that EF has met the goals of either document. The evaluators found minimal evidence to suggest that the EF Network produces significant benefits or economies of scale for individual PFs. While some respondents commented that the Network was useful for identifying new donors and funding opportunities, few of these respondents were able to provide specific examples – the exception being EEF-Ukraine, which stated that NEE (Belarus) had built its energy efficiency project based on the Ukrainian model and the Youth Bank model being implemented by most PFs. Others replied more critically: “We would like EF-DC to be more pro-active, help approach different US-based donors,” according to one interviewed staff member of EFP-Armenia. One senior FNE staffer responded most bluntly: “The big question for EF is whether it can continue to approach this region as region? PFs have a network, a common past, but very different structures, funding sources, governance principles and futures….Unless the situation changes [an opportunity for large regional project], the network is not useful or necessary.” There was similarly little evidence to support economies of scale in the provision of specialized services such as technology, communications, and evaluation. Of the three, efficiencies in communication are the most significant benefit with EF-DC producing good quality semi-annual reports and collecting individual “success stories” to share with potential donors. A few respondents (less than five) cited examples of EF-DC staff providing trainings in design, monitoring, and evaluation techniques or helping to conduct final program evaluations. The only technological benefit cited by the respondents was the provision of the GMS, which, although highly praised, was primarily useful for managing PF grants – not their projects and programs.69 Regarding EF-DC’s role in supporting the PFs’ long-term institutional sustainability and program success, much remains to be seen. The following section describes the financial situations and challenges of each PF – for most, their future is far from secure. The evaluators also did not find substantial evidence of EF-DC improving the implementation of PF projects. The evaluators were told that there were instances in which EF-DC would visit to evaluate the 67

Ibid. Pg. 5. “Eurasia Foundation Business Plan: 2010-2012.” Pg. 1. 69 EEF-Moldova is the exception and claims that the GMSME system is a useful tool for managing both grants and operational programs. 68

achievements but a couple of respondents (both from EFCA) commented that they were never informed of the results of these evaluations.

CONCLUSIONS: STEPS TAKEN TO BUILD PF’S MANAGERIAL AND OPERATIONAL CAPACITY 

Managerial Capacity. EF took considerable steps to improve the managerial capacity of its PFs. EF has placed high emphasis on the development of the Board of Directors as an oversight institution in the PFs, and the team found that these boards generally do carry out their oversight, governance, advisory, and fund-raising functions as expected. However, the team also found that the sustainability of the PFs is largely determined by the immediate leadership of their Executive Director/President. Here EF-DC’s success was mixed with some PFs demonstrating strong leadership while other PFs demonstrated less inspired leadership.



Operational Capacity. PFs staff have demonstrated core capacities as strong grantmakers but need to build their depth of expertise in the technical areas in which they manage projects. PFs face a considerable dual challenge of needing to building technical expertise while at the same time responding to the additional administrative burdens caused by the anticipated loss of administrative staff due to decreases in core funding. The PFs have also inherited strong grants management systems and procedures from EF but still need to adapt their operational procedures to better respond to individual country contexts.



Leveraging the EF Network. The EF Network is of greatest benefit to the least developed PFs within the Network. It has provided PFs with core program management systems and procedures and also helped identify early opportunities for outside funding. However, as PFs mature, the importance of EF-DC decreases and smaller, more regionally focused networks provide greater benefit.

WHAT STEPS HAS EF-DC TAKEN TO ENSURE THE FINANCIAL SUSTAINABILITY OF ITS NETWORK BEYOND THE END OF “As the Eurasia Foundation seeks to diversify its CORE GRANT FUNDING? funding sources and further leverage USG funds by The Grant Agreement stipulated that EF would be sustained through an endowment and could continue to rely on an allocation of funds from USAID (see textbox). Based on interviews with relevant EF staff, board members, and EUR-ACE representatives,70 discussions began in 2004 with respect to EF transitioning to financial sustainability, defined as the ability to continue operations without additional core funding from

attracting non-USG financial support, an effort has been launched to obtain an endowment or trust consisting of matching public and private funds. An endowed Eurasia Foundation will ensure that the USG remains engaged with the former Soviet Union in a way that reduces political exposure and is in line with calls by the US Congress to modify its operating practices around the world. Until the endowment/trust has been assembled, the Eurasia Foundation would continue to depend primarily on annual allocations from USAID under this Grant, as well as expected significant funding from other sources” ~ Core Grant Agreement, Section G: Endowment/Trust

EUR-ACE is the Department of State's Office of the Coordinator of US Assistance to Europe and Eurasia oversees the bilateral economic, security, democracy, and humanitarian assistance of all USG agencies providing assistance to 18 states of the former Soviet Union and Eastern Europe.

USAID. At that time, EUR-ACE felt that with declining assistance to the region, EF needed to transition away from reliance on USG core funding. Initially this was not seen as a total phase-out, but rather a reduction. This led to the sustainability question: EUR-ACE agreed to commit a certain amount of funding over three years and in return EF would generate its own resources. The first example was the FNE in Russia, which was registered as a Russian organization with the same basic charter as EF. In 2006, the decision was made to go forward with establishing other legacy organizations. EURACE provided letters to EF committing to funding levels and allowed core funding to be used for the purpose of building organizational capacity necessary to achieve this objective. In 2009, the last commitment letter was issued covering Ukraine and Moldova. EF-DC, however, wanted an endowment, as envisioned in the grant agreement, or an annual line item budget similar to that of other regional foundations. They were optimistic they could get an authorization bill in Congress but wanted EUR-ACE to provide bridge funding. EF met with the Undersecretary and Congress and started an advocacy effort, arguing they did not want to become “just another NGO hustling on the street.”71 EF’s relationship with EUR-ACE was somewhat contentious at times.72 Ultimately, EF did not succeed in getting a line item and the core budget ends December 2013. In June 2010, after six months of vigorous negotiation, EUR-ACE, USAID, and EF agreed to a Sustainability Plan that established specific fundraising targets for EF-DC and the five PFs. The release of further core funding was conditional on making satisfactory progress in meeting these benchmarks, which included: 

EF-DC will raise no less than $8m in non-directed funds during the first year (June 2010June 2011). Thereafter, EF-DC will increase the annual funds raised by at least an additional $1.5m a year for a total of $28.5m through June 2013 ($8m+$9.5m+$11m).



Of the non-directed funding expended each year in the four-year period, EF-DC will pass through at least 75% to the local foundations.



Each partner foundation will increase the percentage of its own budget from non-directed sources (not through sub-awards from EF-DC) by an average of 5% annually over the four-year period from the 2010 baseline.

For the first reporting period of June 2010-June 2011, almost all targets were met or exceeded. However, performance fell off somewhat in year two (see table below).

71 72

Interview with EUR-ACE. Ibid.

Figure 5: Performance Against Financial Sustainability Targets June 2010 - May 2011 Target

Actual

$ 8,000,000 $ 3,292,515 $ 1,368,343 $ 3,753,357 $ 3,447,352

$ 9,469,465 $ 3,042,835 $ 3,483,853 $ 5,547,426 $ 7,474,087

June 2011 - May 2012 Target

Actual

$ 9,500,000 $ 3,457,141 $ 1,436,760 $ 3,941,025 $ 3,619,719

$ 7,522,580 $ 3,234,984 $ 2,212,068 $ 5,535,752 $ 6,892,902

June 2012 - May 2013 %

Target

Actual

$ 11,000,000 $ 3,629,998 $ 1,508,598 $ 4,138,076 $ 3,800,705

$ 4,424,533

Entity EF-DC Central Asia (EFCA) Ukraine/Moldova (EEF) South Caucasus (EPF) Russia (FNE)

118% 92% 255% 148% 217%

79% 94% 154% 0% 190%

Variations in Financial Sustainability of PFs While the EF network has broadly met their financial sustainability targets to date, there is significant variation among the PFs in their expectations about the future. Furthermore, each PF has taken a different path to sustainability based on country context and available sources of funds. It is also worth noting that some PFs have both a regional headquarters and specific country offices, and there are variations in sustainability among the country offices as well. FNE: was the first PF to be established and is the most advanced in terms of achieving financial sustainability. They currently receive approximately 80% of their funding from Russian sources. Nevertheless, senior staff are “somewhat doubtful” about their continued ability to attract donor funding. This is a reflection of the very difficult and hostile political environment in Russia demonstrated by the new NGO law that requires any organization accepting funds from sources outside Russia to register as a foreign agent. •

FNE made the difficult transition from a grantmaking organization to what they term a “social development agency.” The transition was gradual and painful. No one wanted to give money to FNE because they thought of them as a donor. The old staff was used to spending money rather than earning it and had no background in writing proposals. “There were not too many survivors,” according to the FNE President. Core funds were essential during the transition period.

•

FNE’s strategy is to rely on the ability to sell services to Russian institutions, government, and foundations. They do not have a written business strategy; rather, each of the four operating units has its own fundraising/outreach strategy: “business development is everyone’s business.” It is very difficult to get donors to fund overhead costs in Russia. The current operating budget is $4.5 million, down from $7 million in earlier years. FNE had planned to purchase a building with core funds, which was approved by USAID, but EF decided not to proceed due to the increased risk of operating in Russia. Thus, FNE currently rents office space, which is a significant overhead cost. The FNE finance manager notes that local (Russian) donors are not likely to donate to either the media program or the civil society programs as these programs are seen as US linked and can provoke conflicts with the Russian government. Only big businesses have

40%

the ability to give money and they do not want to antagonize the government. Therefore, the end of core funding may result in a significant shift away from US objectives in program areas. EEF-Moldova: is the newest of the PFs. Senior leadership is “very confident” about continued donor funding. With a staff of 14 and an annual budget of $1.5 million, they have already secured core funding from SIDA to cover the next four years, which includes overhead costs. Unlike FNE, the end of US core funding has not precipitated a wrenching transformation. They have been able to substitute one donor for another. They have a written business strategy that runs through 2012 and are in the process of writing a new one for 2013-2016. EEF-Ukraine: Senior staff report that they are “very confident” about future donor funding while the President notes that it is a very competitive environment and that he expects they will experience a budget decrease. The budget for FY 2013 is $1.4 million, a 13% reduction from 2012. EEF-Ukraine is in the process of purchasing office space with USAID core funding, which will contribute significantly to future sustainability by reducing operational costs. Their strategy is to focus on larger donors like USAID and the EU. They do have a written fundraising strategy/business plan and regard business development as everyone’s responsibility. The President notes that generating money for overhead costs is extremely difficult, while the finance manager says it can be done, but with difficulty. They note the end of core funding is also difficult because many donors want to know that there is a co-funder and the absence of core funds will make this difficult. A knowledgeable board member notes that EEF-U has tried to approach local private businesses, but this has been very difficult because they are owned by oligarchs who are also involved in politics with their own foundations and own programs – and this makes working with them too political. He notes that with core funding ending, EEF-U will have to downscale significantly and/ or change their program strategy. EPF: The three country programs, Azerbaijan, Armenia, and Georgia, are grouped together as part of a regional foundation, based in Georgia, which creates advantages in terms of opportunities to bid on regional activities and programs that advance dialogue between Armenia and Azerbaijan. However, the financial circumstances of each country are quite different. The EPF President and Finance Manager are “very confident” with respect to future funding as is the Georgia Country Director. The Azerbaijan Country Director is “fairly confident” while the Armenia Country Director is “somewhat doubtful.” There are about 100 total staff members in all three programs, with a 2012 total budget of $8.4 million ($2.5 Armenia, $1.7 Azerbaijan, and $4.2 Georgia). The total budget for 2013 is $4.5 million, approximately 45% less than 2012. This is due to major fundraising issues in Armenia and Azerbaijan. The Georgia and Armenia programs are looking to purchase offices with USAID core funding, which again will contribute greatly to sustainability by lowering overhead costs. The biggest challenges cited by EPF staff are difficulty procuring funding (donor fatigue and no respectable local oligarchs/donors); corporate funders refusing to pay for overhead costs; and the need to restructure offices by July 2013 when core funding ends. Armenia faces perhaps the most significant challenges. Transitioning from a grantmaking foundation into a program implementation organization has meant triple the work for program managers: implementing programs, managing grants, and fundraising. This has created a substantial amount of stress in the office. In addition, they are downsizing to a smaller building and salaries will drop. They currently have funding for two people for another six months and

then for two people part-time after June 2013. Without core funding they will survive, but they will be a completely different type of organization.73 They also note that they have been prevented from competing for USAID grants reserved for local organizations because their board structure, with international representation, does not conform to USAID Forward guidelines. The situation in Azerbaijan is similar. They are currently undergoing a shift from grantmaking to running programs. The EPF-Azerbaijan Executive Director notes that the current staffing situation is “a catastrophe.” Leadership must let people go because contracts are ending and they do not have project work and finances to sustain them. Georgia on the other hand is very confident and is looking to USAID and other foreign government assistance programs for funding. They have a written business strategy in place and are able to cover overhead costs, although this does not include funds for the regional office. The regional office itself, comprised of the EPF President and Finance Manager, looks vulnerable with the end of core funding. One of the country managers was clear that they could not get donors to pay for local office overhead or the overhead associated with the regional office. EFCA: Kazakhstan, Kyrgyzstan, and Tajikistan comprise EFCA. The foundation has 72 staff and a combined budget of $4.6 million. The regional office in Almaty is important to allow them to bid on regional projects, and there several currently under implementation. The evaluation team interviewed the EFCA President and five staff members, all of whom are “very confident” or “somewhat confidant” about EFCA’s ability to secure future funding. However, while Kazakhstan is distinct in that it raises substantial funds from the private sector, and thus may have at one time had greater advantages in fundraising due to the oil and gas sector, the costs in Almaty are extremely high. A Former EFCA President noted that while initially concerns were that Kyrgyzstan and Tajikistan may have to close, now they appear stronger in terms of fundraising and Kazakhstan is the weakest of the three. Other major challenges include controlling high staff turnover and keeping quality high and costs low. The Executive Director for Kyrgyzstan notes they have a three-year EFCA strategy that targets multiple funding sources. They are usually able to cover overhead costs in their proposals and EFCA created an overhead rate of 7% that has been accepted by most donors but not USAID, which is noted as extremely beneficial by current EFCA staff. In Tajikistan, the PF did not win US-funded grant competitions reserved for local institutions because of the interpretation that they are an international organization. This is a different issue than in the South Caucus where the board structure does preclude them from consideration as a local organization under USAID Forward guidelines. However, in Tajikistan this appears to be a perception issue and was noted by other donors as an issue that needs to be addressed. EFCA is planning to hire a PR person and reposition/rebrand and possibly rename the organization to promote their perception as a local entity. EF-DC: It is clear from interviews with DC staff and board members that EF-DC is also fighting to adjust to new realities without core funds. The majority of the board wants to keep DC open, but without core funds, this is a challenge. EF-DC has attracted some small corporate money from oil, law, and accounting firms; however, without core funds, they need to go out and compete (and this could create situations where EF-DC competes with PFs in the future). This is 73

According to a Senior EPF-Armenia staffer.

an entirely different business model and they increasingly find themselves moving from a foundation to a service provider model. After a painful reorganization process last year, EF-DC is now “lean and mean” and is actively expanding into Asia and the Middle East (China and Iran). EF-DC’s fundraising efforts are run by one person, and the DC office is primarily focused on implementation and management issues. This raises the question of whether they have sufficient capacity to pursue a targeted fundraising strategy. The need for greater dedicated resources to fundraising was identified by the evaluation but there is no clear solution since private donations are unlikely to support additional staffing. One board member noted that EF-DC is now a “Beltway bandit” bidder; they have become exactly what they were created not to be and may not be sustainable. While the PFs may survive without EF-DC, it was the flow of core funding though EF-DC to the PFs that gave the USG a voice in programming resources. This will now be diminished.

CONCLUSIONS: STEPS TAKEN TO BUILD PF’S FINANCIAL SUSTAINABILITY 

With the end of core funding, some members of the EF Network are financially at risk. EPF and EF-DC are struggling to make the transition from a foundation model to a service provider. EPF-Azerbaijan and Armenia are facing very difficult periods. Ukraine also faces funding challenges, and while EFCA overall is successfully raising funds, the cost of the Almaty office is problematic.



On the other hand, Russia and Moldova are both financially strong. In Russia where donor funds have dried up, FNE has successfully found local sources of funding including the government and local private firms. For FNE, the risk is more due to the the political challenges of operating in Russia than the end of core funding. Even if core funding were to continue, it is likely that FNE would not accept it following the adoption of the foreign agent law. It is also not clear whether FNE will register, accept, and further foreign funds. While they can raise substantial resources from the Russian private sector, the programming will be quite different than was the case with core funds. In Moldova, a much smaller and poorer country, EEF-Moldova has been able to transition from one donor, USAID, to another, SIDA, with minimal disruption.



In some cases, with the end of core funding, PFs have to undertake a significant program realignment to meet the needs of new donors. This is already happening in Russia and, to a lesser extent, Central Asia. Organizations may find themselves in very different lines of business than those undertaken when funding and program direction came from the USG.



Key EF-DC functions, such as fundraising and M&E, have been lost due to the reduction in core funding. It is critical that the PFs have sufficient resources to cover fundraising/proposal development, which traditionally was covered by core funds.



Financial sustainability is compromised in some cases where the USG has determined that the locally registered EFs are not eligible to compete for funding reserved for local organizations, either because of their board structure, which includes internationals, or their perceived alignment to EF-DC.



The transition to a new business model after core funding can be wrenching, entailing significant downsizing and staffing changes. This is a difficult process best begun early. The evaluators found that FNE has made the most significant strides in this effort, but most other PFs have not really undertaken a transformation to date and this has created gaps in technical capabilities since the Understanding the CMI Tool: nature of grantmaking vs. operational CMI Application Process: foundation has yet to be absorbed or 1. Schedule mapping exercise understood.

IS EF-DC’S CMI SUCCESSFUL IN ACCURATELY MEASURING THE ORGANIZATIONAL CAPACITY OF PFS?

A critical step in EF’s effort to support the sustainability of its PFs was the development and implementation of the CMI tool. CMI was designed in 2008 to help PFs “identify specific areas of organizational growth to invest in, target resources more effectively, and evaluate the impact of [their] collective capacity building efforts.”74 It utilizes a diagnostic approach designed to assist EF and PF staff to evaluate their current capacities and work in partnership to determine a path for growth and measuring achievement along the way. History of Implementation

2. Select capacities to be mapped 3. Collect and analyze background documents 4. Conduct site visit and interviews 5. Draft mapping report 6. Debrief and discuss report with PF management before finalizing 7. Present to Board 8. Create an organizational development plan and monitor achievement of capacity building targets CMI’s Seven Key Areas of Capacity: 1. Board Governance 2. Executive Leadership and Management 3. Financial Management 4. Program Management 5. Fund Development 6. Human Resources and Staff Development 7. Communications

Figure 6 below depicts the history of the CMI’s implementation and highlights that only the two longer-standing PFs have conducted the evaluation three times. It also highlights that EF-DC has never used the CMI tool to evaluate its own capacity. Only six of seven capacities were applied in the CMI applications and there is no evidence that the Leadership and Management review was conducted in any of the PFs or in EF-DC. The most frequently used capacities include Board Governance, Financial Management, and Program Management, which correspond to EF’s Sustainable Plan prepared for 2010-2014. Figure 6: History of implementation for each PF 2007 2008 2009 FNE, EPF, FNE, EFCA, FNE, EFCA Board Governance EFCA EEF-U FNE, EFCA FNE, EPF EEF-U Financial Management FNE, EPF, FNE, EFCA, FNE, EFCA Program Management EFCA EEF-U FNE, EPF, EEF-U Fund Development EFCA FNE, EPF, EEF-U HR and Staff Development EFCA CAPACITY/YEAR

2010

2011

EEF-M

Eurasia Foundation Network’s Capacity Mapping Initiative: A Resource for Building Legacy Foundations Implementation Guide.” Eurasia Foundation. 2008. Pg. 4 and 6.

Communication

Leadership and Management

FNE, EPF, EFCA -

FNE, EFCA, EEF-U -

EEF-M

EF has typically provided funds to support each PF’s first CMI application with the PFs then expected to fund follow-on applications themselves. This may explain the uneven frequency of application witnessed among the PFs. Several respondents commented that although they thought the CMI tool was overall helpful, their PFs did not have enough money (or perhaps did not prioritize the process enough) to fund follow-on applications: “We never had enough money to really use it,” according for one EPF Director. The evaluators were struck that the capacities to be mapped and the people interviewed to gauge those capacities appeared to be determined by a few select employees, usually senior EF and PF staff. This process introduces a significant risk of selection bias with PF staff possibly being inclined to select those staff members likely to provide the highest scores. The evaluators noted that unlike other capacity assessment tools – such as USAID’s “Organizational Capacity Assessment” (OCA) tool, McKinsey’s “Organizational Capacity Assessment Tool” (OCAT), or the MSI’s Institutional Development Framework (IDF) tool – the PFs’ implementation of CMI seemed to interview a small group of people to gauge specific capacities. For example, financial capacity would be scored based on interviews with financial staff solely or communications would be scored based on interviews with the PF director and communications person. The other aforementioned tools take a different approach and ask each respondent to comment individually on a variety of capacities and then bring respondents together for a group discussion.

CONCLUSIONS: IMPLEMENTATION OF CMI 

The frequency of EF’s CMI application and the number of capacities mapped within PFs has varied significantly with lack of funding being the most common explanation.



CMI was to be a process that should be applied regularly in order to verify capacity growth; however, it was not applied consistently and was not used to its full advantage.



The limited application of the CMI did not result in the intended training/learning effect for the PFs. It failed to improve the overall understanding of all staff members in the PFs on the importance of building Diversity of PF Staff Opinions on Usefulness of CMI an organization as one Highly Useful: complete structure and solicit “[CMI] is conducted on an annual basis. It was accommodated the input of every staff member for internal use and staff are evaluated based on certain on the needed areas of abilities. When CMI results are ready, an annual organizational development within their development plan is developed and submitted to the Board for foundations. approval. Progress is the checked on a weekly basis during

Accuracy of Measurement This evaluation design does not enable the evaluators to comment on the accuracy of the CMI tools itself. To do so, the evaluators would have to be able to independently conduct an assessment of each PF during their

directors’ meeting.” ~ EPF-Georgia staffer. Somewhat Useful: “Scores were more or less accurate. Not completely but useful for seeing where to improve…we discussed what we could do—like having trainings on logic frameworks and outcomes.” ~ EEF-Ukraine staffer. Not Useful: “The CMI process appears to be ad hoc, not really used to train staff.” ~ former EFCA Senior Manager

time of CMI application and compare the results of their assessment versus the resulting CMI scores. However, the evaluation design does allow the evaluators to present a summary of the opinions of PF staff regarding the accuracy of CMI, as presented below. Overall, the evaluators saw a significant diversity of opinion among PF staff on the utility of the CMI process (see examples in textbox above); however, most PF staff seemed to think that the CMI tool itself generally produced an accurate reflection of their present capabilities. The majority of interviewed PF staff answered that CMI was generally a good tool to paint an overall picture of an organization; to detect problem areas, bottlenecks, or areas for improvement; and to organize their PF’s approach to capacity building. Interestingly, the majority of EPF and EFCA staff mentioned that they intended to use CMI to gauge the capacities of their partners. While the consensus of PF staff responses, and the evaluators’ own review of the CMI tool, indicate that CMI is a useful tool for scoring individual organizational capacities, the evaluators were surprised by the lack of documentation and respondent recall on how the CMI recommendations were actually implemented. EPF-Georgia seems to be an exception, though most respondents could only recall the implementation of one or two recommendations following the CMI process. Lastly, the evaluators noted that the CMI tool appears to consider human resource and staff development as a separate capacity, rather than a more holistic and important part of the development of each capacity. Based on the evaluators’ own experience, staff capacity development is an integral part of all stages of an organization’s development, particularly during periods of transition (such as moving from a grantmaking to operational foundation) when the quality of staff can make a big difference.

CONCLUSIONS: ACCURACY OF CMI  The majority of interviewed PF staff believes that the CMI tool produces scores that accurately reflect their individual capacity areas. However, most also commented that there has been little follow-up on the recommendations produced by these scores.  CMI’s separate treatment of Human Resource and Staff Development fails to capture the importance of staff capacity building in each of CMI’s seven Key Capacity Areas. Instead, CMI’s emphasis seems to be focused on assessing the context of a personnel handbook rather than on understanding staff skills, strategy to staff development, needed training/mentoring, or staff motivation.

OVERARCHING CONCLUSIONS: HAS EF-DC SUCCEEDED IN ITS EFFORTS TO ESTABLISH AND BUILD SUSTAINABLE FOUNDATIONS? 

Ensuring the sustainability in challenging political environments is no easy feat. The adoption of a new business model of sustainability following the end of core funding requires significant transitions in a range of organizational capacity areas beyond solely the focus on fundraising. The process can entail significant downsizing and staffing changes and requires significant lead-time. The evaluators found that FNE has made the most significant strides in this effort and adopted a business model distinct from the other PFs, but most other PFs have not really undertaken a genuine transformation to date. While EF-DC undertook a considerable effort to improve the managerial and operational 34

capacity of its PFs, it does not appear to have fully recognized the need for a transformational change in its PFs (from grantmaking to operational models) and has had limited capacity, in large part due to reduced core funding, to effectively manage its PFs’ transformations.

ISSUE III: WHAT ARE THE STRENGTHS AND WEAKNESSES OF EF AS A MODEL FOR ADVANCING US DEVELOPMENT OBJECTIVES? With the aim of increasing the utility of its findings, the evaluation team modified the evaluation question to focus on an analysis of the strengths and weakness of foundations in general (i.e., the “foundation model”) for advancing US development objectives and then to provide a commentary on how well EF was able to conform to this model. Given EF’s experience, in which a dedicated funding stream was discussed but never implemented, the team felt that there was limited value in commenting on the usefulness of replicating this model, which the team would not recommend. However, the team felt that a more macro-level discussion on the comparative advantages and disadvantages of foundations like EF, and the extent to which EF was able to capture these, would prove more useful to future programming. The section below begins by defining how the team understands and uses the term “foundation.” It then highlights traditional comparative advantages and disadvantages of foundations, in comparison only to USAID contracting, drawing from interviewee responses, publically available secondary data, and discussions with the team’s Advisory Group comprised of former foundation executives. It then analyzes how well EF capitalized, or failed to capitalize, on these advantages/disadvantages, noting a marked difference when core funding was certain and uncertain. The section concludes with an examination of how EF beneficiaries and partners perceived the quality of their operational relationships with EF and EF’s program coordination with USAID operating units.

DEFINING THE TERM “FOUNDATION” The team found surprisingly few commonly accepted definitions of foundations, which was likely caused by the varied structures, activities, revenue sources, and country legal frameworks regulating foundations. For the purposes of this evaluation, the team utilized definitions from the US-based Foundation Center and European-based European Foundation Centre. The Foundation Center (TFC) defines a foundation as a “a nongovernmental, non-profit organization with its own funds (usually from a single source, either an individual, a family, or a corporation) and program managed by its own trustees and directors, established to maintain or aid educational, social, charitable, religious, or other activities serving the common welfare, primarily by making grants to other non-profit organizations.”75 It also notes that a foundation has a “principal purpose of making grants to unrelated organizations, institutions, or individuals for scientific, educational, cultural, religious, or other charitable purposes.”76 TFC distinguishes between two types of foundations: (1) Private Foundations (independent/family foundations, company/corporate foundations, or operating foundations) and (2) Public Foundations 75 76

“About Us.” Foundation Center. http://foundationcenter.org/about/. Ibid.

(community foundations, Women’s Funds, or other public foundations).”77 The European Foundation Centre (EFC) has a similar but slightly different definition: “Foundations are autonomous, non-profit organisations with their own resources that work locally, regionally and internationally to improve the lives of citizens, by running and funding activities in a myriad of areas…They each have an established and reliable income source, which allows them to plan and carry out work over a longer term than many other institutions such as governments and companies.”78 EFC groups foundations into four broad categories: (1) Independent Foundations; (2) Corporate Foundations; (3) Governmentally-Supported Foundations; (4) Community Foundations. While the above definitions differ on the issue of governance, they both emphasize the importance of independence in program and the availability of a steady funding stream (usually in the form of an endowment or trust). It is regarding this latter criterion, the availability of a steady funding stream, where EF appears to have deviated from the traditional foundation model. This has had significant consequences for the development of EF and its overarching success, and this is an issue that is explored in detail below.

COMPARATIVE ADVANTAGES AND DISADVANTAGES OF FOUNDATIONS VERSUS CONTRACTING INCLUDING SPEED, FLEXIBILITY, AND INNOVATION Traditional Advantages The list below outlines some general comparative advantages of foundations, compared to contracting traditional implementing partners, as informed by the evaluator’s secondary research,79 interview responses, and the evaluation’s Advisory Committee. These comparisons are broad and typological in nature; cases may be identified as exceptions to the contrasts drawn. Nevertheless, the evaluation team has found these comparisons useful in exploring the significance of the foundation model. 1) Dedicated Funding Source. As compared to contractors, foundations generally benefit from a dedicated funding source or pool of core funding. This provides more flexibility in planning and programming design, allowing foundations to focus on areas of core interest and adapt longer-term timelines than contractors despite fluctuations in outside political environments or donor support. Foundations are thus better able to “stay the course” despite changes in political or donor support. Some significant related advantages include: a) Flexibility in Programming. Less beholden to short-term donor interests than contractors, foundations have the luxury of being less risk-averse and have the ability to fund less popular and potentially controversial projects, such as activities focused on stateless populations or other vulnerable groups, municipal-level administrative reforms, etc. Foundations also generally operate through grants and cooperative agreements, which usually provide more operational flexibility compared to contracts despite having 77

“Foundation Tutorials.” Foundation Center. http://foundationcenter.org/getstarted/tutorials/ft_tutorial/what.html. “Foundations FAQ.” European Foundation Centre. http://www.efc.be/programmes_services/resources/Pages/Foundations-FAQ.aspx 79 Especially Kevin F.F. Quigley [1997], For Democracy’s Sake: Foundations and Democracy in Central Europe, Washington: Wilson Center Press. 78

similar competitive requirements. b) Increased Speed of Response. Operating under an endowment, trust, or reservoir of core funding, foundations can be divorced from procurement hurdles that often plague contractors and can quickly fund and start up emerging opportunities and new project ideas as they become available.80 It typically takes a year or more for USAID to design and award a new contract and more time still for a contractor to mobilize and begin operations in country.81 While contractors can manage small grants through a “grants under contract” mechanism, it can at times take up to a year or more to get a grants manual approved by USAID. Foundations generally have more streamlined internal procurement procedures, particularly for small grants distribution, which allow them to act in a more decentralized way to fund and start up emerging and innovative ideas. c) Longer Time Horizons. Foundations generally operate outside of the traditional three to five year contract durations under which contractors operate. They are established to provide a long-term local presence with a long-serving full-time local staff. Contractors, on the other hand, face difficulty retaining high performing staff as a project approaches its end. Follow-on projects frequently do not overlap with their predecessors and the competitive process means that the same project may have a different implementing partner making it difficult to maintain continuity in pursuing a long-term goal. 2) Deeper knowledge of and stronger relationships with local partners and development contexts. Foundations, as compared to contractors, are established long-term in the country context and can readily prioritize becoming familiar with the socioeconomic, political, and cultural contexts of assistance, including “who is doing what” at any given time. These attributes leave foundations better suited than contractors to deal with long-term institutional capacity building issues with local partners and changing attitudes and behaviors of populations over extended periods of time. They can thus create and strengthen local organizations, build coalitions, and mobilizing local support around specific issues better than contractors. 3) Flexibility in responding to changes in operating environment. With a long-term presence on the ground, foundations can be relatively quick and flexible in responding to emerging issues and challenges. They are able to more rapidly gauge (“take the pulse”) political climates and public sentiment and adjust their programs accordingly, or advise the USG to adjust theirs, with respect to issues “too politically sensitive to touch” compared to contractors.82 4) Better able to support development or strengthening of new institutions, such as think tanks or other types of research organizations. Such accomplishments are outcomes of 80

Comment from Advisory Group meeting on December 7, 2012. There are of course ways to expedite the normal contracting process. OFDA, for example, operates a single award IQC that allows for the rapid issuance of Task Orders to respond to unforeseen circumstances without the time consuming requirement to engage in a further round of competition. OTI has developed a specific set of tools and procedures that allow them to respond quickly and flexibly to unanticipated needs. The primary contracting vehicle is the SWIFT 3 multiple award IQC, which is a $1.5 billion vehicle built around transition activities. Expedited procedures allow for awards within 5-6 weeks of issuance of a solicitation. They have issued awards in as little as 48 hours with a justification for other than full and open competition. 82 Quoted phrases stem from Advisory Group discussion. 81

strong local partnerships, emphasis on the long term, and a capacity to take programmatic risks. Numerous interviewees stressed that because of their long-term presence, foundation staff generally develop a “sense of responsibility” for the legacy of their projects and partnerships beyond the usual three to five year contract lives that contractors operate under. 5) Ability to serve as “knowledge hub.” Through small grant programs, either targeted or “open-door,” foundations can more easily encourage the sharing of ideas and best practices than contractors. They can provide a, sometimes limited, opportunity for citizens to gain a voice in their own community development and propose projects that emanate from local, everyday needs, as opposed to “their needs” as articulated by an outside donor. While contracts can involve detailed contextual analyses and needs assessments, they are rarely designed by beneficiaries themselves. 6) Degree of autonomy. Foundations are generally “one step removed” from government or business interests. While foundations are undoubtedly associated with, and often beholden to, the interests of their founders, they operate with a degree of autonomy and possess greater flexibility, perhaps deniability, in their actions. Foundations can thus support initiatives that may be less welcomed by their host or home country audiences. 7) Internal accountability. Foundations are generally established with internal oversight or governing authorities to monitor operations and (ideally) minimize instances of corruption and mismanagement. Foundations, particularly Western-style foundations, generally face strict transparency and fiduciary requirements to register and operate. Traditional Disadvantages However, the foundation model also entails comparative disadvantages or, perhaps better phrased, situations in which direct contracting would be more appropriate. 1. Susceptibility to weakened programmatic focus. Given the demand-driven nature of small grant management, foundations can end up supporting numerous disparate programs and projects. Foundations can also suffer from being too closely associated or familiar with local contexts, enticing them to become myopic and pursue micro-level solutions to macro-level problems.83 The complexities of local problems can in other words, cause foundations to lose “sight of the forest from the trees.” 2. Rigidity of ideas. Given their long-term focus and establishment around specific missions and visions, foundations can be less receptive to new programming ideas, scholarly thinking, or funding opportunities as compared to contractors whose projects are bound by shorter timeframes. There is irony here, of course, since this threat of inflexibility can be present in foundations that enjoy relative autonomy and independence from funding sources. 3. Accountability to whom? While often “one step removed,” foundations can be susceptible or beholden to the interests of their founders or board members. Generally operating outside of popular support or short-term funding, foundations run the risk of being captured by an elite or privileged few. 4. Lack of deep technical knowledge. As a knowledge center and a catalyst for innovative 83

As articulated during the Advisory Group discussions.

development ideas, a foundation relies heavily on the technical knowledge it is able to readily access through its staff, its board, or other individuals close to the organization. They generally operate with low overheard, employing a small but capable staff able to program across sectors. However, breadth of coverage can come at the expense of depth of coverage. The majority of interviewed PF staff admitted not having the same level of technical understanding of their programming areas. In contrast, contractors generally have access to a deep pool of technical experts, either on staff or by ongoing recruitment efforts for intermittent consultants, in specific functional areas. 5. Too forgiving, too early. While the establishment of relationships with beneficiaries can bring contextual understanding and situational awareness, it can also result in the loss of objectivity. Direct and long-term relationships can cause foundations to “lose perspective” and fund or support ideas that may be unrealistic or unsustainable.84 6. “Kiss of death.” Just as foundations can benefit from the association with their founders, they can also suffer from them. This was particularly evident to the evaluation team in Russia as FNE had to disassociate itself completely from its US roots in the face of strict and xenophobic host-country regulations. 7. Difficulty recovering overhead and administrative costs. While contractors have the ability to charge their overhead and administrative costs through Negotiation Indirect Cost Recovery Agreements (NICRAs) and General and Administrative (G&A) line items, numerous PF interviewees commented on their difficulty to get donors to fund administrative and overhead costs.

CONCLUSIONS: ADVANTAGES AND DISADVANTAGES OF THE FOUNDATION MODEL COMPARED TO CONTRACTING 

Foundations offer comparative advantages in terms of flexibility, speed, and ability to pursing long-term objectives compared to traditional direct contracting models.



One of the defining characteristics of foundations, and origin of the foundation model’s most significant and tangible comparative advantages, is the availability of a reliable source of long-term funding.



Contractors operating under USAID projects have a defined period of performance and defined scope of work. These attributes mean that contractors may be better suited to work on issues that can be addressed within a defined life of project and which require specific technical expertise.



Contracts by their nature are not quick or flexible and may be more suitable for addressing a specific set of well-defined issues.



The foundation and direct contracting models can complement each other by capturing individual advantages and benefits not captured by the other.

EF’S CONFORMITY TO THE FOUNDATION MODEL 84

As discussed during Advisory Group meeting.

Areas where EF captured the advantages of the Foundation Model and likely performed better than USAID contracting. 

Accumulation of Local Knowledge. EF provides an impressive illustration of how a foundation can accumulate knowledge of local conditions and institutions, translate this knowledge to locally adaptive programs, and invest in longer-term results. Since the PFs are generally not highly specialized in technical sectors, as compared to contractors, their strength is relatively greater in grantmaking processes and promoting institutional capacity among partners. In the cases of some foundations, such as EEF-U, they are considered by partners and donors to be among the best grantmakers in the country.



Flexibility in Changing Environments. EF has been highly successful in quickly getting many small grants “out the door” in many countries. EF could rapidly respond, for example, to the new political environment afforded by the Rose Revolution in Georgia, and EEF was readily able to rally resources and partners for political liberalization in Moldova. Some observers noted to the evaluation team that USG agencies and their contractors are also fully capable of running grantmaking programs and that there is therefore nothing particularly unique about EF’s record in this area. Our interviews,, however, included complaints from local observers that at least in some countries (such as Moldova) the grantmaking carried out by other organizations, such as DOS and their contractors, have been accompanied by poorly defined rules, lack of transparency in communication, lengthy delays in making awards, and lack of support in explaining reporting requirements. While EF’s record with grantees is not unblemished, overall the evaluation team has identified few instances of weak grantmaking procedures by EF.



Supporting local institutions. EF’s record includes cases of essential startup and/or continuing support of institutions of national and regional significance, including the Caucasus Regional Resource Centers (CRRC).

Areas where EF was not able to capture advantages of the Foundation Model and likely performed worse than USAID contracting. 

Did not benefit from dedicated funding stream. EF’s core funding is derived from its USAID grant. While EF organizations have varying levels of alternative funding, with the exception of FNE and EEF-Moldova, all the field-based organizations are to varying degrees dependent on the USAID-originated core grant scheduled to be terminated in December 2013. EF is not supported by an endowment. Other major international foundations are supported by recurring line items in the US federal budget and while EF has sought such funding, Congress has not delivered this support.85 While their design and startup may be difficult and labor-intensive, endowments have been considered part of USAID’s “toolbox” for advancing development goals for some years, and Agency guidance since 1994 has recognized endowments funded via

The 1992 Stremlau memo made reference to funding of this kind: “The foundation’s broad goals and structure would be similar to the Asian, Inter-American and African Development Foundations, that are congressionally funded, independent and provide many small grants to assist grass-roots organizations across those regions.”

appropriated dollars rather than local currency.86 Among other objectives, USAID may support the establishment of an endowment to, for example, broaden or enhance the funding base of an NGO engaged in activities with long-term horizons; insulate the endowed organization from unpredictable government and donor agency budget fluctuations; allow the recipient organization to attract other funds by increasing donor confidence; or allow an activity to be institutionalized and continue beyond USAID’s funding when it otherwise may not have been. Not having a reliable stream of revenue can undermine the other tenets of a foundation model (longer-term thinking, deep knowledge base, etc.). While no EF organization (EFDC or any of the PFs) has established an endowment, to varying degrees they all have stepped beyond the unilateral USAID “spigot” to include other bilateral and multilateral donors, international foundations, foundations based in-country, and private sector organizations to diversify funding. 

Strained Programming Focus. With the exception of FNE, which, although reliant on multiple local sources of funding, has managed to keep its program largely directed toward social assistance and media programs, EF’s other PFs are applying local knowledge and relationships in many increasingly diverse areas. The PFs in Armenia and Azerbaijan identify areas of program focus but at the same time face funding issues that strain commitment to the narrower path. Contractors, on the other hand, usually have specific and focused scopes of work and thereby implicitly a strong program focus.



Lack of Deep Technical Expertise. Generally, EF-DC and the PFs appear to have been able to avail themselves of the technical knowledge needed for designing high-quality programs, monitoring them adequately, and working on a peer basis with grantees and other partners. The evaluation team did encounter a small number of instances in which grantees noted that the program manager was insufficiently prepared to provide support in a relatively new area of intervention, such as film production. As we have noted, however, these cases appear to be rare, and the “process professionalism” of Foundation staff in dealing with grantees and partners has prevented serious undermining of credibility for the organization involved.



Mixed benefit from degree of autonomy. A foundation can carry the advantage of being locally perceived as less interventionist than international donors. At the same time, in EF’s case, this status offers both the challenges and opportunities associated with leveraging a foundation’s “third sector” status to support public-private partnerships, corporate social responsibility initiatives, and similar efforts that engage business and governmental entities as stakeholders in programs of mutual interest. The PFs clearly do operate from an institutional venue in the civil society sphere; however, PFs vary greatly in the extent to which they have chosen to, or have been able to, stake out a set of working relationships with state and private sector entities. In addition, the “brand’ afforded by PFs’ affiliation with a US-based development institution is met with mixed reactions among partner foundation staff and stakeholders: While in some instances this affiliation is viewed as a kind of stamp of approval for a PF’s professionalism, in other

See USAID Policy Determination 21 (Guidelines: Endowments Financed with Appropriated Funds, July 18, 1994): http://transition.usaid.gov/policy/ads/200/pd21.pdf.

cases it is seen as something of a burden on the organization’s image.

CONCLUSIONS: EF’S CONFORMITY TO THE FOUNDATION MODEL 

EF’s performance in terms of speed, flexibility, and innovation in comparison to USAID contracting was largely dependent on the availability of core funding. When core funding was available, EF was able to capitalize on many of the advantages of the “foundation model” and outperformed other contractors in supporting local ownership of programs, fostering sustainability, and supporting the social capital needed to enhance democratic processes. However, as core funding declined, EF was no longer able to capitalize on these advantages and did not perform as well as contractors in terms of programmatic focus and depth of technical expertise.



In terms of sector-specific comparative advantages, the evaluation team concluded that EF’s work has generally been most successful in the civil society arena. This result is in accord with the trends in PF portfolio investments observed earlier in this report. Given the predominance of civil society activity in the overall program of EF, it is not possible to determine the effectiveness of EF’s model in other areas from its work in civil society. So while the evidence is clear that the model “performs well” in supporting civil society assistance, on the other hand, there is no compelling evidence that the model could not succeed as well in other sectors if a foundation were to choose to invest in such directions. Foundations can complement other forms of assistance.

HOW DO GRANTEES AND OTHER BENEFICIARIES REPORT ON THEIR EXPERIENCE WORKING WITH EF AND ITS PARTNERS COMPARED TO WORKING WITH OTHER DONOR PROGRAMS? Beneficiaries often found PF’s operational expectations (such as quality of proposals, reporting, and financial transparency) more rigorous than other grantmakers, but in the end PFs were almost always seen to be supportive, competent partners. The more experienced grantee organizations were able to compare PF expectations with those of other donors. In some instances, the level of difficulty was about the same as with other grantmakers, but more often PF processes were considered more challenging. However, some interviewees told us that high expectations by the PF helped their organizations to build capability and find subsequent sources of new funding. With few exceptions, beneficiaries told us they would work with the PF again. Data from the evaluation team’s administration of an electronic survey of Foundation beneficiaries illuminated various levels of satisfaction with the PFs as partners. With a total of 81 responses, the survey captures only a small portion of all PF grantees (700 contacted), and there may be a tendency for those with more positive attitudes toward the Foundation to respond to the survey. Nevertheless, some noteworthy patterns emerge from these responses: 

When asked to compare their experience with other donors with their EF relationship, 60% said that the experiences are about the same;



85% of respondents indicated that during the grant period they interacted “frequently” or “constantly” with EF staff;



55% of grantees found the EF grant application process “not very difficult,” but 18%

found it “very difficult” or “difficult”; 

76% reported that the local EF staff were “very helpful” or “helpful” in guiding them through the application process;

Beneficiary Comment on Working with EF “In our case, EPF was the real presence next to us (for example, with technical problem solving with interpreters, etc.). They were constantly checking in to be available to help.” --A grantee for a cross-border program supported by EPF-Armenia



60% reported that once they received the grant EF staff were “very helpful” or “helpful” in providing assistance toward achievement of grant objectives;



66% said that the professional competence of EF in explaining grant requirements was a positive aspect of their EF experience;



27% percent noted that “too much emphasis on monitoring and measuring results” (the most commonly chosen factor among several choices) was the least positive aspect of their experience; and



79% said that support from the EF had contributed to their organization’s success in meeting its goals.

In summary, while the application, monitoring, financial management, and reporting requirements of EF grantmaking can be challenging at times for grantees, these operational expectations appear not to be overly burdensome to beneficiaries and indeed prove to be opportunities for valuable capacity strengthening for many.

HOW DID EF COORDINATE WITH USAID MISSIONS AND OTHER USAID PROGRAMS? The EF regional grant is overseen by a USAID Agreement Officer’s Technical Representative (AOR) based with the Bureau for Europe and Eurasia in Washington, D.C. At the country level, there are no formalized coordination points or mechanisms except on occasions (such as in Armenia) when the USAID Mission has awarded one of its own programs to a PF. In Russia, FNE is widely known and FNE leadership and staff talk regularly with donors – although of course, with the closing of the USAID program in Russia, such frequent communications are not expected in the near future. The working relationship in Moldova is not close and mission personnel complained about a lack of coordination, although we note that the USAID Mission Director was invited to visit the most recent meeting of the EEF board. In Ukraine, leadership and staff are on a familiar basis with technical specialists at the Mission. Without an ongoing program relationship one key USAID source described the Mission’s attitude as one of curiosity about how EEF sees its strategic vision and role in the civil society sector. On the Washington side, EF-DC historically has looked to State and EUR/ACE for oversight and coordination rather than to USAID. For its part, USAID’s historical pattern has therefore been to exert little oversight through the AOR.

RECOMMENDATIONS RECOMMENDATION FOR USG AUDIENCE

restructuring and that additional resources and support may be required in the short term to make this transition possible. Policymakers should realize that the foundation’s leadership may not have the right skills or capabilities to lead this transition. Organizational change expertise is required, which the foundation headquarters is not likely to have. Also, if the headquarters entity is undergoing a similar transition, they may have fewer resources to devote to the transition of local “affiliates.” In the EF example, both EF and the USG did not appear to fully recognize the difficulty of this transformation and the level of time and investment needed to do so. From the foundation perspective, the organization’s leadership: (1) must recognize the need for an organizational restructuring; (2) a develop plan to significantly realign staff and identify the needed resources; and (3) implement concrete steps to overhaul its staffing composition. This should include investing heavily in training current staff in and/or hiring new staff that possess the required competencies.

RECOMMENDATIONS FOR EF AUDIENCE

USAID MISSIONS SHOULD SEEK MORE ACTIVE ENGAGEMENT WITH PFS REGARDLESS OF WHETHER THEY ARE IMPLEMENTING PARTNERS OR NOT

There are benefits to having a local foundation on the ground and USAID should leverage that as a means to build understanding and networks and consider areas of collaboration. There are specific actions that could be taken, such as including EF PFs in regular meetings of implementing partners regardless of whether the PF is a direct recipient of USAID funding at that time. USAID AORs could do more to foster the relationship between the field Missions and the PFs by increasing communication to the Missions regarding EF activities. The AORs should continue to visit PF offices while in country and share reports and other relevant information.

ANNEXES ANNEX A â&#x20AC;&#x201C; SCOPE OF WORK C.3.1

STATEMENT OF WORK FOR COMPONENT I: THE EURASIA FOUNDATION AND PARTNER NETWORK EVALUATION The Eurasia Foundation has now been in operation for nearly 20 years, received approximately $320 million in core U.S. Government funding, and has operated in a total of 12 countries (with current operations in 10 through the foundation network). The extent of the Eurasia Foundationâ&#x20AC;&#x2122;s reach and experience presents a unique evaluation opportunity to assess the success of this approach in meeting USAID objectives, the overall strengths and weaknesses of the foundation model, and the efforts to build sustainable legacy foundations. This evaluation is intended to identify valuable lessons learned for multiple audiences and purposes. As noted above, there are three main evaluation issues: (I) assess how successful EF programs were in meeting the three broad strategic objectives identified in the grant agreement; (II) assess the strengths and weaknesses of the Eurasia Foundation as a model in advancing U.S. policy objectives; and, (III) assess the efforts of EF-DC to establish and build sustainable foundations through its partnership Network and to plan for the end of core funding. The primary objective of the overall evaluation is to build understanding of the strengths/ weaknesses and advantages/disadvantages of the foundation model to inform USAID, the State Department and other stakeholders on future program design in the region, including those involving foundations. This will also be of interest to Members of Congress who have supported the Eurasia Foundation model. Furthermore, the report should provide learning opportunities for EF-DC to improve upon their model, sustainability efforts, and support services provided to the foundation network. The findings will also be of interest to other foundations and NGOs involved in the core program area of the EF grant agreement.

A. Evaluation Questions The evaluation is aligned to three primary Evaluation Issues identified above that broadly relate to advancing USAID strategic objectives, foundation models, and sustainability. As noted above, Evaluation Issue I related to advancing strategic objectives is expected to require 20% of the Level of Effort and be based primarily on program documentation and information gathered in stakeholder meetings related to the other two evaluation issues. Thus, the priority areas are Evaluation Issues II and III related to the foundation model and sustainability, which are expected to each require 40% of the total Level of Effort. 1.

Evaluation Issue I: Meeting Strategic Objectives - 20% of Level Effort

Given the breadth of issues addressed by EF over the 10 years of the current grant agreement, and the evolving conditions and uneven pace of reform in the region, it would be difficult to produce a comprehensive and quantitative analysis of the overall effectiveness of the program, or develop standardized evaluation criteria. However, given the sizable investment of U.S. Government resources to advance the three strategic objectives (SOs) in the region, the evaluation should seek to draw high level conclusions regarding how successful EF programs were in meeting these objectives in specific cases. The evaluators should focus on the countries and SOs where the greatest investments were made (See Annex A). The evaluators should also distinguish between high level political developments and the civil society/grassroots level, which is where Eurasia Foundation programs were targeted. Eurasia Foundation and its partner foundations have made hundreds of small grants and it is not envisioned that the evaluators seek to interview numerous beneficiaries in program areas. Rather, the evaluators should select 5-7 focus areas defined as SO-country/region combinations and then seek to gather much of the information on the effectiveness of programs in these specific areas through interviews required for other evaluation issues,

including EF partner foundations, major grantees, and USAID representatives. A few additional supplemental meetings with other donors, leading NGOs, associations and/or civil society groups that are familiar with EF programs and grantees may be held to further explore views on what made programs more or less successful. This section should also incorporate some information based on gender, including the examination of available data on participant trainees and beneficiaries disaggregated by sex. The purpose of Evaluation Issue I is to draw high level conclusions on the factors contributing to or detracting from success in 5-7 specific SO-country/region combinations as a means to inform future program design. Such conclusions would be based primarily on qualitative information from stakeholder interviews and program documentation. Illustrative examples are provided here to demonstrate the level at which conclusions should be reached – they are not intended to be examples of areas where the evaluators should focus. 



Example: After 10 years in Country X, there is a clear record of progress in civil society development. Progress was achieved in specific areas where EF programs were active. Key stakeholder interviews show that beneficiaries felt that EF programs were effective and contributed to a growing and strengthening NGO community. Example: The efforts of the EF through its Small Business Loan Program to advance entrepreneurship were not a significant part of EF’s program in any country. Even in the countries where EF was most active, there were numerous other larger donor programs supporting small business lending. While individual businesses may have benefitted there is no evidence that the programs had a significant impact in any country or a region. Example: Media programs developed in several countries made a significant contribution to improved journalism standards and drew public attention to issues of corruption, placing pressure on government officials to change behavior in some countries. In several countries where EF was active, there has been significant backsliding in terms of free and independent media. While key stakeholder interviews ranked EF programs as effective or very effective, government efforts to control or intimidate the media created a hostile environment. Some beneficiaries of EF programs felt that they were better able to cope with the situation as a result of EF training and support.

In addressing the ability to meet strategic objectives, the evaluators should address the following issues:    

Did EF programs contribute to advancing the three primary SOs outlined in the original Grant Agreement related to: Private Enterprise Development, Civil Society, and Public Administration and Policy? Did the Foundation prove more effective in advancing some SOs than others? Were EF and its partners better suited to carry out programs in some strategic objectives than others? EF and its partner foundations currently operate in 10 countries. Were there countries or regions where the activities of EF and its partners were considered more successful, less successful? Were there local factors that contributed to the prospects for success in some program areas vs. others? Based on available gender disaggregated data, what high level conclusions can be reached regarding the gender balance in EF programs? Were there some SOs where gender balance in participation was achieved and others where it was not? Was it more challenging in some countries to attain gender participation than others? Evaluation Issue II: Foundation Model - 40% of Level of Effort

The evaluation should also assess the overall strengths and weaknesses of the foundation model as an assistance delivery mechanism. This process should also identify lessons learned for establishing foundation programs and for interactions with local stakeholders. This should draw upon the direct experience of the EF and its network foundations in the advantages of operating as a foundation in the E&E environment. The evaluation should address the following issues: 

What were the primary operating models or instruments of delivering assistance employed by the Eurasia Foundation? Were some instruments more effective than others?

 

Were there areas where the EF model or foundation approach had a comparative advantage in delivering assistance, in terms of speed, flexibility, and innovation? Are there clear advantages and shortcomings of the foundation model in meeting certain strategic objectives? Conversely, are there areas where the EF model is not appropriate to meet USAID objectives? If so, what USAID tools might be a more appropriate option (project, local APS mechanism, bilateral agreement, DOAG)? During the current grant agreement, did the EF change its operating models or instruments based on lessons learned in implementation or to address limitations to existing instruments? How do grantees and other beneficiaries report on their experience in working with the EF and its partners? Were procedures open and transparent, were there frequent rules changes, how difficult was it to meet reporting, financial and other requirements? Do beneficiaries note that other USAID or donor programs were easier to apply, implement and report (i.e., EU, IFC, Soros, etc.)? How did EF coordinate with USAID Missions, other donors, other programs? Were efforts made to ensure USAID was aware of the programs supported by EF? Were programs harmonized? How did EF and its partner foundations monitor and evaluate their programs? What are the strengths and weaknesses of their M&E approach? Evaluation Issue III: Sustainability – 40% of Level of Effort

As noted above, the EF grant agreement was modified in 2003 to allow EF-DC to focus on establishing partner foundations in the region. The evaluation should assess the effectiveness of EF-DC in building these institutions, prospects for sustainability of EF-DC and the foundation network, and whether this approach can serve as a model for future programs. The evaluation should also assess if the drive to sustainability has required the partner foundations to alter their focus areas, or if they continue to implement or fund programs that align to the purpose of the original grant. If the program purpose has evolved considerably, the evaluators should note this and provide explanations. In addition, through desktop research, the evaluators should try to determine if other U.S. Government supported foundations have evolved to a sustainability model similar to EF-DC. This includes the Asia Foundation, InterAmerican Foundation, and African Development Foundation. The evaluation should address the following issues:   

 

Is EF-DC on track to meet its long-term goal of financial sustainability without need of further directed core funding from the USG? In which countries and SOs does EF-DC expect to continue operations in the next 5 years? How has the drive to financial sustainability altered EF-DC’s operations and capabilities? Are the number of programs or focus areas being reduced or expanded to operate in accordance with sustainability requirements? EF-DC has also been focused on establishing and building partner foundations in the region since 2003. EF-DC created the Capacity Mapping Initiative (CMI) model to develop and measure the organizational capability of partner foundations. Does the EF CMI model sufficiently address the organizational needs of the partner foundations? Did EF leverage the CMI model with training and other resources to prepare staff for the transition to sustainability? Are there gaps in this process? Are the partner foundations capable of continuing to carry out their programs? Is their fundraising capability sufficient? Are management and operational capabilities (HR, finance, training, etc.) in place to allow for ongoing growth and program sophistication? Although EF-DC and the partner foundations receive funding from other sources, does the Foundation Network continue to focus predominantly on areas aligned to the three SOs in the grant agreement? If their focus areas have changed significantly, a brief explanation should be provided (i.e., changing conditions in the country, changing donor landscape, new priorities, etc.). What key services are provided by the Washington office to the Network foundations? How do the Network Foundations regard the quality and timeliness of these services? Are Network Foundations willing to pay the Washington office for these services? Leveraging desktop research, the evaluators should determine if other U.S. Government supported foundations have evolved to a sustainability model similar to EF-DC, such as the Asia Foundation,

III

Inter-American Foundation, and African Development Foundation. If so, which ones have done so and have any established sustainable partners in the field? B. Team Composition and Qualifications This evaluation will be carried out by a four- to five- person team. Two members will be provided by the Contractor under this task order. Two-three USAID staff will participate as a team member on each team but under the direction of the Team Leader. The contractor team will include a Senior Technical Advisor /Team Leader, and an Institutions and Organizations Analyst. Individual contractor evaluation team member responsibilities and qualifications are listed below. 1.

Senior Technical Advisor/Team Leader - The Senior Technical Advisor/Team Leader should have 15 years of experience in the implementation of foreign assistance programs and an advanced degree (Masters or above). He/she should have extensive overseas program evaluation experience (including USAID related) and be thoroughly familiar with techniques of program performance appraisals. Strong writing and word processing skills are a requirement. Previous overseas experience in the NIS and Russian language capability is highly desirable. The incumbent is responsible for coordinating and directing the overall evaluation effort, including planning for logistical aspects of the evaluation and preparation and submission of the draft and final evaluation reports to USAID/E&E. The incumbent should be either a regular staff member of the contractor or a consultant used by the contractor for evaluations. As evaluation team leader, the incumbent should possess good organization and teambuilding skills.

Institutions and Organizations Analyst - The Institutions and Organizations Analyst should have a minimum of 10 years of experience in the design, implementation and/or evaluation of foreign assistance programs and an advanced degree. The incumbent should have broad familiarity with the organization structure, problems and needs of foundations, NGOs, or civil society groups, including fundraising and outreach programs. He/she should also have expertise in appropriateness, structure and benefits of successful grants programs. Strong writing and word processing skills are a requirement. Previous overseas experience in the NIS and Russian language capability is highly desirable.

The incumbent is responsible for supporting and coordinating logistical aspects of the evaluation and preparation and submission of the draft and final evaluation reports to USAID/E&E. The incumbent will draft sections of the evaluation specific to the countries he/she visits, leading interviews, and developing questionnaires as needed. The incumbent should be either a regular staff member of the contractor or a consultant used by the contractor for evaluations. 3.

USAID Officers - The two to three incumbent(s) should have broad USAID experience, with particular emphasis on project implementation and evaluation. They should also have broad experience in the E&E region as well as a detailed understanding of the Eurasia Foundation programs and operations from both a Washington as well as field perspective. The USAID team members should each have direct experience in one or more of the SO aligned areas â&#x20AC;&#x201C; private enterprise development, civil society, and/or public administration. USAID staff will operate under the general direction and guidance of the Team Leader and will participate in data collection through interviews, meetings, etc., both in Washington and in the field. After the award is made, they will be engaged with the contractor in the evaluation design process and support drafting relevant sections of the report as appropriate. They will also review text and make comments and edits during the drafting and review process.

The Evaluation team shall demonstrate familiarity with USAIDâ&#x20AC;&#x2122;s Evaluation Policy (http://www.usaid.gov/evaluation/USAIDEvaluationPolicy.pdf). All Team members will be required to provide a signed statement attesting to a lack of conflict of interest, or describing an existing conflict of interest. C. Suggested Methodology

There are extensive background information and reports available to the evaluators. The document review should include but not be limited to the following:         

The original grant agreement and all amendments and modifications All annual and bi-annual reports from EF-DC to USAID The 2001 Evaluation under the previous grant Country specific 3-year strategic plans that identify priority areas of focus for that country/region The CMI tool and past CMI reports Sustainability Plan and EF-DC reports under the Sustainability Plan The 2008 Development Plan prepared by EF-DC that includes EF network fundraising plans and targets Network foundation reports to EF-DC Relevant evaluations of programs carried out by EF-DC and partner foundations (for programs to be specifically reviewed by the evaluators)

A review of secondary literature as determined relevant by the evaluation team. This evaluation presents challenges due to the long term nature, extensive geographic reach, and significant funding levels of the EF grant. This is particularly relevant in carrying out Evaluation Issue I – Meeting Strategic Objectives. As noted above, this is expected to require approximately 20% of the evaluator’s Level of Effort. Given that the Eurasia Foundation operated in 12 countries, with 3 strategic objectives each, this creates 36 possible areas to explore. Furthermore, given the number and diverse nature of the grants awarded by EF and its partners, as well as their own programs, it is not feasible to establish quantitative criteria to measure program impact across all activities. As noted above, the purpose of this Evaluation Issue is to inform future program design in the region. As such, the evaluation is expected to draw high level conclusions on how successful EF programs were in 5-7 SO- country/region combinations and the factors that contributed to or detracted from success. As included in the questions section above, these factors can include country conditions or issues specific to EF, its network foundations, or partners. Demonstrating causation or direct linkages between specific programs and outcomes is not expected. Rather, the evaluators should seek to draw high level conclusions about what worked well, what did not, and why (please refer to the Illustrative Examples on page 14). A suggested approach for considering success in meeting the strategic objectives would be to first review commonly used resources of economic and democratic development for countries in the region. Examples include Freedom House “Nations in Transit”, USAID’s “NGO Sustainability Index”, the World Bank “Doing Business Report”, the World Economic Forum “Global Competitiveness Index”, Transparency International’s Global Corruption Reports, and other resources. In addition, USAID’s Monitoring Country Progress report analyzes country performance by drawing from these reports and other data points and creates additional indicators for country comparisons and trends over time. Based on desktop research of these resources, broad conclusions can be reached in terms of the performance of specific countries over the designated timeframe. A matrix could be produced that reaches broad conclusions as to whether a country improved or declined in looking at indicators reflective of each of the SOs. The evaluators should then review the investments made by EF and partner foundations in order to determine the countries and SOs where the largest investments were made. Then, using the analysis of available resources, determinations can be made as to whether EF made significant investments in the areas where the country performance improved or conversely, where it declined. With this information, the evaluators could then identify the 5-7 SO-country/ region combinations that would be explored further. The evaluators will then seek to identify key stakeholders to interview regarding those specific SOcountry/region combinations. As noted above, much of the information to assess Evaluation Issue I will be gathered from the same stakeholders as with Evaluation Issues II and III. However, some supplemental meetings will also be needed to complete Evaluation Issue I. This could include large grantees or organizations that worked extensively with a grantee in a particular area. In addition, external stakeholders

could include other foundations in the region (including US supported Enterprise Funds and their legacy foundations); other donors such as the World Bank and EBRD; leading NGOs or civil society groups; private sector associations and professional groups; and host government officials when appropriate. The evaluators should seek the input of the EF Network offices in each country, USAID staff, and other donors to identify key stakeholders, but should also conduct independent research as needed. However, only a few key additional stakeholder meetings are envisioned to assess a particular SO-country/region combination. The draft Work Plan (see Deliverables section below) should identify the 5-7 SO-country/region combinations that will be explored, the justifications for proposing those areas for further analysis, and the supplemental interviews that will take place as part of assessing this evaluation issue. The final determination will be made in consultation with the COR. With respect to Evaluation Issue II, evaluators should review project documentation and conduct interviews with USAID, EF staff and program participants to understand EFâ&#x20AC;&#x2122;s operating model and areas where the EF model had a comparative advantage to other types of programs. Furthermore, given the long timeframe of the grant, to assess coordination with USAID Missions it might be required to interview staff no longer at that post. EF-DC and USAID will facilitate identifying and contacting those informants. For Evaluation Issue III, the information should be gathered directly from EF-DC and the Foundation Network. Review of the sustainability plan, CMI tool, and current targets and fundraising results of EF-DC and the partner foundations will be critical. Evaluators may request management documentation from EFDC and the Foundation Network to assess sustainability. In assessing sustainability, evaluators should compare observed and reported practices, operations, and capabilities of the foundation and its partners to explicit sustainability criteria, such as best practices, benchmarks, or NGO sector standards. The evaluation team should then develop interview guides to be reviewed and accepted by the COR. The interview guides should identify the target audience for sets of questions by each Evaluation Issue. Minisurveys and/or group interviews of relevant informants, such as USAID staff, should also be considered, as well as web-surveys or group interviews of key stakeholders in some countries. Program documentation should be used, as appropriate, to support evaluation findings. Also note that this approach is also suggested methodology and offerors can propose any approach that they feel will enable them to draw sufficient conclusions to achieve the objectives of this evaluation. Any methodology proposed by the Contractor should allow them to stay within the LOE percentage allocations for each issue area. D. Country Visits EF activities are widely dispersed throughout the E&E region, extending over thousands of miles. For this reason, along with security considerations and travel difficulties, it is not considered feasible to have the evaluation team visit all 10 countries in which EF operates. A minimum of 5 country visits should be made, and visits are mandatory for Russia and Ukraine given the level of investment in these countries. At least one country each from the Caucasus and one from Central Asia must also be included, and the additional country should be proposed and justified by the contractor. The country visits for Component I will be conducted over a 4-week timeframe, and a 6-day work week is authorized while in the field. Ideally, travel will occur during the month of June in an effort to avoid to the extent possible seeking interviews during this regionâ&#x20AC;&#x2122;s popular holiday months of July and August. It is also envisioned that the team split up and visit countries in sub-teams of 2, based on a logical schedule proposed by the contractor that minimizes expense and travel times. Each group of 2 will include one representative of the contractor and one representative of USAID, each visiting a group of countries together. The contractor is expected to propose a specific schedule and arrangement of country visits in the Work Plan. This should include the number of days in each country, as well as identify travel within a country or cross border trips.

The final decision on country visits will be determined in consultation with team members and the USAID COR. Countries selected should provide a cross-section of EF activities, and allow the team to assess the 57proposed SO-country/region combinations proposed in the Work Plan. In each country visited, the evaluation team should visit USAID Missions and meet with Mission staff familiar with EF programs as well as relevant US Embassy personnel. EF partner foundation offices should also be visited and meetings with relevant partner foundation staff to assess each of the Evaluation Issues. Also, for assessing Evaluation Issues I and II in particular, interviews should be scheduled with local organizations that received extensive EF support, project sites, and other donors involved in relevant program areas. While a minimum of 5 country visits is required to consider all the Partner Foundations, opportunities to visit additional programs and stakeholders by making short regional trips should also be explored. While on the ground, the evaluation team should identify other stakeholders to interview and allow flexibility in schedule in order to respond to new information and opportunities in country. C.5

DELIVERABLES87 1. 2.

3. 4. 5. 6. 7. 8.

Work Plan: include evaluation activities, schedule, country visits and days in each country for Components I and II, along with defined roles and responsibilities for team members Evaluation Design: outline of methodology for the evaluation for Components I and II, including detailed evaluation design matrix with evaluation questions, and for each question the methods used to address it, sources of information, and data analysis plan. Data Collection Tools: Draft and Final questionnaires, surveys, focus groups, and other data collection instruments in both draft and final for Components I and II. Updated SOW if applicable Briefings: Out-briefs provided to each Mission that requests one in countries visited, structured as no more than a 10-slide PowerPoint or 3-page Word document Draft Report: 1Electronic version due within two weeks of conclusion of field visits that is structured as outlined above and includes appendices Final Report: 10 copies and an electronic version due within 6 days of receipt of final comments from USAID that incorporates USAID comments and includes appendices Final Presentations: Following acceptance of the Final Report, a presentation will be made to USAID Washington, State Department and other relevant stakeholders on final report findings that will be in Power Point format and follow the structure of the Final Report. A presentation will also be scheduled with Eurasia Foundation in Washington. Evaluation Record: including interview transcripts or summaries, all quantitative data collected by the evaluation team must be provided in an electronic file in easily readable format agreed upon with the COR.

All deliverables should be submitted to USAID/W, including the hard copies of the final report. The Contractor will make the final evaluation reports publicly available through the Development Experience Clearinghouse within 30 calendar days of final approval of the formatted report. A more detailed description of the deliverables can be found in Section F.

All deliverables are to be produced in English

VII

ANNEX B – DETAILED METHODOLOGY, LIMITATIONS, AND THREATS TO VALIDITY SI employed a mixed-method evaluation design with a heavy reliance on qualitative data and analysis, reinforced where possible by quantitative analysis. This consisted of a detailed desk review, semistructured key informant and group interviews, focus group interviews, an electronic grantee survey, and a quasi-Delphi “Advisory Committee” discussion. The team’s desk review consisted of analyzing relevant documents provided by USAID/E&E and EF and program and management documentation received from EF and its PFs. The team also reviewed materials associated with the earlier 2001 EF evaluation. During fieldwork, the team conducted semi-structured key informant (SSKI) and group interviews and also conducted a few focus group discussions where feasible (Tajikistan and Belarus). Interviewees were selected by the individual PFs themselves and interviews almost always took place at the PFs’ offices. The team used SSKI guides that combined both open and close-ended questions (See Annex G). The guides were not pretested and were slightly refined as the fieldwork went on. Key stakeholders were interviewed in Washington, D.C. and in each of the 8 countries to be visited (Russia, Moldova, Tajikistan, Kyrgyzstan, Armenia, Azerbaijan, Ukraine, and Belarus). These included USAID and other USG officials, EF leaders and grant managers, grantee representatives, and a selected number of program beneficiaries, such as media professionals, graduates of training programs, and government officials at the policy and administrative levels. At the conclusion of fieldwork, the team had a robust data set from over 100 SKIs. Interviews were in almost all cases conducted by two evaluators, one asking questions and the other taking notes and asking follow up questions. SI also assembled an Advisory Group comprised of esteemed foundation executives to review draft evaluation findings and conclusions regarding questions on the “foundation model” and the sustainability of EF and its network. The idea was to utilize a modified Delphi technique to attempt to validate the team’s early impressions. The AG’s discussions were incorporated into the draft report. The AG was also asked to read the draft report and provide written comments for incorporation into the final draft. Lastly, the team designed, refined, and implemented an electronic survey to EF beneficiaries across all PFs (not just those visited by the team). The survey data was then analyzed by SI HQ staff and incorporated into the draft report. EVALUATION QUESTIONS The original Request for Task Order Proposals (RFTOP) provided a useful set of primary evaluation questions (shown in bold) and secondary evaluation questions which SI focused and modified. While the additional evaluation questions posed in the RFTOP were relevant and interesting, SI stressed the importance of focusing on those questions of most importance and utility to USAID in designing similar or follow-on activities. COMPONENT ONE: EVALUATION OF EF AND ITS PARTNER NETWORK Has EF contributed to the achievement of the three SOs identified in the original grant agreement (20% of LOE)?  

Was EF successful in advancing the three SOs identified in the original grant agreement and were EF and its partners better able to carry out programs in some strategic objectives than others? Were there any contextual factors that facilitated or hindered the achievement of each SO? What steps did EF take to ensure the achievement of gender balance in each of the SOs?

What are the strengths and weaknesses of the EF as a model for advancing US development

VIII

objectives (40% of LOE)?  



Were there sector or country areas where the EF model had a comparative advantage in delivering development assistance compared to other forms of USAID assistance in terms of speed, flexibility, and innovation? How do grantees and other beneficiaries report on their experience in working with the EF and its partners, and how did this compare to working with other donor programs, i.e., were procedures open and transparent, how difficult was it to meet reporting, financial and other requirements of USAID compared to others, etc. How did EF coordinate with USAID Missions and other USAID programs in implementation?

Has EF-DC succeeded its efforts to establish and build sustainable foundations through its partnership network prior to the end of core grant funding (40% of LOE)?   

What steps has EF-DC taken to ensure the financial sustainability of its partner network beyond the end of core grant funding? Does each of the partner foundations have the financial, managerial, and operational capacity to ensure ongoing program growth and sophistication? Is EF-DC’s Capacity Mapping Initiative (CMI) successful in accurately measuring the organizational capacity of partner foundations and addressing the key areas that allow for this determination?

COMPONENT TWO: EVALUATION OF NEE IN BELARUS How effective are NEE programs (40% of LOE)?   

To what degree are NEE programs effective? What are the internal and external factors influencing the level of success of NEE programs? Would a smaller and more targeted portfolio influence the success and visibility of NEE programs?

Does NEE’s current program portfolio contribute to achieving USAID/Belarus’s SOs (20% of LOE)?  

In what way does the NEE portfolio contribute to achieving USAID/Belarus's SOs? How can NEE modify and adjust its portfolio to better achieve USAID/Belarus SOs?

What are the strengths and gaps in NEE’s organizational capacity (40% of LOE)?   

What are the key constraints and capacity gaps that effect the NEE’s sustainability? Does the CMI tool of the Eurasia Foundation offer opportunities to address the identified capacity gaps, as compared to the OCA and other tools used by the evaluators? How effective is the current mode of operation/division of labor, with EF/Washington providing management oversight and EF/Kiev providing technical and financial oversight of the NEE? Could the delivery of services through the EF Network to the NEE be improved?

EVALUATION DESIGN MATRIX Evaluation Question

Evaluation Method

Source of Information

Data Analysis Plan

Component One: Evaluation of EF and its Partner Network Has EF contributed to the achievement of the three SOs identified in the original grant agreement?

1. Desk review 2. Structured key informant interviews 3. Group

Administrative and management data from EF (desk review) Grant portfolios (desk

Prepare a chart showing range, value, and array of input investments to identify patterns of association and variance

- Was EF successful in advancing the three SOs identified in the original grant agreement and were EF and its partners better able to carry out programs in some SOs than others?

interviews 4. Electronic surveys

- Where there any contextual factors that facilitated or hindered the achievement of each SO? What steps did EF take to ensure the achievement of gender balance in each of the SOs?

What are the strengths and weaknesses of the EF as a model for advancing US development objectives?

- Were there sector or country areas where the EF model had a comparative advantage in delivering development assistance compared to other forms of USAID assistance in terms of speed, flexibility, and innovation? - How do grantees and other beneficiaries report on their experience in working with the EF and its partners, and how did this compare to working with other donor programs, i.e., were procedures open and transparent, how difficult was it to meet reporting, financial and other requirements of USAID compared to others, etc.

1. Desk review 2. Structured key informant interviews 3. Electronic surveys 4. Group interviews 5. Review of evaluation evidence

review) 3. Secondary data sources, as listed in text (desk review) 4. EF staff (SKIs) 5. Cohort grantees including civil society, gender identified groupings, local EF Network staff, and government officials (Group interviews) 6. EF beneficiary stakeholders (Survey) 7. Non-stakeholders familiar with EF program outcomes (SKIs) 1.

2. 3.

4. 5. 6.

Administrative and management data from EF (desk review) Grant portfolios (desk review) Secondary data sources, as listed in text (desk review) Eurasia Foundation staff (SKIs) USAID current and former staff (SKIs) Network Foundation leadership (SKIs) EF grantees (Survey and group interviews) SIâ&#x20AC;&#x2122;s Advisory Committee (review of evidence)

between inputs Disaggregate data by gender to determine extent of gender balance and use desk review and SKIs to assess how EF achieved that level of gender balance Provide aggregate answers to other questions to stimulate additional questions prior to field work Compile and synthesize all data to construct ordinal ratings to permit comparisons across EF countries

Analyze EF strategies and grant-making patterns to determine whether EF changed operating models Compare EFâ&#x20AC;&#x2122;s interactions with grantees to those of other donor grant programs Use Advisory Committee expertise to analyze advantages and disadvantages of the foundation model

- How did EF coordinate with USAID Missions and other USAID programs in implementation? Has EF-DC succeeded its efforts to establish and build sustainable foundations through its partnership network prior to the end of core grant funding?

1. Structured key informant interviews 2. Desk review 3. Electronic surveys

- What steps has EF-DC taken to ensure the financial sustainability of its partner network beyond the end of core grant funding?

- Does each of the partner foundations have the financial, managerial, and operational capacity to ensure ongoing program growth and sophistication?

EF grantor organization leaders (SKIs) Capacity Mapping Initiative (Desk review) SI’s Advisory Committee (Desk review) Other quasi-public, quasi-private foundations such as TAF (Desk review) Administrative and management data from EF (desk review) Other donor organizations that provide funding to EF and its Partner foundations (electronic surveys)

NEE strategy and operational plans (desk review) NEE staff (SKIs) Non-EF stakeholders (such as grantees and beneficiaries) (SKIs) EF-DC and Kievbased staff (SKIs)

Examine and assess M&E output against SI report standards and rating systems

Use SKIs to determine “what works and why”

Analyze EF investment strategies and actual investment patterns to assess efforts to achieve financial sustainability Analyze the CMI and EF’s institution building to determine the effectiveness of EF’s capacity building efforts Assess EF strategies and investments over 10 years and analyze EF’s theories of change Develop a funding data matrix to determine similarities and differences between EF’s approach and that of other similar foundations

- Is EF-DC’s Capacity Mapping Initiative (CMI) successful in accurately measuring the organizational capacity of partner foundations and addressing the key areas that allow for this determination? Component Two: Evaluation of NEE in Belarus How effective are NEE programs? - To what degree are NEE programs effective? - What are the internal and external factors influencing the level of success of NEE programs? - Would a smaller and more targeted portfolio influence the success and

1. Desk review 2. Structured key informant interviews

2. 3.

visibility of NEE programs? Does NEE’s current program portfolio contribute to achieving USAID/Belarus’s SOs?

1. Desk review 2. Structured key informant interviews

- In what way does the NEE portfolio contribute to achieving USAID/Belarus's SOs?

Produce a matrix and “degree of fit” assessment to compare portfolio with USAID’s SOs

NEE strategy and operational plans (desk review) USAID staff (SKIs) NEE grantees (SKIs) NEE staff (SKIs)

NEE staff (SKIs)

EF-DC and Kievbased staff (SKIs)

USAID staff in Belarus and Kiev (SKIs)

Use the Capacity Diagnostic Tool to identify gaps in NEE organizational structure, staff, and procedures

Assess the utility of the CMI in addressing capacity gaps within the NEE

Examine the success of the division of labor through SKIs

2. 3.

- How can NEE modify and adjust its portfolio to better achieve USAID/Belarus SOs? What are the strengths and gaps in NEE’s organizational capacity - What are the key constraints and capacity gaps that effect the NEE’s sustainability? - Does the CMI tool of the EF offer opportunities to address the identified capacity gaps, compared to USAID’s OCA tool and other tools used by the evaluators? - How effective is the current mode of operation/division of labor, with EF/Washington providing management oversight and EF/Kiev providing technical and financial oversight of the NEE? Could the delivery of services through the EF Network to the NEE be improved?

1. Capacity Diagnostic Tool 2. Structured key informant interviews

Possibly also staff of other partner foundations to identify where they are in terms of capacity development to share best practices (SKIs)

THREATS TO VALIDITY Given the broad scope and relative limited resources of the evaluation, the evaluation design and methodology has some significant internal and external threats to validity. Internal Threats to the Validity Selection Bias

XII

The most significant threat to validity faced by the evaluation team was a strong selection bias in interviewees. Given time and budget limitations, the individual PFs were asked to help arrange individual and group interviews. Interviewees were thus purposefully sampled and likely biased towards providing positive reflections as the PFs had an incentive to avoid selecting those that were known to have had a negative experience. The evaluation team attempted to overcome this potential bias by triangulating interview responses with survey and secondary data. It should also be noted that the grants reviewed were classified by the individual PFs and that no classification key was available for the team to review. The evaluation team did not systematically review all the classifications and there is a possibility that they would have classified some differently had they had a standard definition to follow. Some of the grants could also fit into multiple categories and the team made subjective judgments on which single category was most representative of the grant’s focus. Instrumentation Effect The interview protocols were developed by the team during their first couple of days in-country. Although the protocols did not change significantly over the course the evaluation, certain questions and phrasing were refined throughout the interviews. While this potentially threatened the reliability of early interview responses, the evaluators are confident that the degree of consensus around the major findings highlighted in the report was not affect by this threat. External Threats to Validity “Pleasing the Interviewer,” Courtesy Bias, and other Experimental Effects As with most interview data collection, there was a possible courtesy or “pleasing the interviewers” bias in the responses of the interviewees. Interviewees were provided with a standard introduction to the evaluation before being asked any questions and were likely also informed of the evaluation’s purpose during the scheduling of the interview by the PFs. The interviewees were thus aware that they responses might influence the judgments of the evaluators and could be biased towards skewing their responses in the hopes of supporting their PFs, telling the evaluators what the interviewee that they might like to hear, or other potential biases introduced during this type of data collection. Evaluator Effect The evaluation design’s heavy qualitative focus also resulted in the potential for subjective judgments and the opinions of the evaluators’ influencing their data collection. The team tried to mitigate this risk by having at least two people present during interviews and also triangulating their interview data with survey and other secondary data.

XIII

ANNEX C â&#x20AC;&#x201C; LIST OF PEOPLE CONTACTED Name Alexis Loeber Andrei Kortunov Natalia Shuranova Masha Eismont Charles Maynes Svetlana Kurchenkova Elena Golovko Andrei Kortunov Sergey Golubev Nargisa Valamat-Zade Oksana Fodina Mikhail Troitskiy Vyacheslav Bakhmin Sergey Martyshenko Anna Koshman Elena Agranat Irina Ryazanova Alexis Loeber Mark Pomar Ms. Naila Hashimova Ms. Ilaha Rasulova Mr. Gurself Alivey Mr. Zaur Zamanov Mr. Kenan Shikhlinksy Ms. Gunay Zeynalova Tamilla Mammadova Vazgen Karapetyan

Mara Ayvazyan Suren Yeganyan Heghine Manasyan Anna Sargsyan Gegham Sargsyan Gayane Ghukasyan Samvel Manukyan Nairuhi Jrbashyan Kristine Antonyan Yulia Antonyan Zhanna Andreasyan Rouben Gevorgyan Alla Bakunts Deana Arsenian

Interviewees for Component I Title and Organization American Embassy/USAID/Moscow American Embassy/USAID/Moscow Director of Financial and Analytical Department, FNE Director of the Independent Print Media Program, FNE Senior Advisor, FNE Project Manager, FNE - Small Business Projects Project Manager, FNE - The World of Professions through Education President, FNE Project Manager, FNE - Community Universities FNE - Educational Block FNE - Educational Block Associate Director, MacArthur Foundation FNE - Mott Foundation FNE - SUAL Association of Independent Regional Publishers British Embassy FNE - CSPP Counsellor, EU-Russia Cooperation Programme US-Russia Foundation President, EPF Program Manager, Citizen Journalism E-Media Program, EPF Caucasus Research Resource Centers PF Political Officer, British Embassy BP Representative Youth Economic Development Program PF Associate Country Director, SATR Program Manager, Arm-Az Program Manager, SCCP Program Manager Program Development Manager Finance Manager, EPF and CRRC Armenia CRRC Armenia Director Program Manager, Fellowships and CSW, CRRC NDI Yerevan Office Expert, lecturer at YSU Expert, Researcher Expert, Researcher Lecturer, YSU Lecturer, YSU, CRRC-Arm Fellow CRRC-Arm Expert, Social Cohesion Project Trainer, lecturer, YSU DG Portfolio Analyst, UNDP Armenia Vice President, International Program, and Program Director, Higher Education in Eurasia,

Country Russia Russia Russia Russia Russia Russia Russia Russia Russia Russia Russia Russia Russia Russia Russia Russia Russia Russia Russia Azerbaijan Azerbaijan Azerbaijan Azerbaijan Azerbaijan Azerbaijan Azerbaijan Armenia

Armenia Armenia Armenia Armenia Armenia Armenia Armenia Armenia Armenia Armenia Armenia Armenia Armenia United States

XIV

Varya Meruzhanyan Zaqar Boyajyan Isabella Sargsyan Gevorg Ter-Gabrileyan Haykanush Iskandaryan Anahit Khachatryan Gayane Grigoryan Anna Poghosyan Marianna Grigoryan Vahe Nersesyan Vahagn Antonyan Artashes Arakelyan Maria Karapetyan Lusine Sargsyan Karine Aghajanyan Naira Sultanyan Olya Azatyan Gareth Wynn Owen Arthur Ghazaryan Tevan Poghosyan Boris Navasardyan Sorin Mereacre Ana Olaru George Zarubin Dato Jashiasvili Azalia Dairbekova Bermet Tuibaeva Anida Shergalieva Cholpon Dootalieva Dinara Toktosheva Bektur Iskenderov Galina Gaparova Aijan Toktosheva Tamara Djoldosheva Dinara Musabekova Nazgul Asangozhoeva Alina Saginbaeva Samagan Aitymbetov Nazarali Pirnazarov Erkin Konurbaev Aijan Urustamova Erika Bayalieva Gulzada Duishebaeva Nurgazy Tashiiev Talant Toktaliev Miraida Batkulova Aidai Tulemisheva Amantai Toktorov Maksat Abdukarimov

Carnegie Corporation of New York SATR Project Manager Deputy President of UNIBANK Armenia-Azerbaijan Program Manger Country Director, SATR Chief of Party EPF grants manager, Arm-Az grants manager EPF AOR, Project Management Specialist USAID/Armenia Deputy Program Officer Civic Journalist Journalist Animator Film Director Civic Journalist Civic Journalist Photographer/Journalist Blogger/Journalist Project Manager/Political Officer, UK FCO Project Support Officer, UK FCO Deputy Head of Mission,UK FCO SATR, Business Strand Coordinator, Union of Manufacturers and Businessmen of Armenia Executive Director of International Centre for Human Development, SATR Program Manager President, Yerevan Press Club, SATR Media strand Coordinator President, EEF Director of Finance, EEF President, EPF CFO, EPF President, EFCA PM Senior GCM, Central Asia News Service Chief Accountant Executive Director, EFCA Kloop Kloop Youth of Osh Project Director, EFCA Executive Director, EFCA Central Asia News Service Project Director, Central Asia News Service Editor, Central Asia News Service Journalist, Dushanbe USAID/CAR Financial Officer, KICB Financial Officer, Kompanion EdNet Career Center of Osh State university Career Center of Kyrgyz Economic University International Academy Credit Borrower Intern at Begemot Intern Kyrgyz law academy

United States Armenia Armenia Armenia Armenia Armenia Armenia Armenia Armenia Armenia Armenia Armenia Armenia Armenia Armenia Armenia Armenia Armenia Armenia Armenia Armenia Moldova Moldova Moldova Moldova Moldova Kyrgyzstan Kyrgyzstan Kyrgyzstan Kyrgyzstan Kyrgyzstan Kyrgyzstan Kyrgyzstan Kyrgyzstan Kyrgyzstan Kyrgyzstan Kyrgyzstan Kyrgyzstan Kyrgyzstan Kyrgyzstan Kyrgyzstan Kyrgyzstan Kyrgyzstan Kyrgyzstan Kyrgyzstan Kyrgyzstan Kyrgyzstan Kyrgyzstan Kyrgyzstan

Urmat Jumashaliev Dinara Mamytova Aigerim Joldosheva Cholpon Dootalieva Edil Kalashev Kubat Kanimetov Kubat Rahimov Roman Pogojev Askar Beshimov Myles Smith Umida Khaknazar Nurbek Sariev Bagdagul Abdikarimova Dildora Khamidova Mirbek Babakulov Leilek Daanishmani Gazbek Kalmuratov Ravshan Abdullaev Parviz Akobirshoev Rinat Khusnullin Farangis Mamadieva Audrey Janin Sharipov Mahmadnazir Ilkhom Makhkambaev Said Sharipov Nazarali Pirnazarov Vadim Sadonshoev Ben Gibson Rinat Khusnullin Ravshan Abdullaev Anna Shukurova Nurali Shukurov Lutfiddin Tuychiev Nigina Alieva Kristina Bagramyan Victoria Marchenko Yuriy Piskaluk Maryna Nyxhnyk Trond Moe Elena Sayenko Viktor Karpenko Victor Liakh Oksana Mr. Turo Matilla Yaroslav Yurtsaba Elena Sayenko Anna Schiraya

Internal control manager of Atalyk Group Consultant at Jogorku Kenesh PM, EFCA Chief Acct, EFCA National WTO Expert Country Manager Contractor Contractor Contractor

Head of Osh office CFO, EFCA Project Manager, EFCA Partner/mobilizer Partner/mobilizer Partner/mobilizer Executive Director, EFCA Financial Manager, EFCA Program Manager, EFCA Program Manager, EFCA Development Reporting and Evaluation Manager, EFCA President of American Chamber of commerce (AmCham) Executive Director, AmCham Former EFCA's financial manager (2006-2009), AmCham Permanent Representative of CA-news in Tajikistan, member of TRCN Former EFCA's program manager for the project State Department, liaises with AmCham Program manager, EFCA Executive Director, EFCA Executive Director, Union of professional consultants of Tajikistan General Director, Union of professional consultants of Tajikistan Entrepreneur Communications, The World Bank Tajikistan Program Manager Program Manager, Civil Society and Media, UNITER/Pact, USAID Program Manager Regional Development Manager Board Member Program Manager Evaluation Officer Executive Director Regional (Ukr, Bel, Mold) Programs Second Secretary, Embassy of Finland National Project Manager, OSCE Program Manager, EEF PR and Media Connections

Kyrgyzstan Kyrgyzstan Kyrgyzstan Kyrgyzstan Kyrgyzstan Kyrgyzstan Kyrgyzstan Kyrgyzstan Kyrgyzstan Kyrgyzstan Kyrgyzstan Kyrgyzstan Kyrgyzstan Kyrgyzstan Kyrgyzstan Kyrgyzstan Kyrgyzstan Tajikistan Tajikistan Tajikistan Tajikistan Tajikistan Tajikistan Tajikistan Tajikistan Tajikistan Tajikistan Tajikistan Tajikistan Tajikistan Tajikistan Tajikistan Tajikistan Tajikistan Ukraine Ukraine Ukraine Ukraine Ukraine Ukraine Ukraine Ukraine Ukraine Ukraine Ukraine Ukraine Ukraine

XVI

Roland Kovats Irina Bilous Susanne Drakborg John Riordan Andriy Nesterenko Dinu Toderascu

COP COP Country Manager, World Childhood Foundation USAID Program Officer, Black Sea Trust

Ukraine Ukraine Sweden Ukraine Ukraine Romania

XVII

ANNEX D – LIST OF EF’S GENDER INDICATORS 2002

2003

2004

2005

2006

2007

2008

2009

2010

Total

199,692

97,511

276,544

166,461

100,652

75,530

48,724

63,025

19,507

1,047,646

61,025

46,237

56,596

54,688

33,217

37,554

30,112

24,558

110,100

454,087

28,872

10,213

25,110

4,130

2,754

3,870

14,335

2,681

113,504

1,729

1,126

1,706

1,727

1214

631

1586

611

299

10,629

7,420,000

3,029,398

5,272,214

5,379,768

1,563,246

1,524,184

1,383,097

201,292

306,055

26,079,254

178,532

5,211

8,098

9,893

3,671

12,898

8,107

1,980

2,203

230,593

403

646

266

138

269

192

1074

1,001

11,483

254

738

15,924

21,539

398 451

109 2,872

806

308

576

3,758

2390

3,190

3,990

5,561

1,556

561

1,570 53

46,458

47,428

152,802

83,717

52,171

42,661

24,511

29,896

7,913

487,557

153,234

50,083

123,742

82,744

48,481

32,869

24,213

33,129

11,594

560,089

37,672

23,515

28,470

25,816

17,859

22,219

14,241

10,814

54,760

235,366

23,353

22,722

28,126

28,872

15,358

15,336

15,871

13,744

55,340

218,722

47%

48%

XVIII

ANNEX E – GRANT AGREEMENT SOW Background One of the major lessons of the past decade of Western assistance in the former Soviet Union is that reform cannot be sustained from the top alone. Some of the most significant changes in the region over the past ten years have been wrought by private citizens at the local level. At this level, experimentation can take place and the impact of actions taken is readily seen. People develop a personal stake in change because they see that it can benefit their lives and livelihoods. Local successes have been so significant that they should now be sustained and expanded. It would be a historic miscalculation if the West followed the advice of some Western observers and decided that since government-to-government and large-scale international programs have not fulfilled initial expectations, the former Soviet Union is an area where Western assistance programs cannot work effectively, and, consequently, should be terminated. On the contrary, the experience of the past ten years strongly suggests that the West should increase its engagement of the region -- although not necessarily the size of its financial commitment. And, most importantly, it should re-orient its support to focus on bottom-up, grass-roots activity. The Eurasia Foundation was created explicitly to deliver a bottom-up, grass-roots grants program in Eurasia. The impetus to create the Eurasia Foundation was based on a joint U.S. Department of State and USAID initiative. The foundation was proposed in 1992 as an autonomous, private entity to further U.S. development and policy objectives in the former Soviet Union. It began its technical assistance program in May 1993 with an initial grant of $75 million from USAID and has maintained a core program of approximately $25 million a year from USAID ever since. In 1997, the foundation received a second grant from USAID of $104.35 million to continue its programs at a comparable level. Since 1995, the foundation has raised over $30 million from non-U.S. government sources to provide additional support to its programs. The Eurasia Foundation is a publicly-funded, privately-managed foundation whose purpose is to support programs at the grassroots level designed to help the citizens of Eurasia build local institutions that will serve their own best political, social, and economic interests. Today, the Eurasia region is moving into its second phase of transition but at uneven rates. The first phase of macrolevel state building and of the dismemberment of the command economy is not fully complete, and strengthening of democratic political institutions remains critical. The second phase of micro-level political consolidation and incremental economic growth must now begin. On the individual country and local levels, the reform agenda remains extensive. Governments in the region face a daunting set of challenges:       

improving the overall business climate; stimulating domestic production and creating new jobs, primarily through the growth of small business; raising general living standards outside the capital cities; improving legislation, enforcing existing laws, and increasing the effectiveness of government, especially at the local level; combating corruption, not just at the top but at the level that affects the ordinary citizen; and central to all of the above, expanding opportunities for citizen participation in political and economic decision making, especially by affirming civil rights and supporting healthy democratic mechanisms to buttress the role of citizens in a democratic society and ensure oversight, transparency, and accountability of all authorities.

Since the early 1990s, small groups of private citizens have begun to take an active and constructive role in the postSoviet transformation. They have begun to engage directly with authorities to shape the civic and economic landscape, and, in some cases, to provide crucial social services on the local level. Citizens are also more actively asserting their proper role in democratic society as the source of political power. This means citizens and their elected representatives can and do assert their interest in transparency and accountability. At the same time, the August 1998 financial crisis had a silver lining not only in Russia but elsewhere as well, given Russia's influence on regional economies. The sharp evaluation of the ruble gave a boost to local producers, now shielded from foreign competition.

XIX

In some parts of the region, small-scale entrepreneurs are still battling against legal and financial barriers. In other places, such entrepreneurs are for the first time beginning to generate modest amounts of financial capital that can be invested in local private initiatives. New ventures are gradually beginning to replace some of the old industrial enterprises, sclerotic government structures, and outdated institutions. But reform efforts are still thinly rooted. Indigenous grassroots reform efforts urgently need support in order to accelerate and strengthen the push for reform. Purpose The purpose of continued USAID funding for the Eurasia Foundation is to support an independent institution that will combine public and private resources to advance economic and political reform in Eurasia, through small grants, loans, and multi-donor technical assistance programs. The foundation will reach out to a wide range of diverse groups in Eurasia with both open-door and targeted assistance. It will seek to identify and, to the extent possible, fund the most promising and sustainable Eurasian institutions and programs and those initiatives established in other U.S. Government- funded efforts. It will operate largely through small grant and lending programs, but may formulate other innovative and effective mechanisms on occasion, such as establishing consortia or cooperative projects with other donors, to give special attention to strategically important challenges. Foundation Experience A board of private U.S. citizens incorporated the Eurasia Foundation, which was created with the help of funding by the U.S. government nearly nine years ago. The foundation has proved remarkably successful in managing professionally in a highly challenging environment. (Note that the Eurasia Foundation was incorporated in the District of Columbia on April 29, 1992. The Foundation received its 501 (c)(3) nonprofit status ruling from the Internal Revenue Service on December 15, 1992. The first USAID grant agreement was awarded on May 18, 1993. The second USAID agreement was awarded on September 30, 1997.) In addition to its grantmaking, the foundation has also assumed a leadership role in identifying and working with non-U.S. Government funding sources to create consortia and to respond to important, but under-assessed challenges - for example, economics education, and the financial viability of regional media. Outside evaluations, one in 1995 and a USAID-funded and directed evaluation in 2000, have given high marks to the foundation for its programs. The USAID-directed evaluation, which was delivered in February 2001, noted that the foundation has a solid record of accomplishment and, subject to satisfactory progress in correcting certain weaknesses identified in the evaluation -- a process well under way -- deserves further support from USAID. Program Areas The Eurasia Foundation 1 s program aims to address three goals that correspond broadly to USAID strategic objectives: 

private enterprise development o Goal: Accelerated development and growth of private enterprises (SO 1.3 and SO 1.4)  Civil society (including media and nongovernmental organizations) o Goal: Increased, better informed citizens participation in political and economic decisionmaking (SO 2.1 and SO 2.2)  public administration and policy  Goal: More effective, responsive, and accountable local government (SO 1.2 and SO 2.3) As the only private foundation that promotes both economic and democratic development in twelve countries of Eurasia, the foundation is unique in its ability to sponsor cross-programmatic projects that address two and often three of these objectives. Such initiatives would, for example, aim to reduce corruption by involving private business associations in dialogue directly with local governments and supported by reports through the media. Crossprogrammatic projects correspond to USAID SO 4.2. Contributions to actual 80s will vary country-to-country depending upon the foundation 1 s programs and the development needs and demands of the specific country. The foundation will coordinate with USAID in Washington and the local USAID missions in planning its programs. Experience from countries across Eastern Europe and Eurasia suggests that the most advanced economic reformers have also made the most progress in consolidating democracy. At the same time, the foundation believes that the most efficient means to address the three goals noted above is to improve the region 1 s human resources, financial

resources, and enabling environment. For each of the goals above, the foundation will invest in these three functional areas: Human resources. A democratic society with a modern economy requires people with forward-looking vision and the appropriate skills. The foundation promotes the establishment of international caliber programs to train entrepreneurs, economists, journalists, nonprofit professionals, public servants, and legal practitioners, among others. Educational institutions and professional associations can improve skill levels, heighten ethical and professional standards, and promote appropriate legal and regulatory frameworks. International partnerships transfer necessary skills, encourage educational and professional development, and help integrate citizens into the global community. Financial resources. Capital is needed for businesses to grow, for nonprofit organizations to operate, and for local governments to function. The foundation works to increase local sources of funding for small businesses and private organizations as well as to promote sound financial structures in the public sector. Through programs with financial intermediaries such as banks, credit unions, cooperatives, and microcredit organizations, the foundation helps infuse capital into the small business sector while fostering sound professional and ethical business practices. The foundation also supports financial skills training in both the public and private sectors, and promotes volunteerism and philanthropy as mechanisms for nurturing self-sustaining nonprofit organizations. Enabling environment. Democratic societies cannot fully form without strong safeguards for civil rights, and healthy democratic mechanisms that ensure that citizens can demand accountability and transparency from their governing bodies. And a market-based democracy cannot flourish without appropriate policies, ethical practices, and access to information in a law-based environment. The foundation seeks to improve the enabling environment by promoting rational laws and policies, by supporting civil rights and basic freedoms, by reducing barriers to the entry of new businesses, by encouraging responsible and independent media to contribute to public discourse, by supporting reform-minded local governments and NGOs, and by supporting efforts to eliminate corruption in business and public affairs. In Soviet times, the rule of law did not exist. In a modern democratic society, sound laws should guide market relations, limit arbitrary government power, and protect civil liberties. Otherwise, the human and financial capacities within the host countries will not reach full potential. Program Implementation The foundation

meets its programmatic

goals through several instruments. These include:

ď&#x201A;ˇ

Small "Open Door" Grants to NIB Organizations. The majority of the foundation's grants are awarded in response to needs identified and solutions presented by local institutions. As one of the rare institutions operating in a demand-driven manner at the grass-roots level, the foundation will continue to support opendoor grant making as essential to the reform process. Unsolicited applications are a barometer of local buy-in for change. As civil society develops and organizations demonstrate an increased capacity to lead reforms at the regional or national level, the foundation will gradually increase the percentage of its support in favor of sustainability of these organizations and guided grantmaking, as noted below. The balance between reactive and proactive grantmaking will be decided on a country basis and approved annually by the Board of Trustees.

ď&#x201A;ˇ

Larger Partners hip Grants to Non â&#x20AC;&#x201C;NIS Organizations . Partnership grants are awarded to a U.S. grantee or other non- Eurasia organization that provides technical assistance to a Eurasia partner organization on a program within the foundation's mandate. The purpose of partnership grants is to build capacity in the NIS for international caliber programs. To ensure the success of the project, partnership applicants will either have a record of cooperation or there will be a strong demonstration of demand from the local partner for cooperation with a non-Eurasia partner invited proposals. An invited application is solicited by the foundation to meet specific programmatic goals. Invited applications advance a field through a decisive concentration of funding with a multiplier effect, such as training-of- trainers, replication of a successful program, catalyzing a new industry, or stimulating new services. As a prime tool in the foundation's country strategy, invited applications are included in the operating plans of each grantmaking office. They are governed by internal policies to ensure a competitive and transparent selection process. Beginning with this Grant, they will be monitored on a country basis and recorded as proactive grantmaking.

XXI



Competitions. The foundation may award small grants through competitions that are managed primarily by the field offices. Proposals are either solicited through a publicly announced Request for Applications or a restricted list of organizations. Thematic topics contribute to the foundation's strategic goals, are included in the field office's operating plan, and complement USAID activities. Competitions often involve cofunding from non-USAID sources, as well as provision of non-grant assistance such as training and networking opportunities. Recent examples include a business school partnership program in Belarus and anti-corruption campaigns in Russia, Georgia, and Armenia. Grants generated through competitions are monitored and recorded as proactive grantmaking.



Recoverable Grants. On an exceptional basis, the foundation may award "recoverable" grants for projects that may generate sufficient revenues to return a portion or all of the grant funds to the foundation. Recycled funds would be added to the grant funds and used to support new grants. Recoverable grants must be consistent with the foundation's grantmaking criteria.



Cross-Border Programs. The foundation's independence and reach make it possible to support experimental projects that operate across borders. In the Caucasus, for example, the foundation has launched the South Caucasus Cooperation Program to encourage cross-border cooperation among Armenia, Azerbaijan, and Georgia. During the Grant period, the foundation intends to increase the sharing of knowledge across borders by linking like-minded organizations through networking or other projectrelated activities. At present, no other institution has the capacity to deliver programs across borders on a large scale in the politically volatile Caucasus and other areas like it.



Foundation Projects. In cases where the foundation can play a vital role in economic and democratic reform in Eurasia by actively implementing or manging a program instead of through sub-recipients, the foundation will establish a project. Such projects will often bring groups of non-U.S. Government donors together to address commonly identified but neglected reform challenges. Current examples of such programs include the Media Viability Fund (MVF), the Economics Education and Research Consortium (EERC), and the Small Business Loan Program (SBLP). Any new projects will support USAID SOs discussed in Section D, Program Areas. In the past, such projects have enabled the foundation to leverage significant funds from non-U.S. Government sources such as foreign governments, private individuals, foundations, and corporations. In undertaking projects, the foundation seeks to incubate institutions that can ultimately be launched as independent and self-sustaining operations. The following are the three projects that the foundation currently implements: o

Media Viability Fund (MVF). The MVF is a joint effort of the Eurasia Foundation and the Media Development Loan Fund (MDLF) to promote a financially independent news media. The program provides loans to independent regional media for equipment (through MDLF) and technical consulting in financial and media management. Economics Education and Research Consortium (EERC). The EERC promotes increased capacity at world-class levels in economics through teaching and research and their connection to sound federal policy. In Russia, the EERC provides small research grants and a series of complementary seminars, publications, and practical application of research. In Ukraine, the EERC administers a Master's in Economics program at the University of Kyiv-Mohyla Academy. Small Business Loan Program (SBLP). The Eurasia Foundation has developed business loan programs in selected regions to supplement technical assistance and training available to small private businesses by providing access to capital on appropriate terms. These programs seek to establish local models of market-based, sustainable lending in participating financial sector institutions. The foundation's lending activities place a strong emphasis on providing training and technical assistance to borrowers and to participating financial institutions alike.

Program Income In the course of carrying out its activities under the Grant, it is anticipated that the Eurasia Foundation will be

XXII

generating program income. By far the most significant source of program income for the Eurasia Foundation is loan repayment under the Small Business Loan Program (SBLP). The Eurasia Foundation adds the earnings from the SBLP attributable to returns of loan principal back to its loan pool, thereby enabling it to make additional loans. In the event of a loan loss, the foundation reduces its available loan pool by that amount. In order to meet loan losses, the foundation uses earnings attributable to returns of interest on loans made under the SBLP. Since the inception of the program, loan losses have totaled 4.68 percent prior to recovery through sale of collateral. In accordance with the preceding, program income to the Eurasia Foundation may include the receipt of principal on SBLP loans and interest derived therefrom, provided that such program income may only be used for additional lending activities under the SBLP; and provided further that such program income as may be attributable to return of interest (but not return of principal) on loans made under the SBLP may be used to meet loan losses under the SBLP. (Under no circumstances shall Grant funds be used to cover loan losses.) Program income derived from activities other than the SBLP may be used to further eligible objectives of the Grant, as provided by 22 CFR Part 226, § 226.24 (b)(1) Endowment/Trust As the Eurasia Foundation seeks to diversify its funding sources and further leverage U.S. government funds by attracting non-U.S. government financial support, an effort has been launched to obtain an endowment or trust consisting of matching public and private funds. An endowed Eurasia Foundation will ensure that the U.S. government remains engaged with the former Soviet Union in a way that reduces political exposure and is in line with calls by the U.S. Congress to modify its operating practices around the world. Until the endowment/trust has been assembled, the Eurasia Foundation would continue to depend primarily on annual allocations from USAID under this Grant, as well as expected significant funding from other sources. Accordingly, during the life of the Grant USAID will consider how some of the funds that would be obligated under this Grant could be directed toward an endowment/trust. The actual mechanism and operational details will be worked out over the course of the Grant. Strategy Development Each year, in consultation with USAID, grant offices will prepare a rolling, three-year program strategy that sets out annual operating plans against an evaluation of past performance and progress toward future goals. The grant office strategies will fall broadly within the foundation's mandate and strategic plan as well as the USAID country mission objectives. In preparing its program strategy, offices will identify a smaller number of fields of interest within the foundation’s mandate where it will guide grantmaking for more focused results. In identifying fields of interest, the grant staff applies knowledge of country trends, needs, and experience in both the foundation’ s grant portfolio and in programs funded by other donors. Generally, each office selects at least one field of interest in each of the foundation’s three program areas: private enterprise development, public administration and policy, and civil society. In some cases, programs will take a cross-programmatic approach that engages more than one segment of society, such as the media, public officials, business groups, or academia. These strategies form the basis for annual budget preparation, discussion and approval or modification. The Board of Trustees of the Eurasia Foundation approves the annual budgets and the strategic plans of each office. This process will be annually coordinated with appropriate USAID field missions 1 staff to ensure that the foundation's annual operating plans address the relevant mission SOs. The annual coordination process will be conducted with reference to the development of USAID's Annual Report (formerly R4) and strategic plan. The program plan discussed above will also list indicators that can be used to gauge each Eurasia grant office's success in making progress toward meeting the foundation's planned objectives. Reporting The Eurasia Foundation is required to provide an annual report to the CTO at USAID/Washington as defined by the Schedule of the Grant. The report shall provide actual accomplishments, lessons learned and results according to the indicators identified in the annual strategic planning process described in Section H. ("Strategy Development") above. The report will consist of a narrative regarding the results of foundation programs vis-a-vis USAID Strategic Objectives, including relevant information regarding the foundation's program implementation mechanisms mentioned above, expenditure summaries, grant lists and loan lists categorized by Strategic Objective disaggregated by country, and the foundation management benchmarks and results- monitoring indicators set forth in Section J. ("Evaluation") below (and any additional benchmarks and/or indicators as may be agreed upon during the period of

XXIII

the Grant) . In addition to annual reports, the foundation will provide a semi-annual report to the CTO at USAID/Washington consisting of a summary of progress towards addressing indicators in the annual operating plan, a complete grant list, and the benchmarks and results- monitoring indicators referenced in the previous sentence.

XXIV

ANNEX F â&#x20AC;&#x201C; SPENDING ACROSS SOS BY COUNTRY

Central Asian Republics $1,400,000 $1,200,000 Civil Society

$1,000,000 $800,000

Private Enterprise Development

$600,000 $400,000

Public Administration and Policy

$200,000 $0 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010

Ukraine $1,600,000 $1,400,000 $1,200,000 $1,000,000 $800,000 $600,000 $400,000 $200,000 $0

Civil Society Private Enterprise Development Public Administration and Policy 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011

Russia $3,000,000 $2,500,000

Civil Society

$2,000,000 $1,500,000

Private Enterprise Development

$1,000,000

Public Administration and Policy

$500,000 $0 2001 2002 2003 2004 2005 2006 2007 2008

XXV

Moldova $500,000 $400,000

Civil Society

$300,000 $200,000

Private Enterprise Development

$100,000

Public Administration and Policy

$0 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011

Belarus $300,000 $250,000

Civil Society

$200,000 $150,000

Private Enterprise Development

$100,000

Public Administration and Policy

$50,000 $0 2002

2003

2004

2005

2007

2008

2011

Caucasus $5,000,000 $4,000,000

Civil Society

$3,000,000 $2,000,000

Private Enterprise Development

$1,000,000

Public Administration and Policy

$0 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010

XXVI

ANNEX G â&#x20AC;&#x201C; INTERVIEW GUIDES Draft EF/PF Senior Staff Interview Guide (Presidents, PF Board Members, CFOs, Senior POs and PMs) Eurasia Foundation Senior Staff Guide: Moldova Meetings and PF Draft by: Richard N. Blue, Social Impact. EF Evaluation Team Leader Date: September 23, 2012 I.

Background and Purpose statement

Good morning/afternoon. Our firm, Social Impact, is working with the United States Agency for International Development to assess the relevance, effectiveness, and sustainability of their assistance to the Eurasia Foundation. We would like to speak with you as part of our ongoing evaluation to hear about your experience and impressions of working with EF. II.

Confidentiality Statement

This survey guarantees respondent confidentiality and your participation in this survey is voluntary. All data will be used in an aggregate form that will make it impossible to determine the identity of the individual responses. Access to raw data will be tightly restricted to only those individuals directly involved in data analysis. III.

Respondent SES data

Country of Origin__________________

Name of Organization __________________________

Current Position and responsibilities (probe)___________________

How long with EF/PF (# years)_________________

Gender: M/F

Experience with other organizations prior to EF-PF? (Open ended)

Familiarity with current funding and programs? (Probe)

a) Interviewer rating respondent's familiarity with EF-PF:

IV.

Very Familiar with all aspects

Familiar with Specific aspects (e.g., finance)

Familiar with some programs

Familiar in a general way but not in detail (e.g. Board member)

Not very familiar, new to PF. Questions

Background IV.1

How would you describe the PF's primary Mission in your country?

Impact and Effectiveness of PF (Question for PF leadership - if Respondent not in leadership, go to V.) XXVII

IV.2

Please describe your major program goals and strategy overall.

IV.3

How successful has your PF been in achieving the program goals and objectives since it became an independent partner? (Read and circle response) A. Very Successful B. Somewhat Successful C. Limited or not very successful D. Not successful

IV.4

What are the major reasons for this success, if so?

IV.5

What are the major challenges facing the PF in realizing your goals?

USAID Strategic Objectives and Programming

The USG through USAID has provided $XXX millions to support EF's contribution to 3 main USAID Strategic Objectives, Private Enterprise Development (SO1), Civil Society Development (SO2) and Public Policy and Public Administration development at the local level (SO3). Partner Foundations have further refined these Objectives into specific programs, such as media, youth, etc. V.1. From the perspective of your PF, how would you rate the effectiveness and impact of each of these three broad programming areas since your PF was established? (scale of 1 to 5, with 5 being most effective) a) SO1. Enterprise Development: _____________________________ b) SO2. Civil Society Development_____________________________ c) SO3. Public Policy and Public Administration __________________

V.2. For the most effective programs, what are the most important causes related to strong positive impact? (Probe) V.3. Which areas have been most difficult and why? (Probe) V.4

From your perspective, in the future will your PF continue to program in the same general areas, or do you anticipate moving into other programming areas? (Probe and circle one.) a) Will continue same general programs for foreseeable future b) Will continue but refine the program objectives to fit our country needs and opportunities c) Will adjust to priorities of our future donors and supporters d) Will focus on those areas most relevant to our society and government's interests.

VI.

Relationship with Eurasia Foundation Washington and Network 1. From your PF (or EF) perspective, how would you describe your current relationship with EF Washington?

What are the positive sides of the relationship between your EP and EF

XXVIII

Washington? What are the negative sides of the relationship between your EP and EF Washington? (Probe) 2. Please tell which of the following statements comes closest to your view about the EF/DC relationship in the next few years as it relates to present and future success of your PF. (read and ask KI to choose one) a. Very Positive ,helpful and necessary to our future b. Generally helpful and still important to us c. Somewhat helpful but not essential for our future d. Useful, but increasingly irrelevant to our future e. Not helpful and increasingly a threat to our future. f.

Other____________________________________

3. What are the major benefits your PF has gained from associating with EF and the Network? (Probe and rate as below)

(Interviewer circles each one mentioned based on open ended response) a. Assistance with organization performance and capacity building (mentions Capacity Mapping tool/exercise) b. Assisted with establishing financial management, audit, and financial accountability procedures used by our PF. c. Assisted us in using USAID core grant for operating expenses. d. Assistance with our external donor fundraising and proposal development e. Networking opportunities with other PFs is useful for identifying common problems/solutions. f.

We gain a certain amount of international status and respect for being in the EF association.

g. Other:_________________

VII.

Sustainability of Partner Foundation in future

1. Turning to the future of your PF over the next 3 to 5 years (Russia, Armenia, etc,), are you confident that you will be able to attract donor support for your foundation's programs? (Probe and ask Respondent, are they) a. Very Confident b. Fairly Confident c. Somewhat Doubtful d. Very doubtful e. Other:_________________

XXIX

2. Where do you think the donor support will come from in the future? (probe, read and circle choices mentioned) a. Local private companies and foundations b. Our government c. Foreign government assistance programs d. USAID Mission e. US Private foundations (e.g., Mott, others) f.

Multi-lateral organizations (WB, UN, others)

g. Other:_________________ 3. (For Leadership) Does your PF have in place a Fund raising and Development strategy and program? a. Yes b. No c. Working on it d. Don't know 4. Are you able to convince other donors to also allow you to charge your overhead and operating costs as a part of the grant funding? (read choices and ask KI to select one closest to their view) a. Yes, almost always b. Yes, most of the time c. Sometimes, but only if we make a strong case d. No, donors don't like to pay for overhead e. Other:_________________

Sustainability: Organizational Capacity and Procedures The Eurasia Foundation has worked with each of the PFs to strengthen organization capacity, introduce international standards for financial management and accounting, helped to strengthen the Board of Directors and other organizational policies and procedures. 1. Have these organizational procedures become institutionalized in your PF, or do you expect substantial changes to occur as you become increasingly independent and "local"? (probe) Read choices and ask KI to select one. a. We are a strong organization and see little reason to change. b. We will retain international standards, audits, and appropriate accountability for financial management, but will continue to evolve our own programming and reporting procedures.

XXX

c. We will maintain high standards but adjust in accordance with our own countries laws and practices. d. While EF's capacity building assistance has been useful, we will probably move away from many of these practices as they are simply too difficult and costly to retain in the future. e. Other:_________________ 2. Is EF-DCâ&#x20AC;&#x2122;s Capacity Mapping Initiative (CMI) successful in accurately measuring the organizational capacity of partner foundations? 3. Is CMI clear, understandable, easy to use? How often do you use it in your PF and how the process is organized and who is involved? 4. How do you follow up CMI results, how recommendations are implemented and what actions are taken? VIII.

The Foundation as a model for providing assistance

USAID has asked us to examine the advantages and disadvantages of the foundation model as a means for delivering foreign assistance. As a Foundation leader/officer, how would you respond to this question? 1. What would you say are the advantages? 2. What are the dis-advantages? 3. Are there circumstances in which the foundation approach would be better than a standard USAID type Mission program?

Are there other issues, observations or information you would like to share with us, or questions you may want to ask of us about this assessment? Thank you for your time and your full cooperation. END.

XXXI

Grantee/Beneficiary FGD Guide Eurasia Foundation Evaluation Grantees/Beneficiaries (Component One) Date of Interview ____________ In Person or Phone____________ Participants: Total___________ (___M/____F)

IX.

Background and Purpose statement

Confidentiality Statement (read to KI)

Knowledge of Eurasia Foundation

1. What type of assistance/support did you receive from FNE/EFCA/EPF/EEF?

2. How would you describe your experience working with FNE/EFCA/EPF/EEF? a. Was the grant application process clear and easy to negotiate? Please describe any problems: b. During grant implementation, how was your relationship with the EF office? (Probe for positive and negative occurrences) c. How would you describe the reporting relationship with EF? d. Did EF staff provide any assistance that helped your organization beyond the grant making? e. What kinds of actions/approaches might have been more helpful in carrying out your program responsibilities? f.

How would you rate the relationship?

XXXII

Very positive and helpful

Generally positive and helpful

Somewhat positive and helpful

Not at all positive and helpful

3. Have you worked with other donors? If yes, how did the experience compare to working with FNE/EFCA/EPF/EEF? a. Ease of application and administration? b. Helpfulness of Grantor staff c. Overall effectiveness in terms of achieving project/program objectives? 4. What was the outcome of your grant/project? 5. Did you face any challenges/constraints in implementing your program? Did/could FNE/EFCA/EPF/EEF have done anything to help you overcome these challenges/constraints?

Thank you for your time and willingness to discuss these issues.

XXXIII

EF-DC SKI Guide

Eurasia Foundation Evaluation Special Guide for EF Headquarters Staff Date of Interview ____________ Introduction: Confidentiality Protocol •

•

The evaluation team will collect information on individuals’ names, organizations, and positions. A list of key informants will be made available as an annex to the final evaluation report, but those names and positions will not be associated to any particular findings or statements in the report. The team may include quotes from respondents in the evaluation report, but will not link individual names, organizations, or personally identifiable information to those quotes, unless express written consent is granted by the respondent. Should the team desire to use a particular quote, photograph, or identifiable information in the report, the evaluators will contact the respondent(s) for permission to do so. All data gathered will be used for the sole purposes of this evaluation, and will not be shared with other audiences or used for any other purpose.

Respondent Background: 6. 7. 8. 9.

EF Position: Years with EF: Gender: Principle Responsibilities at EF (Describe):

Questions: 1. By 2000, EF had developed several “flagship” institution building projects including a. Economics Education and Research Consortium b. Media Viability Fund c. Small Business Loan Program 12 years have passed. How would you describe the status, strengths and weaknesses or issues with each of these initial projects? Has EF developed new “flagship” projects? Yes No Please explain:

2. Looking over the period 2002-2006, and then 2006 to present, how would you assess the extent to which EF programs have contributed to USAID’s 3 main Strategic Objectives? Let’s start with a. Accelerated Development and Growth of Private Enterprise

b. Increased, better informed citizens’ participation in political and economic decision making.

XXXIV

c. More effective , responsive and accountable local government 3. EF has established 5 EF Partner Foundations, beginning with Moscow in 2004, EFCA in 2005, EPF (Caucasus) in 2007, Ukraine 2007, and Moldova 2010. How would you describe the success, strengths and weaknesses of each of these PFs from your perspective? a. Russia FNE b. EFCA (CAR) c. EPF (Caucasus) d. Ukraine e. Moldova 4. We’ll be reviewing funding sources and talking to some of the donors who contribute to various PFs, but how would you rate the “sustainability” of each of the PFs as you look out over the next 3 to 5 years? a. Russia (especially in the light of new law regarding foreign donations) b. EFCA (the only PF to be short of funding target) c. EPF d. Ukraine e. Moldova 5. As PFs rely more on funding from other sources in the future, how do you think this will affect the major program categories: Youth Engagement, Local Economic Development, Public Policy and Institution Building, Independent Media and Cross-Border Programs. Do you expect to see major changes in this agenda, especially in those states that regress to a more authoritarian political climate? 6. As you look to the future, how do you see the relationship between EF Washington and the PFs evolving? 7. Turning to NEE, the legal-political status is quite different, and the role of Ukraine PF is essential. How do you see the NEE program evolving over the next 5 years? What are the key opportunities and constraints that affect programming in this kind of political climate?

XXXV

ANNEX H – ELECTRONIC SURVEY

XXXVI

XXXVII

XXXVIII

XXXIX

XLI

XLII

XLIII

XLIV

XLV

XLVI

ANNEX I – INDIVIDUAL PF CASE STUDIES Case Study: FNE FNE is the oldest of the PFs set up after EF-DC decided on a new strategy for ensuring the long term viability and sustainability of its local offices. FNE has become a self-styled “social development agency,” which does not make grants through open competitions but is rather an organization promoting specific program objectives. FNE has developed a decentralized management structure with several major program areas, each led by a senior staff expert who has considerable autonomy for program development and management. Originally established as EF’s Russian grant making office, FNE’s relationship with EF continued from 2004 to 2012 through three main channels. First, EF retained a position on the board of directors of FNE. Second, EF provided technical assistance for organizational strengthening and financial accountability/management. Third, perhaps most important, the USAID core grant was apportioned among the emerging PFs including FNE. In the period 2004 to 2012, EF core grant funding accounted for 32 percent of all funding for FNE. The impact of this funding was substantially enhanced by the fact that the core grant was “unrestricted,” that is, not attached to any particular program. EF core grant funding paid for salaries and overhead when other donor sources were reluctant to do so. The USAID Core Grant ends in December 2013. EF has also engaged FNE through the establishment of an active network of EF Partner Foundations, which meet regularly to share experiences, solutions, and common problems. FNE interlocutors indicated that such sharing was useful and they would continue to benefit from their participation in the network. At the same time, no one thought the network was critical to the future success of the FNE in Russia. With regard to the board of directors, in 2012, after the untimely death of EF Trustee Sarah Carey, two EF Trustees continue to sit on the FNE Board: Randy Bregman SALANS, and Lawrence English, BARING VOSTOK. Both individuals have experience and investments in Russia. FNE has created a separate board of trustees which includes one American, Randy Bregman of SALANs international law firm. Advancing US Strategic Objectives As FNE’s program has evolved, its overall structure has taken on content of a Russian program, determined largely by two main factors: 1) what Russian sources are prepared to finance, and 2) what fits the issue agenda and vision of FNE. What foreign sources such as USAID, the Mott Foundation and the MacArthur Foundation are prepared to support has continued to be important, but increasingly less essential to the long term vision of FNE. The following excerpt from the FNE 2012 annual report demonstrates the breadth of FNE’s programming vision: SPHERES OF OPERATION • Implementation of complex regional and municipal socio-economic development programs • Modernization of the regional general education system, as well as primary and secondary vocational education systems • Management modernization at institutions of higher learning • Support of international activities by Russian educational institutions • Development of technology transfer systems at institutions of higher learning • Development of regional innovation support infrastructures

XLVII

• Development of youth initiatives • Support of local initiatives and community engagement in regional socio-economic development • Support of housing and utility system reforms • Coordination of migration processes • Development of conflict prevention systems at the regional and local levels • Support of small and medium-sized businesses in Russia’s regions and municipalities • Support of regional and municipal mass media organizations • Support of Russian civil society institutions in their international engagement endeavors88 The USAID final grant agreement with EF stipulated three strategic objectives. These are: 1. Private Enterprise Development Goal: Accelerated development and growth of private enterprises (SO 1.3 and SO 1.4) 2. Civil Society (including media and nongovernmental organizations) Goal: Increased, better informed citizens participation in political and economic decision‐making (SO 2.1 and SO 2.2) 3. Public Administration and Policy Goal: More effective, responsive, and accountable local government (SO 1.2 and SO 2.3) The three SOs are sufficiently broad that most of the FNE program “spheres” can be subsumed under one or more of the USAID objectives. However, the FNE program has increasingly shifted toward supporting improvements in Russian higher education, mainly in collaboration with the Government’s Ministry of Science and Education, and with the support of the US-Russia Foundation among others. The last two program spheres listed by FNE, regional mass media and Russian civil society, are largely financed through specific agreements with USAID. It should also be noted that FNE programs operate in 32 localities throughout Russia’s very large territory. Relatively few programs operate in Moscow. This gives FNE a national scope that is unmatched by any other non-governmental agency in Russia. FNE’s international reach is impressive. Three of the eight members of the board of trustees come from the US (2) and Poland (1). FNE has a major program of collaboration with university centers of excellence around the world as shown in this table: FNE Programs: Collaborations with foreign universities 2005-201289 USA 8 Europe (west) 13 Europe (east) 8 Israel 4 China 3 Current Sustainability According to the 2012 FNE Annual Report, FNE has nearly regained its overall funding amount, having 88

New Eurasia Foundation (FNE) 2004-2012 Report (English Version). The report covers the period 2004-2012 and represents the most comprehensive compendium of FNE's evolution, program areas, management structure and funding base as it evolved over the eight year period. 89 FNE Report 2004-2012 p.39

XLVIII

recovered from the impact of the 2008 recession, which affected the both the global and the Russian economy. In 2012 the EF core funding made up 11 percent of FNE's overall funding. EF’s core funding from USAID will terminate in 2013. FNE has substantially shifted its funding base to Russia supporters. Of the 81 separate funding sources that have provided support to FNE, all but 24 are Russian, including four government ministries, 21 public universities, 15 Russian businesses, 11 Russian charitable foundations and NGOs, and six private individuals. Among the 24 organizations providing support, five have been USG funded organizations, including EF-DC, and four are US-based foundations such as the Mott Foundation. However, it is also true that US foreign sources, among others, have been the principal source of FNE support over the 2004-2012 period, constituting 71 percent of all funding, with 32 percent coming from EF alone. Russian sources constituted about 20 percent of the total. FNE’s effort to shift its funding base to Russia sources has shown gradual success, with 41 percent of funding coming from Russian government and private corporate sources. EF-DC support had declined to 11 percent, but reliance on international donors and international private sources continued to represent close to half of FNE’s funding. FNE also receives funding from USAID Moscow, which is channeled through the EF Washington office. These funds support the Independent Regional Media support program. Other funding from USG sources includes support for the USAID Moscow funded, but Eurasia Foundation Washington managed civil society partnership program. The program brings together American and Russian civil society organizations engaged in providing services and dealing with a variety of community level programs that support disadvantaged and marginalized elements of both populations. The Russian Economic and Political Context Russian private charitable and foundation development is still in the early stages. Corporate foundations tend to support social development activities in the regions where they are located, For the most part the program areas, such as youth programs, community development, and the like, are non-controversial. Russian government support for FNE has been concentrated in two areas, mainly higher education and what FNE refers to as “territorial development,” (i.e., those often neglected regions far away from Moscow). FNE has been remarkably successful in developing positive relationships with the Russian government and the Russian political leadership. However, this may be changing, and may threaten several of the more sensitive program areas. During and after the most recent 2012 Russian elections, the Russian Federation, under the leadership of President Vladimir Putin, passed several important laws that could affect the sustainability of FNE, and even its independence as a private, not-for-profit social development agency. Potentially threatening laws and actions that may affect citizens’ right to organize, criticize, and to work with international funding entities are:



Requiring any Russian private NGO receiving funds from non-Russian sources to register as a “foreign agent,” which in Russia holds a negative connation of “spy.”90



Libel Law has been shifted from the commercial back to the criminal code, with severe fines imposed.

The International Center for Not for Profit Law (ICNL) has published a number of detailed analyses of this law.

XLIX



USAID Russia was terminated in September 2012, therefore ending a major source of funding support for two FNE programs; regional independent media and civil society cooperation.



Pressure has mounted against the US-Russian Foundation, an endowed foundation focused on economic and business development and the commercial rule of law. This foundation is a registered Russian foundation which provides funding support to FNE.

The more general rise in Russian nationalism and concurrent xenophobia has been promoted by the Putin regime as a means for solidifying its political base, especially outside of Moscow and Saint Petersburg. FNE's leadership has worked carefully to avoid portraying itself as having a “special relationship” with the United States, or even with the Eurasia Foundation, its founding organization. FNE President Andrei Kortunov is well respected and well-connected in Russia leadership circles. His strategy has been to develop FNE as a Russian agency with strong and diverse international ties, of which the United States is but one of many. Whether this strategy is sufficient to weather the current storm remains to be seen.

Case Study: EPF-Armenia Institutional Arrangements: Evolution and Current Status The field office of EF in Armenia began operating in Yerevan in 1995, supporting EF centrally-funded small grants and programs such as the Small Business Loan Program in Armenia91 (1996), the Media Strengthening Program (1997), the South Caucasus Cooperation Program (1998) and Caucasus Resource Research Centers (CRRC) (2002). In 2007, EF launched the Eurasia Partnership Foundation (EPF), an umbrella regional office for the local offices in Armenia, Azerbaijan and Georgia; the Armenian office was registered locally and was officially operating by February 2008. As a division of EPF, the Armenian office shares a board of trustees with the Georgian and Azeri offices, which is comprised entirely of citizens outside of Armenia. The tension between Armenia and Azerbaijan prevents inclusion of citizens from either country. EPF-Armenia also has an Advisory Committee of four Armenians. The current staff of 23 cover management, finance, human resources, media/ communications, grants and program management, internet technology and administrative services. Staff are led by Armenian Gevorg Ter-Gabrielyan, who joined the office as country director in 2007, and by Associate Country Director Vazgen Karapetyan, both of whom studied and lived abroad for significant periods of time92. While there is no evidence that EPF-Armenia staff are as reliant on the leadership as staff may be at other partnership foundations (PF), remarks from interviews suggest that the EPF director is well-respected. Ter-Gabrielyan can claim positive standing with the UK Embassy, which has consulted with him as a subject matter expert in the past regarding the nature of the embassy’s conflict portfolio and has set up meetings between him and the UK ambassador on issues beyond the scope of UK-supported EPF projects. A staff person credits the director with increasing overall quality of output and for envisioning the strategy necessary to ensure continued improvement of staff capability. EPF-Armenia’s full program for 2012 operated at USD $1.7 million. The PF expected to spend about $1 million in core USAID funding during 2012-2013, using only about a third of it in 2012 (supplemented by current Sida funding) and the rest in 2013. The PF is also receiving $2.3 million over two years from USAID/Armenia, $1 million over 1.5 years from Sida, and small amounts of funding from other bilateral donors, including Belgium, Finland, Poland and Britain. The World Bank provides some funding through EPF-Armenia to CRRC Armenia, though that is expected to cease when CRRC becomes independent of the PF. While EPF-Armenia receives some funding from the private Gulbenkian Foundation, it is not supported by any private corporation funding. It receives no funding from the Armenian government. Fundraising is managed in the regional office, but EPF-Armenia employs a development manager who collaborates closely with Tbilisi. According to staff, recent fundraising has had some successes – in 2012 the PF submitted an estimated 12 proposals and won four – but donors have slowly faded away, making EPF-Armenia now “heavily dependent on USAID core funding.” Though easy to raise money for small projects ($20-60K), success comes less frequently at the $100K level. Europe in particular has proven difficult to break into; the PF has applied to European donors 15 times over the last three years and has only been successful once; some of these proposals were submitted elsewhere and received funding, according to Ter-Gabrielyan. Staff disagree on how difficult it is to convince donors to pay for administrative costs, but regardless, the PF hopes to lessen these costs by securing a small office. Country Context

Evolved into the Izmirlian-Eurasia Universal Credit Company, which became operational in 2004. Ter-Gabrielyan earned his Master’s in Public Administration from Bowling Green State University and spent 1993-2007 outside of Armenia, working eight years in London with International Alert and then relocating to Moscow as a freelance consultant. Though Karapetyan has worked for EPF since 1998, he took a short hiatus to study public policy at the University of North Carolina at Chapel Hill. 92

Relations between Armenia and Turkey and Armenia and Azerbaijan continue to stagnate, and in the latter, some say may deteriorate further. Turkey continues to deny the Armenia Genocide, and segments of both the Turkish and Armenian populations oppose the Protocols. Azerbaijan continues to apply pressure to the Turkish Administration discouraging normalization of relations with Armenia. Following Turkey’s waning interest, Armenia’s has lessened as well.93 The Nagorno-Karabakh conflict with Azerbaijan remains unresolved, and tensions between Armenia and Azerbaijan worsened when the latter pardoned Ramil Safarov in September 2012 for the 2004 murder of an Armenian solider at a NATOsponsored English course in Hungary. The lack of regional cooperation has increased demand for programming in this arena94 but simultaneously makes achieving progress a challenge. From the perspective of EPF staff, neither the government nor private corporations currently carry enough wealth to be in a position to fund EPF programs, regardless of program focus. Despite a precipitous fall in GDP after Armenia’s independence from the Soviet Union, the GDP saw steady growth between 1994 and 2008, but this trend was interrupted in 2009 when GDP contracted significantly following the global financial crisis. Armenia experienced one of the most severe drops in the region.95 Regardless of wealth, EPF staff are not optimistic that the national government will soon become a prominent donor. One notes that working with the government would be problematic both because the government is oftentimes “unfriendly” and because EPF is not a big enough player. Another points out that the local NGO sphere is “struck by nepotism” and because ministers tend to have their own NGOs, EPF-Armenia has limited opportunities to truly compete for government funding. Only youth banks have been able to secure modest funding from municipal governments. Has the PF served and will it continue to serve US interests with respect to broad programming areas? The official EPF Mission is “to empower people to effect change for social justice and economic prosperity through hands-on programs, helping them to improve their communities and their own lives.” With respect to current programs, the country director asserts that the current program portfolio indeed focuses heavily on human rights, freedom of expression, civil society peace building and democracy, and that ideally there should be no switch to a focus on economic development in the near future. The deputy director agrees that recently EPF-Armenia has done little to support either public administration or private enterprise, instead concentrating efforts predominantly in civil society. The most recent Annual Report (2011) allocating funding according to programming supports these statements with figures: cross-border cooperation (56%), civic participation (15%), media development (14%), research (10%), youth (4%), public administration (1%). Overall, staff feel that while outcomes are nearly impossible to measure, the PF has been effective to some degree in all three of the USAID SOs listed in the original grant agreement: public administration (PA), civil society (CS) and private enterprise (PE). The country director expressed high satisfaction in the quality of the PF’s products and services, noting that while this may seem minor, these quality outputs are critical to achieving impact. The one staff member who rated the PF’s performance in the SOs rated CS highest (5), followed by PE (4),based on a small CSR program that established social enterprises in various parts of Armenia, and another PA (3). The staff person stated that EPF-Armenia was likely the driving factor in establishing at least 25-30 out of 150 of the strong, sustainable local NGOs throughout the country that are currently active and function as the core of emerging civil society. The evaluation team strove to examine programs in each of the three USAID SOs, though because the current overall portfolio of the PF tended toward CS, most of the activities reviewed are CS-focused. The 93

SATR beneficiary Both USAID and UK Embassy noted a desire for cross-border programming. 95 Armenia 2012 MCP Gap Analysis 94

LII

team had the opportunity to review in some depth the following current or recent programs: 

South Caucasus Cooperation Program (SCCP) (CS)



Support to Armenia Turkey Rapprochement (SATR) (CS with elements of PE)



Armenia-Azerbaijan Cross Border Project / Unbiased Media Coverage (CS)



Caucasus Resource Research Centers (CRRC) (varies, though preliminary outputs center on education; ultimate goal is improved PA)

Responses from staff and donors on their perceptions of the success96 of these programs garnered modest endorsements overall.97 One person involved in its early development of the SCCP regarded it as a successful venture, stating further that the anticipated partnerships —between 30 and40 through linkage and contract grants—were indeed created and sustained. and . A staff person working with the program today was more reserved in judgment, though noted that she was not aware of any evidence of whether partnerships developed via SCCP have endured and could not state any lasting effects of the project, “though it does not mean they do not exist,” she said. Neither staff person believes that the project succeeded in developing any sustainable trilateral partnerships. The views on SATR were similarly mixed. While donors and beneficiaries alike could point to a number of tangible outputs—business conferences, media products (cartoons, videos, books, etc.), and new partners—outcomes were harder to pin down. A midterm evaluation found that “although the project does function, the impact of its activities has been modest. Few people know about the project or its activities, which are not sustainable at present due to the problematic design of the consortium and problems with implementation.” One beneficiary noted that the implementers faced serious challenges due to the lack of political will in Armenia, and a donor pointed out that while EPF has established contacts and networks of organizations both in Armenia and in Turkey, the objective to significantly expand that reach was not fully met, particularly in Turkey. The Armenia-Azerbaijan Cross Border project appears to have been successful at the participant level, but to have struggled to expand its reach beyond immediate beneficiaries. One participant commented that his cartoon project was good but wondered whether his efforts were but a “drop in the sea.” Similarly, an evaluation of the project found that the expected long-term impact was too ambitious for the intended interventions, and that while raising awareness of the need for unbiased media coverage was within the project’s control, the actual publishing of unbiased reports was not. Furthermore, while trainees are likely to have gained new knowledge and skills, they are neither prominent nor sufficiently active journalists for their behavior to significantly deter biased media reports. CRRC received the most consistently positive responses from EPF staff, CRRC staff, beneficiaries and donors. Though achievement of the broad goal—building the capacity of local social science researchers and promoting evidence-based policy analysis—may be difficult to confirm, identifying success in achieving desired outcomes98 proved a more feasible task. Multiple beneficiaries confirmed and praised access to quality research resources and shared that they regularly use CRRC data in their research, though none were able to provide specific examples of collaborating with policy practitioners. EPFArmenia staff familiar with the PF’s financials noted that CRRC brings in more money than any other program and has the highest prospects for remaining sustainable; the group is expected to spin off from 96

Whether the program accomplished the objectives initially agreed upon. Given the difficulties associated with measuring outcomes and impact, particularly for such broad goals, it is not surprising that stakeholders were hesitant to comment on whether outcomes had been achieved. 97

According to webpage: “to increase the accessibility of high-quality research resources, to strengthen capacity and to increase the dialogue and collaboration between social science researchers and policy practitioners”

LIII

EPF-Armenia in the next couple of years. Despite its notable successes, stakeholders identified areas for improvement as well. While one donor expressed a need for CRRC to improve dissemination of and demand for its products, another pointed out issues with implementation quality99 and CRRC’s inability to provide analysis and interpretations of their data, a weakness also self-identified by CRRC. A review of grants made by EF-DC and EPF-Armenia using USAID core funds to support development efforts in Armenia shows that while CS received the majority between 2001 and 2010, PE is not far behind, having peaked in 2001 and falling since. PA, on the other hand, received only about half the amounts allocated to the other two SOs (see Figure 1 below.)100 Figure 1

2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 Total

Civil Society $303,801.00 $462,957.25 $146,359.76 $462,752.16 $95,961.96 $319,341.06 $310,965.15 $621,518.72 $141,176.29 $3,345.02 $2,868,178.37

Private Enterprise Development $607,374.28 $423,644.37 $418,925.33 $444,761.69 $90,790.65 $175,193.42 $36,175.00 $22,917.26

Public Administration and Policy $75,382.99 $286,024.80 $464,356.63 $83,595.29 $297,865.10 $16,942.00 $13,615.00 $33,843.73

$17,240.38 $2,237,022.38

$1,271,625.54

Total 986558.27 1172626.42 1029641.72 991109.14 484617.71 511476.48 360755.15 644435.98 175020.02 20585.4 $6,376,826.29

Is the PF well-positioned to be sustainable as core funding comes to an end? Is there a strategy? From an organizational capacity standpoint, EPF-Armenia appears well-positioned to be sustainable. The staff are so dedicated to the job, as one member explained, that though contractors tend to pay more than EPF’s market-level salaries, the hours are flexible, and there is a “family feel” to the organization along with a sense of ownership and purpose. Both staff and donors noted Ter-Gabrielyan’s high attention to detail to the quality of work produced by staff and grantees, superior internal communication practices, staff’s professional demeanor and staff expertise. Ter-Gabrielyan explains that he felt a need to “change business as usual” when he came on-board and as EPF-Armenia shifted from a grant-making to a program implementing organization, thereby hiring new people who he ensured were extremely qualified and the “real stars” in the country. EPF-Armenia still inherited the expertise of the grant managers from earlier days, but Ter-Gabrielyan challenged staff who remained to concern themselves with program content and outcomes rather than only management. Staff and donors recognized the PF’s reliable grant management tools, solid financial management procedures and overall high international standards, allowing one donor to “sleep at night knowing [the project] is being managed well.” The PF switched its GMS recently to allow it to connect grants to the financial department. EPF-Armenia carries other both positive and negative traits in the eyes of potential donors. The director considers one of its biggest selling points the EPF-Azerbaijan office, as it makes EPF one of the few organizations with this relationship. This perspective was seconded by donor who noted the difficulty of achieving this connection between other organizations, even when they’re encouraged by funding opportunities. The PF also prides itself as a neutral party. A donor supported this perception, noting the importance of messages being delivered by a neutral organization. They have confirmed this reputation 99

Meeting deadlines, insufficient focus group management, substandard desk research, poorly written report Classifications were recorded by EF and may differ from how USAID would classify activities.

100

LIV

for neutrality with others in the NGO community, public focus groups and colleagues. With respect to traits that may hinder the PF’s ability to become sustainable, staff cite issues that affect fundraising. For example, staff believe there are mixed views by donors on whether EPF-Armenia is truly “local” or not, and believe that the perception of the organization as “American” has stymied fundraising, particularly with the Europeans.101 A more concrete roadblock than the name, however, may be the international board, as USAID Forward encourages doing business with both locally registered and locally managed entities. Furthermore, staff acknowledge that the PF is significantly more expensive than other local NGOs due to fees paid to Washington and Tbilisi offices, and a donor estimated EPF’s administrative costs at approximately 20-24 percent of project costs, above the typical Armenian max of 10 percent.The deputy noted that a strategic fundraising plan is ongoing, but staff seem to agree that all members need to be engaged in securing funds. Leadership believes that substantial funding is only possible from USAID or other bilateral donors; multilateral organizations can be counted on for small provisions. The PF expects no funding from either local private companies/foundations or the national government for at least the next 10 years. Local businesses may offer in-kind donations. All staff who responded regarding the future of fundraising at the PF stated that they are “fairly confident” the PF will be able to attract donor support for the foundation’s programs over the next few years. The director cautions that the PF will survive without core funding but as a very different organization. In short, the PF will adjust its priorities to future donors and supporters. The PF will become more opportunistic, exploring new areas such as cultural rehabilitation (renovating churches, educational program, etc., which appeals more to Armenian diaspora than civil society or democracy), as well as sports and health. TerGabrielyan expects that the PF will not be able to meet social obligations (i.e., granting maternity leave), so rather than using employment contracts they will issue service contracts. Procedures may need to become more flexible, no longer meeting international standards, and a high quality of work will no longer be ensured. PF as a “foundation” and the current and future relationship between the PF and the EF in DC. For the most part, staff commented little on the benefits/challenges associated with being a foundation or with being tied to EF-DC. Staff responsible for grants management cited the legal “foundation” status as a local registered organization a boon, noting that this allows the organization to provide services for payment, whereas as merely a representative office, it could not. Views of staff on the PF’s relationship spanned the gamut from “very positive, helpful and necessary to our future” to “somewhat helpful but not essential to our future” to not having enough interaction with the central unit to have an opinion. The primary example of collaboration given was help with proposal writing from one EF-DC staff member when requested by EPF-Armenia; others mentioned EF-DC involvement via the CMI tool or simply noted that EF-DC would assist whenever EPF-Armenia requested the support. The EPF regional office in Tbilisi was identified by one staff member as the closer partner, working with EPF-Armenia on regional bids, reviewing other proposals and responding quickly to all questions and requests for help.

101

On the contrary, two donors noted the association with Americans as an attractive and reassuring quality, with one specifying that they “almost always give preference to US organizations when they submit good bids.”

Case Study: EEF- Moldova Background EEF-M is the newest of the partnership foundations, established in 2010. They have a staff of 14 and an annual budget of $1.5 million. They are led by Sorin Meracre, who started with the EF in 1998 as a project assistant and who has been President since 2010. Current programs focus on local economic development, good governance, and social action. Local Economic Development Goal: to enhance economic development in Moldovan communities by engaging local authorities, civil society and the private sector. Objectives:  

Public-private partnerships established, aimed at effective implementation of local economic and social development programs Increased participation of youth in community development

Duration: 2010-2011 Budget: $250,000 Funding source(s): Swedish Government, DANIDA Good Governance Goal: enhance civic engagement by monitoring reforms and facilitating cooperation between the government, civil society organizations, the media, and the public in the areas of European integration, free and fair elections, and anticorruption. Objectives/outcomes:     

Increase the level of civil society input in the decision-making process and increase transparency of government agencies’ activities Strengthen print media in Moldova by supporting an Independent Press Council (IPC) and Audit Bureau of Circulation Support the development of sustainable media in Moldovan regions through grants, loans and technical assistance to emerging media outlets Improve implementation of the government’s anti-corruption policy Promote free, fair and democratic elections on the national, regional and local levels in Moldova

Duration: 2010-2011 Budget: $1,000,000 Funding source(s): Swedish Government, DANIDA Social Action Goal: To promote the social engagement of private companies in addressing the needs of the vulnerable groups Objectives/outcomes:  

To promote the concept of corporate social responsibility in Moldova To increase opportunities for social integration and employment for social orphans in Moldova

LVI

Budget: $232,000 Funding source(s): Swedish Government, DANIDA New program areas include social entrepreneurship, addressing lack of employment and social opportunities for vulnerable groups, and Transnisteria, focusing on the deep divide between societies on both banks of Nistru River due to the unsolved political conflict. While EEF-M is the newest partnership foundation they have rapidly gained the trust and confidence of the donor community. A recent Sida evaluation of EEF-M concluded: 

EEF’s systems for internal control have been found to be impressive, reliable and relevant to the organization



EEF’s financial management policies, procedures and routines are consistent, adhered to and result in clear and reliable reports.



EEF’s board of birectors is actively and regularly involved in the organization’s development, in fundraising and in program monitoring.



EEF uses the Results Based Management Methodology in its program and project work but needs to focus also on measuring and monitoring outcomes.

Success Story EEF-M leadership regards the coalition for free and fair elections as their greatest success. The program was funded by the Sida with additional support provided by USAID, among other donors. EF reporting to Sida concludes that the program met its overall objective of developing a coordinated and effective method for Moldovan civil society organizations to promote free, fair and democratic parliamentary elections in 2005. Specific results achieved include: 

An effective nationwide network of civil society organizations engaged in non-partisan elections-related activities was established.



Efficient mechanisms ensured coordinated public oversight over the election process. Aspects of the election process included political party campaigns, local election committee and polling station activities, and/or ballot counting.



Public monitoring of election-related legislation and procedures increased. Violations were noted and addressed in accordance with the appropriate legislation.



Election coverage in national and local mass media was monitored on a regular basis; the results of this monitoring were widely disseminated.



Voters had increased and diversified access to information on election legislation, election rules and procedures, and on the candidates’ political platforms. Voters from more than 600 communities participated in election-related civic activities, including public debates and other community actions.

According to the final OSCE/ODIHR election observation mission report, “civil society organizations were active in monitoring the electoral process. The main domestic non-partisan organization to observe the 2005 elections was the Civic Coalition for Free and Fair Elections ‘Coalition 2005,’ which was created on May 12, 2004 and included almost 200 civil society groups.” Country Context Moldova’s progress with respect to USAID’s three strategic objectives—private enterprise development, civil society, and public administration— has been between average and good by regional standards. Data for this section is drawn from USAID’s Monitoring Country Progress (MCP) report. Performance is

LVII

ranked on a scale of one to five, with five being the best. In each case a composite score for the northern tier countries of Central and Eastern Europe, which have graduated from US assistance is shown for comparison. With respect to private sector development, Moldova is in the middle of the pack with a good trend line regarding private sector growth. They lag Azerbaijan, Tajikistan, and Belarus, but are well ahead of Kyrgyzstan, Ukraine, and Russia.

With respect to the development of civil society, Moldova is one of the strongest performers, and shows excellent progress in recent years. Kyrgyzstan and Russia are average while Azerbaijan, Tajikistan, Kazakhstan, and Belarus are the worst performers. To gauge progress in public administration the graphic below draws on Freedom House data on â&#x20AC;&#x153;more effective, responsive, and accountable local government.â&#x20AC;?

LVIII

Moldova ranks in the middle, behind top performers, Ukraine and Georgia, but well ahead of Azerbaijan, Kazakhstan, Kyrgyzstan, and Belarus. In this area there has been significant backsliding across the board and a large gap compared to the northern tier countries on Central and Eastern Europe.

Analysis of Foundation A. Alignment with USAID SOs Clearly the predominate focus of EEF-M has been in the area of civil society. Fully 83 percent of the $2,112,269 in grants made from 2001-2011 were categorized as civil society. In response to a question in which area EEF-M has been more effective and had the most impact, senior leadership ranked civil society as a five and declined to rank either private enterprise development or public administration as they had done little or no work in these areas. This focus is a result at least in part due to guidance and directions given by the US Embassy. In Moldova, there was close cooperation between EEF-Moldova and the Embassy between 2003 and 2004. Because the Embassy was coordinating the efforts of all implementing partners and wanted a focus on media and election observers in preparation for the parliamentary elections, they guided EEF down this path. The Embassy told the foundation that there were other USAID providers working in private sector development. B. Sustainability Leadership is â&#x20AC;&#x153;very confidentâ&#x20AC;? about continued donor funding. As noted above, leadership has already secured significant funding from Sida to cover their major programs for the next four years which includes overhead costs. They have a written business strategy that runs through 2013 and are in the process of writing a new one for 2014-2017. Their annual budget since they were launched in 2012 shows a strong upward trend line.

LIX

Year Annual Budget

2010 $1,222,496

2011 1,603,751

2012 1,515,173

2013 $2,302,445

Looking longer term, EEF-M is still highly dependent of donor funds with little or no support from indigenous sources of funding from either the private sector or government. This is in part a reflection that Moldova is a very small country and the poorest in Europe. While EEF-M has successfully ended reliance on core funding from USAID they are in turn reliant on other donors, primarily Sida. In addition, they have not made any plans to purchase office space, which could help lay the ground work for sustainability when donor funding inevitable begins to decline. C. Relations with EF Senior leadership on EEF-M characterizes the relationship with EF-DC as “very good.” They find EF-Dc to be generally helpful and still important. Major areas where they have received support include: 

Assisted with establishing financial management, audit, and financial accountability procedures used by EEF-M



Assisted EEF-M in using USAID core grant for operating expenses



Assistance with external donor fundraising and proposal development



Networking opportunities with other PFs is useful for identifying common problems/solutions, for example, they learned of youth bank in Georgia and copied in Moldova

The Eurasia Foundation has worked with each of the PFs to strengthen organization capacity, introduce international standards for financial management and accounting, and to strengthen the board of directors and address other organizational policies and procedures. The principal tool for assisting the PFs is EFDC’s Capacity Mapping Initiative (CMI). In general, senior EEF-M staff found the tool to be very concrete and verifiable and helpful maintain high standards. They also found it very clear and understandable. EEF-Moldova was not involved in the CMI development. EF conducted an evaluation using the CMI tool in 2011 covering four of the six topic areas: financial management, funds development, HR and staff development, and communication. Program management and leadership were not covered. Eurasia foundation DC staff conducted interviews and looked at files. A summary report was prepared and presented to the board. Based on recommendations an implementation plan was prepared and its progress has been reported regularly to the board. They are considering repeating the CMI assessment every two years. EEF-M is also considering proposing CMI as a tool for use by local NGSs, as the Georgia office did. This will engage NGOs in a self-assessment exercise and help change their mind set on how to improve management practices. D. Foundation Model EEF-M leadership felt that the strength of the Foundation Model was their ability to take a long-term view. They do not operate within a defined period of performance and can establish long-term relationships with local partners. They are a local institution, which is integrated into the local landscape. They are a trusted partner on the ground. They feel that groups that rely on ex-pats have less local knowledge resulting in higher costs and higher risks. They highlighted the fact that the foundation approach also provides continuity. This avoids the start and stop nature of most contracts. They're able to maintain a strong staff, which is important since contracts with a limited life have trouble keeping good people due to the pull of high paying NGOs. Some issues require a consistent long-term approach, while changing attitudes and behaviors requires time and a more a generational approach.

Country Case Study - Tajikistan EF first opened a representative office in Dushanbe, Tajikistan in 1996. As part of its drive to establish partner foundations throughout the region, the foundation set up Eurasia Foundation of Central Asia (EFCA) as an independent organization in 2006. Public Foundation EFCA-Tajikistan was registered in 2009 as a nonprofit organization according to the Law of the Republic of Tajikistan on Civic Associations. The foundation is registered as a non-membership organization with a geographical span of activities that covers the whole country. The goal of the foundation is to support the development of private entrepreneurship and civil society institutes in the Republic of Tajikistan. The by-laws do not provide detailed information about the foundation’s founders but define their reserved matters/core competencies to include decision making about the establishment of the foundation and approval of its statute, election and approval of the first advisory board, allocation of the foundation’s property, and appointment of foundation’s director once the foundation is established. The founders are not entitled to the foundation’s property. While the foundation is a non-membership organization, its by-laws include a chapter on terms and conditions of membership in the foundation. The governing bodies of the foundation are the advisory board and the director general. The advisory board performs strategic and controlling duties according to the statute and the special regulation developed for the board. The advisory board meets at least two times a year and it should not exceed 19 people in composition. The director general is appointed by the advisory board and represents an executive body of the foundation. He/she can delegate some of his/her responsibilities to the executive director of the foundation. The organizational chart of the foundation depicts only the position of the executive director and does not mention the advisory board or the director general. EFCA-Tajikistan has young executive director who was appointed in October 2010 after winning the competition for this position of the foundation. Since 2009 the executive director worked for the foundation as the program and communication Manager and in 2010 became an acting director of the foundation. The Dushanbe office currently has 14 staff members and in May 2012, the Tajik office created the following Departments: i) Financial and Grant Management Office (three people); ii) Program Dept (six Program Assistants and Program Managers); and iii) Development, Evaluation and Monitoring Departments (two staff and interns). The Foundation receives interns for three or six months, from the University of St. Edwards in Scotland and from NYU, to write proposals and support development. EFCA-Tajikistan has been doing this since 2007. The review of the foundation’s by-laws has revealed a mixture of grant-making and operational activities of the foundation. First of all, there is a conflict between Clause 1.6 about the non-membership nature of the foundation and Article 4, which describes the procedure for acquiring and canceling membership to the foundation. Two governing bodies of the foundation lack clear-cut division of the lawmaking/constituent and controlling responsibilities as the advisory board has both duties according to the statute. In reality, the advisory board approves contracts above $5,000, develops and endorses the strategy, reviews the contracts and monitors the implementation. The peculiarity of the foundation is that all three foundations in Kazakhstan, Kyrgystan and Tajikistan have a common board of birectors but the by-laws do not mention it. Therefore, the foundation should adjust the statute to the operational activities and consider revision of: i) membership/non-membership status of the foundation; ii) relevance of the position of the director general; iii) formal arrangement of relations with the board of directors of the regional office. The adjustment can be done either by introducing changes to the by-laws or by developing separate regulations. Mission of the EFCA – Tajikistan is to help people focus on civil society and economic development. Major program areas include: 1) Youth programs – empower and engage; 2) Civil society – strengthen capacity and build bridge between people and government; and 3) Vulnerable groups – integrating them into society and provide support. Other areas include: economic development; local governance; access to justice; media; and charity development. At present the foundation carries out the following projects:

LXI

1. Building Tajik Youth Confidence in Democracy. Youth in the country remain marginalized and lack the necessary mechanisms in the current system to express their views. This project aims to build the trust and confidence of Tajik youth in democracy. Moreover, it aims to empower young people to become agents of change in their communities. The project will seek to engage youth between the ages of 16 and 24 in the process of democratization by creating a framework whereby young people can stand for and win elections to a Youth Council (YC) in the Gorno-Badakhshan Autonomous Oblast (GBAO) of Tajikistan. Funded by UNDEF in the amount of $250,000, implementation began in January of 2012 and will last 24 months. 2. Access to Justice. Supported by the US Embassy, 14 months since June 2012. The project is funded in the amount of$93,000. 3. Equal Before the Law: Access to Justice in Central Asia Program (EBL) is a joint program of EF and the Ministry for Foreign Affairs of Finland aimed at increase of access to justice for vulnerable populations in Central Asia, in particular for rural women, at-risk children and persons with disabilities. Funded by the Finish Ministry of Foreign Affairs for three years, it began in January 2011 (Tajik part - $1,087,000). 4. Reducing Youth Radicalization in Tajikistan is aimed at reducing of the threat of political and religious radicalization in five of the poorest districts of the Sugd and Khatlon regions of Tajikistan.The initiative provides NGOs with the knowledge, tools and opportunities to facilitate a coordinated approach among stakeholders to address the triggers of radicalization. The project also works with young people to engage them in the social, political and economic sectors of their community, making radical religious movements less attractive options (Ministry of Foreign Affairs of Denmark, two years since June 2010, $170-180,000, until the end of 2012). 5. Community development’s the overall objective is to improve the exercise of fundamental democratic rights and electoral processes at a local self-government level in 15 communities in Fayzabad, Nosiri Khisrav and Khuroson districts. EFCA and partners work closely with local self-governments, local councils, civil society activists and other community members to increase their knowledge and skills on exercising fundamental democratic rights. Supported by the Ministry of Foreign Affairs of Finland, since 2010 the project is in its second year, $110,000. At this stage the foundation qualifies itself as a combination of half grant making and half operational organization. The core competencies of the foundation include civil society, local governance, youth bank and project and grant management. Main activities of the foundation are grant giving, training deliveries and consultation support. The foundation uses its own staff and invites independent outside experts to carry out its activities. The beneficiaries of the foundation are rural communities, journalists, youth in rural areas, vulnerable groups and NGOs. The foundation cooperates with the local and international organizations and local self-government bodies; it also tries to build partner relations with the national government in order to implement projects and perform its main activities. Tajikistan’s context is a very important factor that influences EFCA – Tajikistan activity. Tajikistan is a Central Asian country of 7.6 million people (January 1, 2011) facing major development challenges in eradicating hunger and poverty. Over 73 percent of the population of Tajikistan lives in rural areas and 46 percent is under the age of 19. The country is faced with geographic challenges, as it is 93 percent mountainous with limited access to other regions. The Tajik government relies largely on foreign state-led investment and loans from China, Russia, Kazakhstan, and Iran, as well as assistance from international financial institutions, for major infrastructure projects. One million or more Tajiks—up to 50 percent of the labor force—are labor migrants, and 40-50 percent of more of the remaining population lives in poverty. Almost one-fourth of households in Tajikistan receive some remittances from migrant workers.

LXII

These remittances reportedly account for almost one-half of Tajikistan’s GDP, making the country first in the world in terms of such dependency. Other problematic issues include corruption and inefficiency of public services, tensions in Pamir and the Presidential election in 2013. Today in Tajikistan, it is possible to identify organizations that have achieved significant success in cooperation with governmental structures both at the village, district, and oblast level. Moreover, there are unique examples of cooperation between civil society organizations and state bodies that has led to permanent change in legislation and state practice. However, CSOs differ in terms of the scale of their projects and direction of activity. There are CSOs whose activity is connected with public transformation and influencing the development of legislation and state practices. At the same time, some CSO activity is connected with that transformation on a local scale and the CSO’s work becomes more focused on the needs of local development. For all civil society organizations a priority of activities is to increase the cooperation between CSOs and state bodies and structures. Cooperation with state structures provides an opportunity to achieve optimum results in the realization of CSO activity, which has already been affirmed through practice in the third sector. Partnership and cooperation between the non-governmental sector and state structures seems to be an uneasy. The social-ideological context of cooperation between the non-governmental sector and state structures has been caused by the difficult transition period of the independent national state of Tajikistan. The Tax Code defines “charitable activity” and the Government may provide material and financial support to youth organizations, children’s organizations, charitable organizations and organizations of the disabled; may provide favorable tax policy; may give children’s organizations the right to use school buildings, non-scholastic establishments, clubs, palaces and houses of culture, sports facilities and other structures, free of charge or on favorable terms. Significantly, however, no tax privileges are provided for charitable organizations in Tajikistan. There are no legal barriers to resources. An NGO may engage in economic activities to the extent they advance the purposes for which the organization was created, but may not pursue the generation of profit as its primary purpose. Profit from the economic activities of NGOs, including charities, is generally taxed in the same manner as for commercial organizations. There is little awareness of corporate social responsibility in the Western sense in Tajikistan. Many, if not most, corporations have contributed to the Roghun dum campaign, but many of these contributions were coerced. Some corporations do engage in voluntary community assistance on an ad hoc basis. Whether the PF has served US interests with respect to broad programming areas and investments. Here use the existing tables showing alignment of PF programs with USAID's threeobjectives. Given various forces influencing the PF, will the country programs continue to relate to the set of broad US interests? Based on grantee data the Tajikistan programs have been very strongly oriented to Private Enterprise development. However, detailed analysis shows that supported projects were not always correctly qualified with an appropriate strategic objective, this does not allow making informed conclusions about the balance of funds across the three strategic objectives (SOs). When private enterprise (PE) and civil society (CS) development were main program areas of the EFCA-Tajikistan, public administration (PA) was a lower priority. As shown in the table below, PE accounted for 51 percent of grants, followed by CS at 32 percent and PA at 17 percent.

LXIII

Year 2000 2001 2002 2003 2004 2005 2006 2007 2008 Total

Civil Society 63,065.00 143,907.59 49,398.09 75,557.00 181,447.68 106,331.54 39,819.00 25,187.00 $684,712.90

Public Private Enterprise Administration and Development Policy 181,627.25 58,915.65 155,003.24 50,492.80 176,065.93 99,014.22 215,438.82 50,539.00 150,137.00 53,790.00 29,390.00 36,868.00 87,286.18 14,228.00 62,200.88 1,468.00 $1,058,617.30 $363,847.67

Total Grant Amount 303,607.90 349,403.63 324,478.24 341,534.82 385,374.68 172,589.54 141,333.18 87,387.88 1,468.00 $2,107,177.87

It proved challenging to identify a balance of recent programs across the three SOs. The following programs were reviewed by the evaluators: SO 1: Private Enterprise SO 2: Civil Society

1. American Chamber of Commerce Project (2008) 2. Civil Society Poverty Reduction Project (2009-2011) 3. Youth banks or Decreasing Youth Radicalization Project (20102012) 4. Central Asia News Service (7/07-3/11) Through Tajikistan Regional Correspondents Network.

The Foundation provided a seed grant to launch the American Chamber of Commerce (ACC) in 2008. The main activities of the ACC include advocacy/lobbying and promotion of favorable business climate. Despite the frequent turnover of directors the organization managed to develop proper arrangements and increase the number of its members by attracting representatives of local businesses. The ACC contributed to Tajikistan signing several UN conventions and adopting 10 new laws vital for doing business in the country and at the international level. Unfortunately, the ACC does not keep in regular contact with the Foundation and it is completely unaware about its activities. However, they did conduct a joint charitable ball. The organization is open and ready for close cooperation with the Foundation. The Civil Society Poverty Reduction project was implemented between June 2009 and August 2011 with support from the Ministry of Foreign Affairs of the Kingdom of the Netherlands in the amount of $72,340. The project aimed to strengthen the capacity of local NGOs working with vulnerable groups in Khatlon and the Districts under Republican subordination on poverty reduction issues and to address poverty among vulnerable groups in society. A total of 11 grants were given to NGOs to conduct income generating activities. In addition, 40 NGOs participated in three NGO clubs in Kulob, Qurgonteppa and Dushanbe. Each NGO developed an action plan to combat poverty in their community. The vulnerable primary target groups NGOs are working with to alleviate poverty includes orphaned children, poor and/or refugee women, and disabled women and men in the Khatlon and DRS regions. ,. Seven local NGOs implemented income-generation projects to improve the living conditions of vulnerable groups by assisting beneficiaries and their families in generating income to support their long-term economic needs and reduce their social vulnerability. The target groups utilized skills acquired during trainings and workshops to run the following income generation activities and businesses: sewing, shoe-repairing, livestock farming, milk processing, and crop farming. For example, public organization â&#x20AC;&#x153;Parastorâ&#x20AC;? near Dushanbe implemented a two-month course on small business enterprises and a year-long machine

LXIV

sewing course for girls. A mini-sewing workshop equipped with sewing machines was established, and the beneficiaries made and sold finished goods. The funds generated from the sale of goods were used to buy new fabric and was reinvested into the school where the girls lived and studied. In general, three out of five of the supported projects increased beneficiaries’ income. All participants improved their job skills that not necessary helped them to get jobs. One project is still operating a sustainable business, and two are still functioning. The Reducing Youth Radicalization in Tajikistan project that aims to engage youth in their communities in order to make radical groups appear less attractive. The program, which started in the summer of 2010, has established Youth banksYouth banks in five areas of Tajikistan: Isfara, New Mastchoh, Muminabad, Shurobad and Shartuz. The project is supported by the Ministry of Foreign Affairs of Denmark. A total of 28 young people are active in the project and have been given nine days of training in project management and the issues of radicalization. The budget for this program is $172,867, and it is expected to finish at the end of 2012. Each Youth Bank gets $5,000 in grants to develop space for extracurricular activities for youth outside of the mosque. Initiatives support youth involvement in sports, community initiatives, or whatever is important to them. The project is seen as Somewhat Successful. EFCATajikistan succeeded in sending out a message to youth that it is good to be engaged and there is more than religion to focus on in their life. Partner NGOs went into schools to increase the level of information available to youth to introduce the concept and to generate support from school teachers and directors in terms of free access to school and space through in-kind contributions. Moreover, local authorities support the project as well. However, there were some internal and external challenges faced by the projects. Some external issues include a great diversity in the capacity of these regions especially between the north and the south – some have no internet and others have no computer literacy. There was an issue with youth bank administration because youth were not registered with organizations and thus did not have bank account so any means to accept the funding. In addition, there was a high turnover of youth – the project would train youth and then some youth would leave the country or get married, and so additional youth needed to be identified and trained. For example Mascho have not been able to do anything yet because of this turnover. In Shartuz have more girls – 9 of 12 members are girls – whereas all others are male. In some cases, girls did participate and parents agreed, but then they were pressured by teachers and others not to participate so dropped out. The internal challenges include two major interconnected issues. First, there was issue with fraud that resulted in firings of several people. As a result there were delays with some project activities and deliverables. The donor was informed and agreed on a plan with the EFCA-Tajikistan to solve the fraud issue. Second, the project manager was changed in the middle of the program. It should be mentioned that the issue of fraud was solved in an open, transparent and honest manner that showed maturity of the EFCA-Tajikistan leadership. The Central Asia News Service Regional Project (7/07-3/11) was implemented via the Tajikistan Regional Correspondents Network. During the project the network had five correspondents; at the time of the evaluation only one correspondent was working. In general the project is assessed as very instrumental and important since it provided a possibility to build the capacity of journalists, to develop and introduce standards of journalist ethics, to receive information from alternative sources and get experience in working for the international mass media. The network correspondents not only gathered and transmitted information, and prepared news reviews about Tajikistan, but also promoted new professional standards among their colleagues. They continue the educational activities now when the project is over by delivering various trainings and workshops for journalists and CSOs. The challenges the project has faced include reluctance of journalists to raise issues that might upset the public authorities (very high degree of self-censorship), great dependency of journalists on their income and living conditions such as electricity to run computers and access to the Internet. The sustainability of the project and the correspondents network could be improved by establishing a representative office of the Agency in Tajikistan and by ensuring a regular update of the project website in all languages (www.ca-news.org). Since the partner organization has been registered the foundation successfully transforms into an

LXV

operational organization losing its grant making talent, but maintaining the program and financial procedures and management in place as well as strong financial management and strong financial control. In the opinion of the foundation the key success factors include: board and governance structures; an empowered board that makes decisions and drives policy; driven by local staff and dependency from stakeholders and local NGOs; getting active people and communities involved; addressing key issues; work with partners and credit them for the foundation success; trust of stakeholders; good staff and trust relation to staff from leadership; very targeted technical and capacity building work with NGOs; not overlapping with government or other NGO activity. EFCA-Tajikistan faces challenges from various sides: 

Government sees the Foundation as a threat after the Arab Spring as there is lack of understanding by the national government which does not want to cooperate with NGOs, especially ones that is seen as supported or funded by the US. The Ministry of Foreign Affairs has a special Department on International Organizations that has a meeting with the Foundation every 2-3 months to check what the EFCA – Tajikistan is doing. Also, the Ministry of Justice will have meetings with all NGOs and tell them not to do certain things. Although working with the local government is easier, there is little decentralization in Tajikistan and de facto, everything still comes from the top. In addition, the Tajik government will likely adopt a law like Russia on foreign agents and it might happen next year.



Donors. In some cases the USG through its rules and rulings has determined that the EFCA – Tajikistan is not eligible to compete for funding reserved for local organizations. Also, some RFAs are not necessary focused on important issues for Tajikistan. For example, the youth radicalization project (funded by Danish) was a response to published cartoon on Mohammed in newspaper. But it is older people that need to be addressed on radicalization issue rather than youth.



Other external factors include corruption, lack of NGO transparency and accountability as well as their low representation of their communities and time needed for development of stable civil society and for visibility of the NGO activity results.



However, the biggest challenges for the EFCA are internal one. They include: balancing between mission or/and donor driven foundation; consolidation of the programs and focusing on core competencies; high staff turnover as staff going to UNDP, Oxfam and OSCE because they pay higher salaries. EFCA – Tajikistan hires a lot of young people and after a year or two they move on. In addition, communication, positioning, and marketing of the EFCA-Tajikistan is a big challenge as the Foundation has new status (a local operational organization) that requires new PR approaches, communication channels and messages.

Whether the PF is or will become sustainable as EF Core Funding comes to an end. This will entail an examination of the PF "strategy" for fund raising, but also whether the PF can maintain international standards for audit, accountability, transparency, and the like. Here also the issue of leadership may be a factor. In the 2010-2014 sustainability plan prepared by EF there were specific fundraising targets established for Eurasia Foundation Washington headquarters (EF-DC) and the five partner foundations, including EFCA. Each partner foundation should increase the percentage of its own budget from non-directed sources (not through sub-awards from EF-DC) by an average of five percent annually over the four year period from the 2010 baseline. For the first reporting period of June 2010-June 2011 EFCA (Central Asia) reached 92 percent of target. As for the year ending May 2012 EFCA is slightly below at 86 percent of target. While EF network has broadly met their financial sustainability targets to date, country context and available sources of funds are different not only for each partner foundation, but even for different countries in the same region.

LXVI

EFCA – Tajikistan’s 2012 budget is $564.747 that supports implementation of five to six projects and 14 staff members. Most of the interviewed EFCA-Tajikistan current staff is ‘fairy confident’ about the ability to secure future funding and funding is secured by the end of 2013. Now EFCA-Tajikistan is expecting results from 10 submitted proposals. Proposals were submitted to a variety of European, USA and multi-lateral donor organizations. However, EFCA-Tajikistan had recently faced a situation when the foundation was excluded from several US Department of State grant competitions reserved for local institutions because of an interpretation that they are an international organization. Currently the Foundation has no USAID Mission programs as the latest one the Foundation applied for was an agricultural project that was rejected due to “no agricultural expertise” even though the proposal was great. EFCA – Tajikistan does not have a strategic or fundraising plan as there is an organizational strategy for EFCA on a regional level for 2012 – 2014. The Foundation has an annual budget and operational plan for Tajikistan that identifies needs for fundraising and internal needs, but is more about money than strategy. In 40 percent out of all attempts the EFCA – T gets indirect costs covered and European donors are very flexible on this (for example. Danish and Finnish allow seven percent while UNDP does not allow anything at all). EFCA – T established overhead costs at 7.2 percent level. With donors, by building this in we don’t ask them to buy us hardware and other stuff – we can tell them this is built into the indirect rate All financial reports are submitted to EFCA regional office. EFCA regional office bought all three country offices 1C accounting software. Every year EFCA-T undergoes A-133 audit and sometimes project audit if it envisioned in the project. It is envisioned that future support might come from local private companies and foundations, foreign government assistance programs, USAID Mission, private foundations, and multi-lateral organizations. PF as a “Foundation,” and the current and future relationship between the PF and the EF in Washington, DC. Brief history of EF-PF relationship with respect to up-grading organization effectiveness and performance...and current status re international standards. The importance of the EF CORE GRANT should be assessed. After the EFCA-Tajikistan was registered, the foundation has cooperated with EF on various projects. The Foundation reports to the regional office of EFCA on a monthly basis . The majority of polled individuals assess the relations with the EF/DC in the next few years as Generally helpful and still important to us. In the opinion of the polled individuals the positive aspects of cooperation with the EF included common grants and financial systems, solid reputation, branding, experience, materials, and expertise. In some cases, the foundation acted as a partner in bids (ie EBL) and local entity could not win or handle a a million dollar project. Among the negative sides of the relationship between EP and EF Washington the people mentioned the following ones: as funding declined, started to lose resources like Grants Manager in DC that oversaw all grants policy; still have some M&E from DC but reduced resources; DC is too slow to respond and too expensive to work with since DC salaries are seven to eight times more than local salaries. Among the major benefits PF has gained from associating with EF and the Network the EFCA – Tajikistan respondents mentioned: assistance with organization performance and capacity building (capacity mapping tool/exercise); assistance with establishing financial management, audit, and financial accountability procedures; assistance with our external donor fundraising and proposal development; networking opportunities with other PFs is useful for identifying common problems/solutions; gaining of a certain amount of international status/reputation and respect for being in the EF association; seen as an honest broker because we have done this in the past and are not seen as being critical of the government; this network is very important benefit. This includes using CRRC or EERC as institutions to provide technical assistance. In other cases, EFCA –Tajikistan leverages and employs past program experience. For example, a Ferghana Valley program was modeled after a similar program in the Caucasus.

LXVII

The procedures put in place by EF-DC were strong and contributed to the reputation and organizational capacity. However, some changes were made in 2010 when EFCA-Tajikistan became an independent organization. Among introduced minor changes. EFCA â&#x20AC;&#x201C; Tajikistan reduced the size of a grant to be approved by the Advisory Board to $5k while, for example, in Kazakhstan it is $15k. Other changes might be introduced if the foundation needs them or NGO legislation requires it. It was also noted that EFCA is seen as an international and not a local organization by some donors, like the US Embassy and SOROS, and perception of EFCA -Tajikistan as an international organization actually hurts them. As for the core grant, EF/DC assisted the foundation in using USAID core grant for operating expenses. Since2010 the foundation has $79,000 from the EF; this money will be spent on civil society or economic development programs and 10-15 percent will go to the overhead expenditures. There are significant advantages the leadership and PF staff observe such as the Foundationâ&#x20AC;&#x2122;s knowledge and experience in working with NGOs; ability to develop NGO organizational capacity and implement projects in the $100k to $1 million range while local NGOs can do less than $100k; ability to work with NGOs that USAID could not work with as they do not meet international standards and build their capacity. In addition, the Foundation has become one of the strongest NGOs in the country with a good understanding of international standards; well-trained local staff with knowledge of local tradition and with connections in government as well as an important combination of staff skills (professional with human). Contractors leave after 3-5 years and permanent funding from one source allows for strong organizational development, stability in the long term, and ability to earn reputation. The disadvantages include strong connections to the US that not only raises suspicion, but also makes PF being seen as a US entity and not a local organization. As a result EFCA is contemplating rebranding and not using EF title. In addition, there are needs to rebrand and even reposition with US Embassy as well as with other international donors and organizations. Also, there is a strong perception of PF as of grant making foundation.

LXVIII

Country Case Study Kyrgyzstan Background and Country Context The Kyrgyzstan Partner Foundation (PF) was established in 2005 as part of EFCA. The Bishkek office currently has 23 staff members and an annual budget of $1.6 million, and the Osh office has seven staff members and an annual budget of $495,000. The current portfolio includes country specific and regional programs, and some programs are jointly implemented with the Osh office, which also manages its own cross border activities. EFCA Kyrgyzstan has been very focused on civil society (CS) activities. Their core programmatic areas include local community development and governance, youth, CSR, vulnerable Groups, cross border/conflict resolution, and media. Several staff members defined their mission as building the capacity of local organizations and also noted that working at the local level is a comparative advantage for them because other organizations are more involved at the national level. Current funding partners include USAID Kyrgyzstan Mission, DRL, the Ministry of Finland and the OSCE. In terms of country context, political instability, corruption, and conflict issues in the south influence the programmatic direction of donors and the PF. While corruption and a lack of government capacity have made it very difficult for the foundation to work at the national level, EFCA Kyrgyzstan is well positioned to work at the local level due to its networks and knowledge. The relatively democratic conditions in Kyrgyzstan provide opportunities for civil society development as well as work in media. Civil society remains vibrant, but weak. NGOs are the second largest employer in the country, following government, but capacity remains low. This provides the PF with a competitive advantage as a strong local NGO that adheres to international standards, and also allows them to position themselves as an expert in NGO capacity building. However, the rich donor environment has contributed to staff turnover issues, which remains a threat for the future of the organization. The Kyrgyz Office has a new Executive Director, although she was previously a Program Manager, and the Osh team is all relatively new. Weaknesses in program management and implementation were identified that appear to be related to staff turnover. Economic development remains quite low and there is a very limited opportunity to engage with the private sector. However, the strong mining sector and presence of Canadian mining company Kumptor has led to some work in CSR and local governance issues, which EFCA Kyrgyzstan sees as a potential growth area. Whether the PF has served US interests with respect to broad programming areas and investments. Here use the existing tables showing alignment of PF programs with USAID's 3 objectives. Given various forces influencing the PF, will the country programs continue to relate to the set of broad US interests? The Kyrgyzstan programs have been very strongly oriented towards civil society in both grants and other donor funded initiatives. In terms of grants, in the early years there was a relatively even balance between private enterprise (PE) and civil society (CS), but over time the focus shifted heavily towards CS, with public administration (PA) a lower priority. This is consistent with the findings for Central Asia overall. As shown in the table below, CS accounted for 59 percent of grants, followed by PE at 32 percent and PA at nine percent.

LXIX

Row Labels

Civil Society

Private Enterprise Development

Public Administration and Policy

Grand Total

2000

$144,270.17

$43,008.20

$27,536.23

214814.6

2001

$109,107.84

$109,260.73

$10,992.55

229361.12

2002

$98,854.01

$116,814.11

$68,668.05

284336.17

2003

$210,738.88

$273,106.71

$37,865.00

521710.59

2004

$610,193.36

$287,011.18

$153,959.33

1051163.87

2005

$380,792.50

$76,113.77

$1,864.00

458770.27

2006

$224,647.65

$117,261.00

$11,881.00

353789.65

2007

$122,036.78

$53,103.79

$4,914.66

180055.23

2008

$40,008.50

40008.5

2009

$59,192.00

59192

$1,081.00 $2,000,922.69

1081 $3,394,283.00

2010 Grand Total

$1,075,679.49

$317,680.82

While the evaluators sought to identify a balance of recent programs across the three SOs, this proved to be a challenge due to the strong emphasis on CS. The following programs were reviewed by the evaluators: 

Professional Youth Journalism Development Program in Central Asia – PYJDP - (9/12-12/12)



Central Asia News Service – CANS - (7/07-3/11)



Education Loan Program – ELP- (5/09-5/12)



Regional Trade (2007-2011)



Cross Border Youth Cooperation Project – CBYCP- (3/10-3/12)

All were categorized as CS programs, with the exception of the Regional Trade initiative which was considered a PE program. However, for this program, the primary role of EFCA was to organize and manage a contracting process for numerous technical specialists. This was leveraging their experience as a grant making entity, and did not demonstrate a core competency in the PE area. Thus, the only program area explored by the team in depth was CS. The evaluation found mixed results in terms of program success in the CS programs. The evaluators considered whether or not the program met its indicators; staff, donor and beneficiary interviews on the specific programs; and consideration of sustainability and/or legacy of programs. The two media programs reviewed – both CANS and PYJDP (implemented by local partner KLOOP) – appear successful and demonstrate a core competency of EFCA in regional media activities. PF Program Managers categorized these as between “Very Successful” and “Somewhat Successful”. PYJDP, a $500k program funded by DRL, exceeded its key indicators in terms of number of attendees, graduates, internships, and website hits. The program expanded to include five languages and news from all countries in Central Asia. CANS (www.ca-news.org), a $295k program funded through DRL, also had the goal of increasing coverage of Ferghana Valley. CANS leadership was pleased with the results, which included increasing website visitors to 26 thousand a day, and producing over 150 articles a day in three languages. Both are implemented jointly with EF-DC.

LXX

In terms of legacy, both KLOOP and CANS felt that the overall goal of improving journalism standards had been met. Both organizations are making progress toward financial sustainability, although both remain donor dependent. KLOOP has developed some fee for services, including commercial photography classes and subscriptions and advertising, and CANS currently receives 50 percent of its revenue from subscription fees and ad revenue. Both identified areas where they developed organizational capacity through working with EFCA, including accounting, budgeting and proposal preparation, but felt EFCA could have been more active in supporting them to plan for sustainability. The two other CS programs reviewed by the team did not meet expectations from the perspective of the donor, beneficiaries and counterparts. Both of these programs were funded by Mission RFAs and include the Education Loan Program (ELP), a $500k effort that ran from May 2009-2012, and the Cross Border Youth Cooperation Project (CBYCP), a $457k program that ran from March 2010-2012. The goal of ELP was to promote a new DCA with two banks that was introducing student loans on the market, and then to support students who received loans in getting jobs and internships through an association of University Career Centers (EdNet). The CBYCP paired four villages on the Tajikistan-Kyrgyzstan border to carry out joint community projects as a conflict mitigation effort. The ELP partner banks made about 150 loans combined and one of the banks had expected to make more than 1,000. There were many issues with the PR materials and events led by EFCA that required significant involvement by partners and the donor, including editing all materials. The job fair events also did not meet expectations in terms of turnout and capacity building of the career centers to carry out this activity in the future. The CBYCP components included youth engagement, developing a school of conflict management, and a youth bank. While some positive aspects were noted, including a 3-day training on youth banks, it was not clear that the program had an impact in mitigating conflict. During the project unrest broke out amongst youth in the region and EFCA did not develop specific interventions to respond to this development. Some deliverables were late or not produced, including a public outreach strategy, brochures and a movie to highlight program successes. With the latter, money had to be returned to USAID because the film was completed after the POP. The youth bank grants component also had issues. Some required construction for which EFCA did not seek BEO approval, and the sustainability of these efforts was not clear. Thus, there was a range in the quality of the programs reviewed. One of the key factors that contributed to success included high quality partners and NGOs (such as KLOOP and CANS), but this was not universal since the existence of strong local partners in ELP (banks and EdNet) did not guarantee results. General issues with implementation can be linked to significant staff turnover for both Bishkek and Osh based staff involved in these projects. For ELP, the Program Manager changed twice and the IT staff person tasked to create a website changed three times. For CBYCP, there were three different Program Managers, the 3rd of which arrived with five months left; the Chief of Party also changed. Donors and partners also noted that the staff assigned to the projects did not seem to have the requisite skills in specific areas, including education and conflict mitigation. There were also issues with poor planning and project management, which was acknowledged as a weakness by the PFâ&#x20AC;&#x2122;s own staff. This demonstrates that the greatest challenge to the EFCA Kyrgyzstan going forward will be retention and skill development of its own staff. PF leadership noted that they will continue to operate in the same general program areas in the future, which will remain CS oriented, but did express the need to adjust based on donor priorities in the future. As the Kyrgyzstan PF completes the transition to a local organization that is potentially more donor driven, the need to ensure adequate technical skills could be a challenge. The quality of programs must be maintained to enable the organization to successfully compete for funds. The weaknesses in project management and technical skills should be addressed to continue strong implementation and remain results oriented. Is the PF (EFCA Kyrgyzstan) well positioned to be sustainable as EF Core Funding comes to an

LXXI

end. Is there a strategy in place for fundraising and can the PF meet international standards for audit, accounting, transparency and the like. Currently, EFCA Kyrgyzstan receives little core funding, and as of 2009 had anticipated that all remaining core funds would be used for matching in programs or to support office and payroll functions. The 2009 Grant Proposal to EF-DC outlined a strategy for EFCA to transition from core funding and several changes have since been made, although not all of the proposed changes were implemented, including eliminating the Bishkek Executive Director position. EFCA has come close to meeting sustainability plan targets - from June 2010-May 2011, EFCA achieved 92 percent of its target with $3.04 million raised, and from June 2011-May 2012 reached 86 percent of their target with $2.98 million raised. However, this is not specific to Kyrgyzstan. According to the most recent fundraising report, covering April-August 2012, Kyrgyzstan had won three awards totaling $1.28 million or 54 percent of total EFCA fundraising, which shows a relatively strong fundraising position. Kyrgyz PF staff interviewed also expressed a high degree of confidence regarding their financial sustainability in the future. Leadership responded that they were “Very Confident” or “Fairly Confident” about their ability to attract donor funds over the next three-to-five years. They also felt that funds would be generated from all sources listed except for the host Kyrgyz Government. EFCA also benefits from having an established overhead rate of about seven percent that was developed by the 2nd EFCA President and allows them to recover these costs from all donors, with the exception of USAID. A few threats to future financial sustainability were identified. One key issue relates to the status of the organization, which is not perceived as truly local and is competing for funds as a local organization in some cases. The Kyrgyz PF is planning to hire a PR specialist to rebrand and reposition the organization, and EFCA may change its name as part of this process. The perception of the PF will impact its future fundraising capability and there needs to be a clear understanding among donors of who they are and what they do. This raises the question of whether US interests will be met in the future if the branding of EF and its US affiliation are lost. Another question is how their development team will be funded in the future. Currently, the Development Manager position is covered from Local Core Funds, and this transition occurred in November 2011. This position has been held by a locally based expat and does not require allowances associated with expat staff; however, going forward the office must generate sufficient revenue to cover this position. The leadership stated that EFCA Kyrgyzstan will maintain international standards for audit and accounting in the future, but that procedures will evolve as needed. However, there is a specific issue with the Kyrgyzstan office that deserves mention. Two of their current programs are USAID Mission funded cooperative agreements that were structured on a reimbursement (rather than advance) basis. This includes a $1.3 million transparency in local government project and a $1.2 million women’s peace bank project. This was evidently done in response to USAID audit findings that had not been adequately addressed in terms of cost allocation issues. EFCA has requested a change in the terms and USAID Central Asia RCO recently conducted an audit, but the results are not yet known. EFCA Kyrgyzstan does not have the resources to manage these two large programs on a reimbursable basis and the loss of these programs would be a substantial hit to their current portfolio. Thus, EFCA Kyrgyzstan will need to ensure that accounting and finance practices remain consistent with international standards to ensure current levels of program funds. PF as a “Foundation,” and the current and future relationship between the PF and the EF in Washington, DC. The leadership and PF staff all characterized the relationship with EF-DC as either “Generally helpful and still important to us” or “Somewhat helpful but not essential for our future”. The current relationship is seen as “formal” and primarily related to finance and reporting. The grants management system (GMS) and the legacy of accounting and financial management systems were seen as the greatest benefits of the

LXXII

relationship. Once they transition to full independence, the PF will cover the costs and management of the GMS, which is currently covered by EF-DC. An additional positive aspect of the relationship is that EF-DC identifies and leads joint bidding on opportunities. However, the view was also expressed the view that EF-DC was not helpful in attracting significant funds when the transition to a sustainable entity began in 2005. Thus, there is a distinction made when they bid together and EF-DC takes the lead in identifying and preparing the bids, vs. when the PF undertakes bids on its own, where they feel they get little support from EF-DC. Furthermore, the staff sited the expense of EF-DC staff and their NICRA on joint programs as a downside to the relationship. The reduction in core funding has also led to a decline in human resources provided by EF-DC. For example, the loss of a DC based grants manager to support the field and a decline in M&E has been noted by field based staff. It is unlikely that EFCA will pay for this in the future, although they will pay for EFDC to conduct next yearâ&#x20AC;&#x2122;s CMI audit. However, staff did not indicate that the CMI tool had a major impact on their organizational capacity or operations. The network, rather than EF-DC, may be more beneficial to EFCA going forward. Staff noted that the network provides direct benefits through exchanges with other PFs that allows for learning opportunities and they are looking at doing bids together with some other offices, including an upcoming EU tender on womenâ&#x20AC;&#x2122;s issues. However, it was also noted that they seek to leverage the network to make it more useful and this is an area they continue to explore.

LXXIII