KM INSIGHT
Assessing the Impact of High-Quality Microfinance Programs v3
Rigorous research in Myanmar and Paraguay highlights changes in income and assets amidst challenging program implementation contexts.
Research Overview
The Microfinance Plus Study (2019-2024) conducted by Opportunity International, aimed to assess the impact of highquality microfinance programs on poverty in Uganda, Myanmar, and Paraguay.1 The study used an improved design, compared to past randomized controlled trials (RCTs) published in 2015, by incorporating a robust study design, assessing quality microfinance programs offering “plus” services such as training, and conducting research with strong partners. Despite substantial challenges to field interventions caused by the pandemic and subsequent economic recession, natural disasters, civil war, and/or control group taking up significant number of loans, the study was implemented successfully. And, it captured positive impacts – sometimes quite positive – on income and assets.
Key Conclusions
The following high-level conclusions emerged from the research:
1. The critical 2015 RCTs can no longer be taken at face-value. The meta-study of those studies conducted as part of this project and the direct results from this study show that those studies are flawed and are a distraction from serious discussions. Those who still refer to them should be invited to read the meta-study at the very least 2
2. Microfinance can have significant positive impact even in difficult contexts. In Myanmar, clients showed 40-52% income increase over the comparison group. In Paraguay, clients experienced a .087 standard deviation increase in assets over the comparison group – a significant, medium-level impact.3
3. However, contextual factors can become too severe to the point that positive impacts cannot be expected. While no country in this study showed negative impacts, they could have if had factors been sufficiently severe. Social restrictions and distance in Paraguay, and climate disasters in Uganda are examples.
4. Discussions of effectiveness going forward must address study design, actual operations, and context. This study made the weaknesses of past studies apparent. We recommend that discussions (whether critical or positive) focus on conditions/factors that make microfinance more, or less, impactful and how to address them.
5. This study highlighted the importance of working with theories of change as practiced or practicable. Past studies do not seem to have engaged such details. The various results from these studies did not undermine the theory of change – even the Uganda study – only further affirmed their efficacy.
RESEARCH CONTEXT123
The 2015 Studies. Microfinance provides financial services to poor and financially excluded populations. The development objective is to improve client income and assets, and – through these – to enhance consumption, food security, and access to social services (e.g., education and health), thus reducing poverty. Researchers had begun to measure microfinance's impact on poverty through randomized controlled trials (RCTs); and, in 2015, a journal published a collection of some of these studies.4 These examined the effects of microfinance on borrowers' income, assets, and consumption levels by comparing microfinance clients to a similar “control” group meant not to have access to microfinance.
1 Unfortunately, due to flooding, locusts, and then drought that caused hardship for farmers in northern Uganda making it difficult to recommend continuing take-up of services. For these reasons, the Uganda study was closed down early.
2 Mahesh Dahal and Nathan Fiala, “What Do We Know about the Impact of MicroFinance? The Problems of Power and Precision.” World Development (Vol 128: 2020): https://www.sciencedirect.com/science/article/abs/pii/S0305750X1930422X
3 Experienced by clients in comparison to control in Years 1-2 in Myanmar and in Year 3 in Paraguay. There were drops experienced in Year 3 in Myanmar and Year 4 in Paraguay which is discussed elsewhere.
4 Banerjee, A., Karlan, D., & Zinman, J. (2015). Six randomized evaluations of microcredit: Introduction and further steps. American Economic Journal: Applied Economics, 7(1), 1-21.
Findings . . . and serious problems. The various studies seemed to find that while microfinance increased business activity (investment), it did not significantly boost income, assets, or average household consumption. However, these studies revealed significant flaws, bringing their conclusions into question. These included
• Insufficient statistical power. These studies were underpowered (too low take-up rate), such that they could not detect reasonable changes in income and assets.5
• Weak microfinance programs. Care was not taken to study microfinance following best practices. The programs used “minimalist” microfinance with no training or additional “plus” services. Some programs were implemented by institutions new to microfinance.
• Short evaluation duration. Results were often measured after only 1-2 loan cycles, which is too short to observe significant impacts. Microfinance programs strive to offer multiple loans of increasing value over several years in order to have a substantial impact in income and assets.
• Validity of Comparisons. Control group use of finance from other providers was common, and take-up rates among the treatment groups were often low (not as many target respondents took loans as expected). These factors undermined the validity of comparisons between clients and non-clients.
• Market Conditions. Serious questions regarding whether normal market forces were operating given the locations or the low-take up rate of the studied programs.
A decision to study more carefully. To measure the impact of highquality microfinance programs on clients, Opportunity worked with researchers capable of deploying RCT methodology more carefully and with mission-driven microfinance institutions (MFIs) with high-quality, microfinance “plus” programs. The research intended to inform policymakers, practitioners, and investors/donors about the actual benefits and limitations of microfinance, ensuring effective resource allocation and realistic expectations. ♦
OPPORTUNITY’S RANDOMIZED CONTROL TRIAL (RCT) STUDY DESIGN
Careful preparation for a robust study. Opportunity’s “Microfinance-Plus” Study aimed to improve on past RCTs with an adjusted study design, measuring the impact of high-quality microfinance programs that offered training and other services in addition to microfinance. Key aspects of the improved study design included
• Scoping studies for appropriate partners and markets: Selected high-quality implementing organizations offering “plus” services with access to unsaturated markets (areas) these organizations were serving or intended to serve, so that the “control group” would be unlikely to access microfinance from other providers. These scoping studies were conducted in six countries before selecting partners in Uganda, Myanmar, and Paraguay.
• Designed sampling and follow-ups for high statistical power: Pre-screened participants’ interest prior to randomization to ensure high loan take-up rates, with potential drop-out rates in mind; planned for multiple follow-ups in subsequent years (Paraguay 4 years; Myanmar 3 years) after offering loans.
• Quality Research Teams: We worked with lead researchers with strong experience in economics and research, Dr. Nathan Fiala (University of Connecticut) and Dr. Lise Masselus (RWI - Leibniz Institute for Economic Research), and with country research partners known for their careful work – Gaplink and Innovations for Poverty Action. Adjusting to client needs was considered a part of the theory of change. Prior to the start of the study, partners conducted a “client journey mapping” exercise that detailed the client journey in obtaining financial services, identifying pain points and generating information partners used to improve service delivery. ♦
THE MICROFINANCE PROGRAMS
The study assessed 3 programs. Opportunity Bank of Uganda, VisionFund-Myanmar, and Fundación Paraguaya.
• Uganda. The MFI partner, Opportunity Bank of Uganda, Ltd., targeted rural and agricultural population, with group finance, loan orientation, good agriculture practice (GAP) training, and holistic wellbeing training.
• Myanmar. The MFI partner, VisionFund, operated 25 branches in rural areas, supporting both farms and enterprises with group finance and financial literacy training.
• Paraguay. The MFI partner, Fundación Paraguaya, offered group finance, financial literacy training, and discount cards for medical services, groceries, and movie theatres to female clients. ♦
Client Training in Myanmar, During Pandemic
CONTEXTUAL FACTORS
A variety of contextual factors severely impacted partners and moderately impacted the study. These included pandemic, economic recession, and natural disasters in Uganda; civil war and control group loan takeup from other providers in Myanmar; pandemic-related limitations impacting services and loan use in Paraguay.
• Uganda: Extreme natural disasters forced closure of the study. Unfortunately, the program was disrupted by natural disasters – including floods at the beginning, locusts partway through, and drought for two consecutive seasons toward the end – as well as by the pandemic, leading to its cancellation mid-way through the study.
• Myanmar: Pandemic and domestic conflict led to end loan disbursements in final year. The program initially proceeded as planned but was severely affected by the pandemic and civil war. Domestic travel restrictions, lockdowns, and stay-at-home orders, combined with a politically-motivated internet shutdown, hampered operations. To support borrowers, the program modified loan repayment schedules, offered digital repayment options, reduced the number of group meetings, provided insurance for medical expenses, and implemented COVID-19 prevention measures. In spite of these efforts, clients began defaulting on their loans. The Portfolio at Risk (PAR) – the value of the outstanding balance of all loans in arrears more than 30 days – rose from 0.1% in FY 2019 to 48.3% in FY 2023. When clients default, their group members have to repay their loans, further straining these clients financially and reducing demand for loans. As the economy stagnated,
IMPACT STUDY RESULTS
demand for loans decreased. Loan disbursements fell from USD 3 million in Fiscal Year (FY) 2019 to USD 500,000 in FY 2023. The number of borrowers decreased from 105,672 in FY 2019 to 48,049 in FY 2023. Lower borrowing means lower impact on client income and assets. In addition, an unexpected surge in other lenders in the target areas led to a much higher than expected number in the control group borrowing from other microfinance institutions, further reducing the impact difference between VisionFund clients and the control group, making impact harder to detect.
• Paraguay: The pandemic decreased client take-up rate making it difficult to detect effects. The program was implemented partly as planned. However, the distance of clients from the sources of “plus” services – health care providers, grocery stores, and movie theaters – turned out to be a bigger challenge than anticipated, inhibiting clients from accessing discounts provided by the program. While trainings and group meetings took place for a portion of the program, pandemic-related lockdowns inhibited the frequency of group meetings, the provision of in-person training, and other plus services that could include activities such as business fairs. Further, as was the case in Myanmar, client demand for loans declined due to some clients defaulting on group loans and due to weak business performance, economic stagnation, and uncertainty. The client retention rate dropped to 50% compared to the usual 78% retention rate. With lower borrowing, impact of loans on client businesses was also lower than expected. The study was generally implemented as planned, although the first follow-up survey took place later than planned because of COVID restrictions and difficulties in reaching respondents via phone ♦
Positive changes in income and assets. Despite contextual challenges, the study demonstrated asset increase in Paraguay and income and asset increases in Myanmar.6
• Myanmar: Clients experienced a 40% increase in Year 1 and 52% increase in Year 2 of the study. Due to the pandemic and political unrest, loan disbursements were stopped in Year 3, dropping income readings to 12.4%.7 While this last coefficient did not exceed the statistical significance threshold, a statisticallysignificant, medium-level change in assets that same year (0.123 standard deviation) suggests that previous income was impactful and was invested in assets.
• Paraguay: There was a 7.2-8% increase8 in incomes in Years 3-4 and a statistically significant mediumpositive level affect in assets (0.09 standard deviation) Here, again, the statistically significant change in assets cannot be possible without income change – suggesting that the income increase is more significant than statistics suggest. ♦
6 No statistically significant effects were found for consumption, employment, food security, access to health care and education. However, it is possible that they do not show up statistically because study participants are making the choice to invest rather than consume, or spreading the qualitative changes to consumption to the point that the changes are less discernible, lowering the averages across the board sufficiently so that measurements do not exceed the minimum detectable effect size.
7 The Year 1 and Year 2 coefficients on income exceeded the threshold that enables reseaerchers to assert that these figures are statistically significant. The Year 3 coefficient did not. These thresholds are established by margin of error calculations that rest on sample size and effect size – so they speak to degrees of confidence. What we do see in Year 3 is a statistically significant impact on assets which could not have been possible without income change.
8 The income increase coefficients did not exceed the threshold that enables researchers to assert that this is statistically significant. However, perhaps most importantly, asset increase – which was statistically significant – could not have occurred without concomitant income increase. The indicators are moving in a positive direction, and had the pandemic not led to a drop in the take-up rate, the threshold for minimum detectable effect size would have remained lower keeping it statistically significant.
Client Training in Myanmar, Pre-Pandemic
The study underscores that high-quality microfinance-plus programs can positively impact client income and assets even under adverse conditions. The key insights from the study include the following:
• Regarding the 2015 RCTs that seemed to show the ineffectiveness of microfinance: These studies can no longer be taken seriously.
o The meta-study conducted as part of this project and the results from this study show this conclusively. Those who still refer to the old negative studies should, at the very least, be invited to read the meta-study (see links on Footnote 2) and Opportunity’s study findings.
• Regarding this particular study and what it showed:
o This study was the most carefully designed and implemented RCT to date. The care with which researchers engaged this study have made weaknesses of past studies apparent including flawed study design, insufficient attention to microfinance as actually practiced and its context, and tendency to make blanket statements about effectiveness.
o This study showed that “microfinance plus” can have positive, even very positive, impact on clients – even in difficult contexts. In Myanmar, clients showed 40-52% income increase compared to the control group in Years 1-2. In Paraguay, clients experienced a statistically significant medium-level positive impact on assets
o But, hindering factors can become severe to the point that positive impacts cannot be expected. No study showed negative impact in our study, but had factors been severe enough, they could have. Social restrictions and distance in Paraguay, and climate disasters in Uganda are examples of significant, hindering factors.
• Regarding broader discussions about whether microfinance is effective or not :
o Blanket discussions on whether “microfinance works or not” are too vague to be useful. We understand that under certain conditions, impacts can be quite positive, but under different conditions, they can be less impactful, or even negative.
o Discussions going forward on this broad topic must focus on conditions/factors that make microfinance more, or less, impactful and how to address them.
• Regarding the general theory of change that microfinance operations reflect. The study highlighted the importance of working with theories of change of microfinance as actually practiced :
o Past studies do not seem to have engaged such details. The various results from these studies did not undermine the theory of change – but further affirmed its efficacy.
• Overall Insight: The study shows that using appropriate study designs to assess high-quality microfinance-plus programs can capture the impact of microfinance on client income and assets, the drivers of poverty alleviation. The findings challenge earlier studies reporting null effects, highlighting the potential for significant positive outcomes with well-designed microfinance programs and RCT methodology. ♦
This study shows Opportunity’s strengths in:
• Microfinance work – The work of microfinance remains a significant intervention that continues to deserve respect for its potential to positively impact the lives of the poor.
• Collaborative Partnerships – Collaborating with strong partners to ensure robust program implementation, study design, and reliable findings.
• Research Management – Demonstrating the capacity to manage complex, rigorous, multi-year, multicountry research studies.
• Global Contribution – Positively contributing to, and influencing, global and industry perspectives and practices that empower the poor.