Impact Evaluation of the Program for the Development of the Industrial Districts in Brazil
Impact Evaluation Workshop Multilateral Investment Fund Washington D.C., October 15th 2010
Program background Assistance in business plan development, trademark strategy, quality assurance, market development, logistics, product design, marketing, technology transfer, and building strategic partnerships.
Training and technical assistance in many areas regarding technology and innovation, production processes, lay-out, equipment, rationalization of production, etc.
Organization of Production
Information and Market Access
Industrial District Program at a glance Funding: SEBRAE: $3,913,757 (65 %) MIF: $2,075,000 (35 %) Total budget: $5.98 million Components: - Information and Market Access - Organization of Production - Strengthening Industrial District Dynamics - Access of SMEs to International Markets
Program Goal Help increase competitiveness of four Industrial Districts in Brazil.
Actual Date of First Disbursement: August 15, 2003
Strengthening Industrial District Dynamics Awareness-building and business promotion to reinforce business collaboration and collective efficiency of the industrial districts. Creation of clusters among businesses (consortia, business condominiums, and service cooperatives)
Access of SMEs to International Markets
Actual Date of Final Disbursement: May 31, 2007 SMEs assisted: 591
Product promotion, consortia formation, participation in trade fairs and trade missions to other countries and distant locations within Brazil, and marketing and advertising support.
Program Outputs and Outcomes Summarized Sample
Outputs Strengthening Industrial District Dynamics
Outcomes Were SME Information and beneficiary’s Market Access operations strengthened?
Organization of Production
• Creation of “Forum Distrital” (Tobias Barreto)
• 2 market researches at the end of Yr2 (Nova Friburgo) • Marketing strategy implemented in the 1st 6 months (Campina Grande) • 17 companies completed business plan preparation to enable access to credit (Nova Friburgo) • A significant group of participants were paying between 10% and 20% of TA cost (All districts)
Access of SMEs to International Markets
• 3 trade trips and events at the end of yr1 (Nova Friburgo)
Outcomes
• Creation of export consortium with APEX support.
• SMEs improved regional and national trade shows and fairs presence expanding their markets.
• At least 25% of participants in Campina Grande increased productivity by 50%
• Increase of sales to other states and international markets
Key findings of Impact Evaluation
Assessment Rating (1) Highly Satisfactory (2) Satisfactory (3) Moderately Satisfactory (4) Moderately Unsatisfactory (5) Highly Unsatisfactory
Overall judgment
Strategic relevance
3 Moderately satisfactory
• Program was part of SEBRAE’s strategy evolution to support SME development through an experimental methodology. • Clarity of depiction of objectives (consistency at different levels, coherence, and being amenable to be translated into operational goals) leaves room for important improvements. • Top-down approach in the program design and execution did not take into consideration key knowledge absorption capacity factors of beneficiaries
Internal quality
3 Moderately satisfactory
• Program started with a diffused set of activities. It was only until the second year, as it gained traction, that activities started to be implemented with a sound link to main program objectives. • In general, beneficiaries have a favorable opinion on activities quality, however some activities exceeded beneficiaries’ required experience and capacities. • Program’s budget and duration fell short of completing all planned activities in order to reach cluster self sustainability impact in consonance with each district level of development.
Key findings of Impact Evaluation
Overall judgment
Dialogue and dissimenation
2 Satisfactory
• The program was successful in positioning the SME development discussion on the national public policy agenda and helping in creating and developing the concept of APL in Brazil. • SEBRAE’s broad scope of activities and programs, together with that of other national and regional economic development agencies provided additional level of support to SME development.
Stakeholder and beneficiary roles
3 Moderately Satisfactory
• SEBRAE’s recognition and experience in working with SMEs was key in reaching out, motivating participants and facilitating the implementation of the program. • By partnering with MIF, SEBRAE gained access to international project management experience role and added value came in the form of best practices for project design, implementation and M&E. • MIF participation in execution was limited to fiduciary aspects of the program and general execution follow up with little involvement on technical aspects.
Likely impact
3 Moderately satisfactory
• Program general outputs significantly vary per industrial district with unclear pattern for conclusive overall analysis and interpretations. • Outcome results had been sustained by some program beneficiaries, while some institutional and collective efforts have lost footing.
Lessons
Alignment of MIF norms and procedures with those of local partners could helpful enhance the synergy and strategic fit of the program.
MIF interventions will achieve sustained impact when program outputs and tasks are embedded within broader long term initiatives.
Moving beyond generic support, tailoring programs and projects to the circumstances of the country.
General terms of the program development (particularly in the design phase) open room for improvement in different areas.
Recommendations • Work on alignment and agreements on norms, procedures and other type of program management arrangement from the beginning of the design. • Include support (capacity building) with beneficiaries to overcome adverse situations beyond MIF intervention.
• Adapt strategies more towards varying local conditions (even within the same country), to move from "good practices" to "good fits”.
• Use of a differentiated approach. • Make a clear depiction of the task and activities to be developed for each component and district from the beginning of the program. • Clarify the different categories of participants and the M&E to be used to track their performance.