Global Health Training Programme - Evaluation Report 2014 by Anni Leppanen

global health training programme evaluation report 2014

October 2014

contents 4 7 8

executive summary acknoledgements abreviations and acronyms

chapter 1 Introduction

chapter 2 Summary of the project chapter 3 Context of the project

chapter 4 Logic of the project

chapter 5 Methods of the evaluation

chapter 6 Findings

chapter 7 Issues

chapter 8 Recommendations

chapter 9 Lessons learnt

annex A the evaluation team

annex B detailed description of the project

annex C methods in the evaluation

annex D survey methods and results

annex E list of meetings and persons i nterviewed

annex F list of main documents reviewed

annex G terms of reference

literature notes

Executive summary

THE PROJECT has been running since the year 1999 until today in periods of three years. The plan is to have an Exit Phase in 2015-2017. The main activity has been training courses in global health, at first for 8 weeks and later 4-5 weeks consisting of a theoretical part and a visit part. Trainees have been carefully selected and mostly young health professionals from Finland (50%) and from middle and low income countries. A total of nine courses have been held in Finland and nine in the four partner countries, which have entered the project at different times. A visit part has taken place in Finland for the trainees from partner countries. The Finnish trainees have visited a partner country, organised by the partner university in each. Also eLearning courses have been designed and made available for the wider public.

project to provide advice to the Exit period and to possible continuation of the activities through other means. The evaluation was conducted by Dr Anneli Milen assisted by a junior expert Anni Lepp채nen. Methodology included also conducting a survey and a mission in Nepal. THE FINDINGS showed weaknesses in the hierarchy of objectives and in its logic hence the impact could not be assessed and effectiveness was evaluated only through achievement of the pre-set results, not achievement of the purpose. Results were phrased vaguely and in general terms with no indicators, hence they were difficult to measure and evaluate. RESULT 1 on a gap to be filled in the Finnish medical education was achieved well among the Finnish trainees, yet only among those few selected to the courses. The undergraduate medical education system has not changed in the meantime as it has in many other countries to include global health (GH). However, today there are many other opportunities e.g. eLearning courses and exchange programmes that are more cost-effective than a separate course. It is recommended that the project in Exit phase in co-operation with other actors would put pressure on the Finnish medical faculties and assist in integrating GH in undergraduate education utilising the developed model with globalisation and multidisciplinary approach and the five eLearning courses designed.

THE FUNDING has been received from the Finnish Ministry for Foreign Affairs, Development cooperation funds for NGOs, a total of 2 million euro during 2001-2014 with bias on the later years. About 15% has been self-financing in money or kind by the leading organisation The Finnish Medical Society Duodecim and by the Finnish Medical Association. The role of the University of Tampere team has been central. Most of the funds have been used for payment of the Finnish staff and international flights of the trainees, but volunteer input is also worth mentioning.

RESULT 2 on raising awareness among the trainees was achieved as most trainees reported a change in mind-set and thinking as well as having learned much on the Global Health courses especially through a visit to a low or middle income country (Finns, 80%) or in Finland (the others). Several indicators in the survey we conducted showed that awareness was not well sustained: the trainees had not studied the project eLearning courses on their own and most reported not to follow any websites, journals or newsletters on GH or did networking. The motivation and interest that the courses have evidently created are not easy to keep up. We recommend attention to be paid to provide more practical information on opportunities to participate in volunteer work and NGO activities. Also, the alumni could be engaged more in functions of the global health units established in the partner universities, as has been positively experienced in Chile and the Philippines. The eLearning courses could also be designed to be interactive and international with group discussions and joint assignments done via the Internet.

THE EVALUATION initiated by the project management was to study the relevance, impact, effectiveness, efficiency and sustainability of the

The project has a good idea to advocate global health and increase awareness on global health in the Finnish society also. We recommend that the

initiative could be worked on further: we suggest organising and funding events in the future in partnership with other organisations, to do much stronger marketing and base the programme on clear pre-defined objectives. To improve effectiveness in learning we recommend systematic planning tools such as problem based learning or competencies based learning to define specific learning targets and competencies to be achieved and learning of the alumni to be evaluated against these definitions and targets. The revised model with possibly blended learning could be used, with local modifications, in the partner countries in their undergraduate medical education. RESULT 3 aimed to provide an entry-point into international positions to the trainees. It has been assumed in the project that the GH course would be a merit when applying for international work. There are increasingly also other opportunities to study global health through the Internet achieving a certificate and also to take such courses abroad as part of student exchange programmes. Tailor-made visiting programmes of the project have evidently offered more than any ordinary visit. However, concerning Finnish students there are today several other opportunities to gain the first experience of a developing country, for example through student exchange and volunteer programmes. Furthermore, global health is much wider than issues related to developing countries - it is about the globe and globalisation. Another assumption in the project has been that through the course the participants get motivated to seek international work. In our survey among the Finnish alumni, 40% reported that they started to seek for international jobs abroad or in Finland or started to look for GH related work or modify their work so that GH was part of it. Only 35% has done any GH related work after the course, which may indicate the difficulty in getting such a job. RESULT 4 aimed to enhance international network of students, teachers, and scientists. The project has managed to identify motivated and serious institutional and individual partners. Using individual contacts and setting clear criteria for a potential partner institution and team have been crucial and successful. Also, having partners from different continents has been a great idea enhancing â&#x20AC;&#x153;globalâ&#x20AC;? aspects in the project. Networking among the partner institutions seems to have focused on organising the courses, not so much on global health topics or information itself. Meetings intended to the teams could have been targeted to them. There seems to be no formal or informal network structure as such

among the partners. Also, networking should take place more widely than among the partners, and the project could assists in this. The network among the participants in the courses has naturally taken place during the courses and exchanging emails immediately after it. However, there is no networking structure of the alumni as indicated in our survey. On the other hand, it cannot be expected that alumni keep in close touch for many years. We recommend that the project would link the trainees into wider networks internationally. RESULT 5 is about increasing institutional capacity of partners. This project is one of the few that has included organisations in low or middle income countries aiming to strengthen their capacities in training in global health, as global health today is largely a discipline of the North. In the interviews, team members in Chile, Nepal and the Philippines reported that the need for global health training at their universities was recognised as a result of 1-2 key persons of the faculty having attended a GH course in Finland. This did motivate the faculties to be involved in the co-operation: to send participants to Finland and to receive Finns for the visits part. This way, an essential pool of GH experts was created. In Chile and the Philippines, global health activities have been started in the early years of the project. In Nepal, a GH Unit was established. It has renovated the premises and named the key experts. The activities are yet to be developed as well as the human resources assigned by the Faculty.

â&#x20AC;?

The key challenge is how to ensure that global health training and learning will continue after the exit of the project and its funds. There is a clear need to improve sustainability.

There is, however, a great need to strenghten the institutional capacities in the partner organisations to improve sustainability. In order to enhance the capacities to carry out GH training courses, the project has organised only two two-day sessions for training of trainers during the years. In the recent one, in 2014, an important issue of modern learning methods was discussed and it gave a good start to emphasise more

the defining of clearer and concrete objectives and competencies to be achieved rather than merely raising awareness in GH courses. The latter part of the Result, on building capacities in research on GH, has no planned activities in the project documentation. The key challenge is how to ensure that GH training/ learning will continue after the exit of the project and its funds. There is a clear need to improve sustainability. Improving sustainability requires capacity building based on jointly identified needs of the partners. The capacity building should not be oneway traffic but the competencies that each partner team already has should be utilised: for example, the Nepalese team and the Tanzanian new team know much about competencies based learning design, the team in the Philippines has a lot of experiences in advocacy and concrete action and the team in Chile is gaining experience in regional co-operation in organising GH training in Latin America. Also, the work on basing the learning on objectives and competencies needs to continue further. ASSESSMENT OF THE RELEVANCE showed that the need for knowledge on global health among health professionals is without any question. This is the case in the developed world, and even more in the developing countries, which are influenced greatly by globalisation - in positive and negative. In this perspective, the projectÂ´s to include low and middle income countries is important and exceptional. Yet, health professionals need not only to be aware and to know, but also to build skills, networks and channels to advocate global health issues for concrete action and decisions. The lessons to consider are related to dialogue and sustainability /dependency. For the GH training to influence the global health concerns in a country, it needs to focus and address issues important and relevant to the country and in its context and reality. The same holds to building competencies in advocacy, political influence and networking.

The large parts of expenditure are salaries of the Finnish personnel in the project and travel between the countries and accommodation of the trainees. From the environmental and sustainability point of view, the extensive travel between the partner countries is questionable. The management of the project has been of high quality in implementing the project plans. The Finnish consortium of partners is large. Only very few members participate in any other way than paying their part of the self-financing. Activating the Finnish partners is a difficult challenge and could be more thought of in the Exit phase. SUSTAINABILITY is a major concern in development co-operation. From the very beginning every project should plan and implement systematic actions to gradually ensure that the results will sustain and/or activities will continue after the donor has pulled out. Mere capacity building of individuals seldom results in sustainability, but a variety of actions are needed to ensure commitment, integration into existing systems and functions as well as human and financial resources. As the partner institutions function in quite different realities, we have discussed in the report sustainability issues separately on each partner and country. As a summary, we recommend the Exit phase to be redesigned in order to enhance sustainability and give detailed suggestions on approaches and activities. THE EVALUATION should be viewed in the context. During the over 14 years of the project much has changed. Opportunities for getting professional experiences in any part of the world have expanded rapidly. The web and eLearning are providing increasingly more options for learning and finding information also on global health. Furthermore, the concept of global health has changed profoundly.

EFFICIENCY as to the use of the allocated funds was found to be good: with proper management, the project has been able to make savings in the budget. The team in Nepal did not consider they needed onethird of the funds allocated to the course organised in Nepal. The team in the Philippines saved also some money, which they used to partly finance their own courses offered by the faculty. In Tanzania, in the course in 2012 the budget was exceeded. (Chile has not received any funding for organising courses as it is not eligible for development aid.)

ACKNOWLEDGEMENTS The evaluation process has given us an opportunity to work with highly committed experts, and to learn ins and outs of this development co-operation and training project.

We wish to express our heartfelt gratitude to the following individuals.

Dr Ulla Ashorn at the University of Tampere, for patiently providing pieces of information as per our endless requests, Professor Per Ashorn, from the University of Tampere, one of the initiators of the project and actively developing it throughout the years, for the open attitude to discussing and sharing the intentions and ideas in the project, Dr Juha Pekka Turunen, from the Finnish Medical Society Duodecim, who as been actively involved throughout the years, for being keen to learn more about the specifics of development co-operation and its evaluation and openly sharing information, MSc Anna Pulakka, the present co-ordinator of the project, Dr Ulla Harjunmaa, the present volunteer Chair of the Executive Board and the former co-ordinator, and Dr Kirsi-Maarit Lehto, who has been taking care of a huge number of arrangements and financial accounts in the project and tirelessly providing pieces of information to us, The team in Nepal: Professor JP Agrawal, Dr Archana Amatya, Dr Santosh Timalsina, Professor Mandira Shahi and Dr Pradip Gyawali and many others, for very open discussions on the project and its possible future, The Nepalese participants on the course in Nepal and in Finland in 2013, for making the effort to travel even long distances to our interviews, The team of the Philippines, Dr Edelina Dela Paz, Dr Gene Nisperos and Dr Louricha Tan; the team of Chile, Dr Alejandra Fuentes GarcĂa and Dr Cristian Rebolledo, and the new team in Tanzania, Dr Elia Mmbaga and Dr Germana Leyna, who dedicated time to their busy schedule, in June 2014 while in Tampere, for the interviews and providing additional information on our request, Dr Tom Sundell for discussing the previous evaluation and Dr Taneli Puumalainen, a member of the former project team with large experience in development cooperation, for sharing his views, The Finnish participants of the project courses in Finland and abroad for responding to our survey, All trainers and facilitators of the courses in Finland, and in a number of partner countries, who made this project possible and, The Ministry for Foreign Affairs of Finland and its Unit for NGO development co-operation, for invaluable discussion on the findings and recommendations of the evaluation. Finally, we want to acknowledge all the people around the globe, who have to struggle with problems in health and wellbeing due to lack of understanding and action on global challenges by health and other professionals and politicians and other decision-makers. This project has been a small grain of sand in a desert, yet many grains can form a storm of change.

Picture 1. Meeting of project partners, August 2014, Tampere, Finland

ABBREVIATIONS AND ACRONYMS MFA

Ministry for Foreign Affairs in Finland

GH Global health OECD/DAC The Organisation for Economic Co-operation and Development / Development Co-operation Directorate EU European Union ToT Training of Trainers TOR

Terms of reference, agreement

CHAPTER 1 INTRODUCTION Evaluation is a systematic and objective assessment. It focuses on the programmeâ&#x20AC;&#x2122;s design, implementation and its achievements. An evaluation should provide information that is credible and useful, enabling the incorporation of lessons learned into the decision-making process of both recipients and donors. The OECD/DAC definition of an evaluation

1.1 Importance of global health World is gradually becoming aware of the number of important links between globalisation and health. Decisions are required from politicians, yet health professionals are in the key position to disseminate information and also to advocate for action globally, nationally and locally. Triple burden of disease everywhere Global health as a topic has expanded from interest in triple burden of diseases everywhere developing countries to threats anywhere in the world. Such dangerous diseases as HIV, Bird-flu and Ebola have or may expand to anywhere due to the increased traveling of people and the inability of the fragile states to control them. Besides infectious diseases, chronic illnesses (cardiovascular diseases, cancers and diabetes) have become a major problem also in many developing countries, while the health systems are not able to cope with them. These two types of illnesses form now a double burden of disease everywhere in the world. Global health issues today are much more than diseases and their causes. Global health is about socioeconomic impacts on health, and about inequality in health and access to health care between countries and between population groups within a country. Globalisation influences health - triple burden And yet, global health is even more than that. It is about transnational foreign companies, or local ones violating basic labour rights such as use of

â&#x20AC;?

child labour, poor wages and long working hours or such an extreme case as in Qatar where Nepalese migrant workers commit suicides at the World Cup 2022 constructions sites. Global health is concerned about environmental protection such as negligence in several African states where Chinese or Taiwanese garment industry causes dangerous water pollution. International trade agreements are part of global health as they may open national health services to transnational companies seeking maximum profit and some of them avoiding paying national taxes. Intellectual property agreements are also of great importance to global health and they may prevent or allow expanded delivery of cheap medication on major diseases. Global health is less than before only about diseases. Globalisation relates to the third burden of disease everywhere in the world. It is less about treatment and control of exotic diseases or the threat of them entering the rich countries, and more about viewing the globe as a whole where everything influences everything. It is less of only awareness, and more on advocacy and action of health professionals, politicians and civil society.

1.2 Reasons for the evaluation Why to evaluate? Evaluating a development co-operation project is part of the internal control and monitoring system of the MFA. Its purpose is to improve quality and promote

accountability in development co-operation and development policy. Evaluation is regulated by the MFA’s internal norm on development evaluation.1 The evaluation was designed to serve the three main general purposes of the Development evaluation as defined in the MFA`s Evaluation Manual: • Learning for improvement of future development policy and programmes. To achieve this there must be systematic feedback and use of lessons learned in planning and implementation processes. • Accountability for the planning and implementation process, results and impact, including the provision of information to the public for continued commitment. • A platform for dialogue amongst stakeholders during the evaluation process. This evaluation is a thorough external evaluation of a Finnish NGO Development co-operation project – not commonly undertaken. It could provide valuable information and insight to the NGO Unit in the MFA and other NGO projects. Initiative of the project In the late 2013, the project management made the laudable initiative to evaluate the project. With the acceptance of the MFA desk officer, a part of the MFA allocated project funds saved from the planned activities were used to issue an independent evaluation of the project. The reasons for the evaluation, as the project management members expressed it, were to: “Find out what the project has achieved and what difference it has made.” Also, it was to learn for the next phases of the project. This, however, could materialize only partly, as the results and recommendations were not available in May 2014 - the time for the funding application for the next phase - due to delays in the process of issuing the evaluation. Past evaluation The first years of the project, 1999-2004, were evaluated in 2005.2 During this period, the project organised the first three Global Health (GH) courses in Finland with participants from various, selected developing countries, and with the Finns taking the field visit parts in the collaborating countries. In his excellent report, Dr Tom Sundell drew conclusions and made recommendations that still are most valid. We will refer to them in this report. In 2009, the project management carried out a survey among participants from different countries on the first six courses held in Finland. Its goal was to find out about achievement of the results. Its usefulness suffered from a very low response rate due to the outdated contact information.

Exit In the evaluation, we did not take the perspective of the past but rather the perspective of the near future. It is planned that the project will phase out (end, exit) in the next three years. It may also continue in some other form and also in that case, the evaluation should look at the future development. The evaluation is most timely, since the Phasing Out or Exit period will offer a great opportunity to focus on the main needs identified in the evaluation.

1.3 Aims and strategies in the evaluation The evaluation team made the following main strategic decisions: • The Project needs to be evaluated first of all as a development co-operation project and only then as a training/education project. • Regardless of the project not being a typical NGO project, it has to be evaluated using International and Finnish MFA guidelines for evaluation of development co-operation. This means first of all that internationally accepted criteria for development cooperation evaluation is to be used, as terms of reference state: • • • • •

Relevance Impact Effectiveness Efficiency Sustainability

The evaluation is not intended to be a detailed description and assessment of the activities of the GH project during its over 14 years of existence, but rather it focuses on the major, fundamental issues. The aims as in the Terms of Reference were to: • Assess the relevance on development needs of the countries • Evaluate the impact (contribution to development objectives) • Assess effectiveness (purpose and results) • Assess efficiency of the operations • Evaluate sustainability (what will remain) • Provide concrete recommendations for the Exit Period • Identify practices and lessons learned for the future design of similar projects As discussed later, the evaluation focused on achievement of the results and sustainability and on the Exit Period. Dr Anneli Milen was invited to conduct the evaluation. She is responsible on the study and the views expressed, while Junior expert, Anni Leppänen assisted in the work (for the bio-sketches, see Annex A).

CHAPTER 2 Summary of the project This is a brief description. For more information, please see Chapter 4 for the Logic of the Project and Annex B for a detailed presentation of the project. Project for awareness The aims as stated in the project has been to increase awareness of health professionals on global health issues and to provide an entry point for international work and assignments for health professionals. The focus has been mainly on young persons. Most of the training has been provided to Finns while during the recent years, projectfunded courses have been organised also by the partners in Chile, Nepal, the Philippines and Tanzania. Academic education and development co-operation The project is simultaneously about training and education and about development co-operation, which makes it challenging to implement. It is relatively large due to the scope of global health and the number of partner organisations in Finland and abroad, as well as its costs and duration. During over 14 years of operation, it has trained around 500 final year students or health professionals, mainly medical doctors and some dentists and veterinarians, from Finland and selected low and middle income countries. In fact, there has not been only one project designed in the beginning for the 14 years but rather a sequence of projects of 3-4 years. During its life, some activities besides the courses have been added such as designing eLearning courses. The role of the partners in the low and middle income countries has expanded to organise project funded courses besides sending their trainees to Finland and organising field visits to Finnish trainees.

The leader organisation has been the Finnish Medical Society Duodecim, which has the official responsibility towards the funding agency. The University of Tampere, School of Medicine, Department for International Health has had a major role as it hosts most of the project staff being responsible for organising training courses in Finland and for supporting the courses organised in the four partner countries. 14 years and 2 million euros The main funding source for the project since 2001 to date has been the Finnish Ministry of Foreign Affairs (MFA) Development Co-operation Funds, specifically the funds allocated for co-operation between non-governmental organizations. The total amount funded by the MFA is two million euros during 2001- 2014. In addition to this, the Finnish partners have provided 15% of the grand total in cash and volunteer work. The Finnish Medical Society Duodecim and Finnish Medical Association have made the largest contributions. Courses in Finland The main activities of the project have been organising training courses on global health themes for the length of 8 weeks (2001-2004) or 4-5 weeks (2006-2013). At first, the courses were held only in Finland, having half of the participants from Finland and the other half from some selected low and middle income countries. Half of the course has consisted of so called field visits: the foreign participants have remained in Finland after the theoretical part, while Finns have travelled abroad for a visit programme (recently to Chile, Nepal, the Philippines or Tanzania). Courses in four partner countries During 2008-2012, project-funded courses have been organized, besides in Finland, also by the partners in Tanzania and the Philippines. Participants have been Finns and nationals and the courses have included also a field visit part. Recently, in 2013 and 2014, the partners in Tanzania, the Philippines, and in 2013 for the first time in Nepal, have organized a course with participants from all the other four partner countries, including Finland. Besides training courses held, the project has published four eLearning courses in GH for open use. Also, the project has organised in Finland two half-day open Summits on major GH topics, two open seminars on updating on current GH themes, and two meetings to training course organisers.

FINNISH DEVELOPMENT CO-OPERATION DEVELOPMENT CO-OPERATION

GLOBAL HEALTH TRAINING PROGRAMME

TRAINING AND LEARNING

GLOBAL HEALTH

TRAINING IN GLOBAL HEALTH

Figure 1. The context of the Global Health Training Programme

CHAPTER 3 CONTEXT OF THE PROJECT The Global Health Training Project operates in a broad environment. It consists of five main domains that influence the project. These should be taken into account in the projectâ&#x20AC;&#x2122;s execution as well as in this evaluation.

3.1 What is global health? Actions for global health, including training, depend on how global health is defined.345 A consensus definition to this term remains elusive. There are still many different views co-existing and with some simplification they could be classified as follows, modified from the article by Frenk et al in 20146 and another by Beaglehole and Bonita in 20107, in which they excellently review the evolvement of the terminology and present new concepts. International health can be viewed as being about: 1. Threats of infectious diseases from the South to the North As international health since the early last century, the concern has been on control of epidemics across borders - mainly from the developing countries to the developed ones. International activities were, and still are viewed, unilateral and about aid and defence. 2. Making health technology available to all International health problems are seen in some important health initiatives, for example the Gates Foundation, to be solved through ensuring that all have the latest health technology available (medicines, vaccinations, information, etc.). The idea ignores the many influences of other factors and sectors on health such as social, behavioural, cultural, political, powerrelated and economic determinants. 3. Focusing on vertical, disease-specific programmes The rationale is that if we strengthen activities and resources to a disease at hand at different points of time and geographic area, for example HIV/

AIDS, polio and Ebola, we will be able to improve international health. The thinking is gradually changing to the need of strengthening entire health systems in order to impact a disease and to make a long-term and more sustainable difference. 4. Increasing foreign aid (development co-operation) to health Being necessary as such, this thinking conveys a notion that problems and risks flow from south to north while solutions and resources move the opposite direction. This is most out-dated as all nations are increasingly interdependent on each other on health protection and promotion. Also, a shift is taking place in global distribution of resources, influence, power and capabilities as the emerging economies, such as China, India and Brazil, are expanding. Global health is about interdependency of all The term international health has shifted to the term global health partly due to a linguistic change. Frenk et al. (2014) claim that â&#x20AC;&#x153;Global health is still identified with problems supposedly characteristic of developing countries and global cooperation in health with a sort of paternalistic philanthropy that is armed with technological developments of developed countriesâ&#x20AC;?.6 Hence, this use of the notion of global health fails itself to capture the essence of globalisation. There is a triple burden of health or disease both in developed and developing countries: of infectious diseases, of non-communicable illnesses and of those related to globalisation. There are two key notions

Health and population policies 6% Government and civil society 9%

Unallocated 34 %

Humanitarian aid 10%

Education and social infrastructure 5%

Agriculture, forestry and fishing 5% % Industry and business 5% Water and sanitation 4%

Multisector 16% Administrative costs 6%

Figure 2. Disbursements for development cooperation by sector in 2013

to be included: 1) global health is about health of the global population (=all people) and 2) global health is not a manifestation of dependence but rather as the product of health interdependence- a process that has arisen in parallel with economic and geopolitical interdependence.

years9. One reason seems to be a strong demand by students who see global health as an essential part of their competencies. Other reasons are perhaps the increasing globalization, cross-border movement of pathogens and international migration of health care workers.

All countries, even the most powerful, are interdependent with the other nations and in relation to globalisation such as global processes created to support production, trade, communication and movement of people. For example, a free trade agreement negotiated between EU and India may lead to expanded malnutrition. Or, release of environmental protection measures by a poor country to attract investment from an emerging economy may result in increasing health problems in long-run in both countries.

Programmes in global health have been established in undergraduate, graduate and continuous education of medical doctors, less to other health professionals. Such courses are available in all continents. While there is no worldwide data available, the CUGH (Consortium of Universities for Global Health) Global Health Program Database is a resource that provides some information on such training around the world. Almost 25% of U.S. medical schools in 2012 offered a structured GH track10. Over 35% of U.S. medical graduates felt that there was inadequate time devoted to GH in their schools11.

Global Society This important opening for redefining global health calls for global joint action. Nations, and their different actors, should construct gradually a global society based on the principles of human rights. All actors (states, firms, civil society, international organisations etc.) should accept to share the risks, rights and duties related to protection and promotion of health of every member of this global society. This requires a fundamental shift in thinking and would lead into major changes in approaching global health. Discussion continues The redefinition of global health as by Frenk et al. is also raising opposing views. De Ceukelaire and Botenga argue that we do not live in a global village but in a world where the distribution of economic, political and military power is extremely inequitable killing people on a grand scale8. Therefore, global solidarity as an ethical concept is not an answer. A regulatory framework is required to deal with imbalances. Defending sovereignty of a nation offers, in their view, a defence against the narrow selfinterest of global economic forces. AS A CONCLUSION, it is of great importance that the global health training project takes into account the changing notions on global health. This makes design and implementation of such training even more complex as it requires broad understanding on globalisation, and more challenging as global health is essentially more political and related to powers and ideologies than previously. In the evaluation of the global health project, the broader definitions on global health must form the context against which to evaluate.

3.2 Training in global health On increase Global health teaching in under-graduate medical education has markedly grown over the past ten

The survey carried out by Rawson et al. in 2007 of medical schools across the world indicates that teaching in global health is rising in prominence, particularly through global health elective/exchange11. It was also found that teaching of subjects such as globalization and health and international comparison of health systems was becoming more prevalent indicating that global health teaching is moving away from its previous focus on tropical medicine towards issues of more global relevance. In his key note speech Building a Global Health Curriculum in the CUGH 5th Global Health Conference in 2014, Timothy Brewer pointed out that there is very limited data on global health education, lack of consistency in definitions and approaches in training and little information on competencies students are expected to acquire. However, Battat et al. have noticed some concordance on recommended topics12. Regional and global networks mushrooming An indication of the growing interest on global health education or training of health professionals (and others) is the establishment of the Consortium of Universities for Global Health (CUGH) in 2008. Placed in USA it is rapidly growing with over 130 academic institutions and other organizations from around the world. Its support functions for global health education include: â&#x20AC;˘ Defining the field and discipline of global health, â&#x20AC;˘ Standardizing curricula and competencies for global health, including in medical education and â&#x20AC;˘ Developing, collecting and distributing global health educational materials, including eLearning modules. It is starting a Global Health Educational Program Advisory Service to help its member institutions to develop new, or expand and improve an existing

global health educational programme.

2.4 Part of the Finnish development co-operation

The Latin American Alliance for Global Health (ALASAG) is a network of academic institutions created in 2010 to strengthen global health in the Region and to have a voice in Latin America and the rest of the world. Its mission is to promote Global Health through teaching, training, research and technical cooperation in Latin America by the collaboration of different institutions.

The project evaluated is first of all viewed as development co-operation as it has been funded by such funds.

The World Federation of Academic Institutions for Global Health was established in 2013 to link and represent globally a number of regional bodies and networks that coordinate academic institutions with a commitment to public and global health as well as coordinating bodies for National Societies of Tropical Medicine and International Health.

3.3 Learning and instructional design To support learning is arts & science One domain that the project functions in is the world of learning or training. Organised training should be based on instructional design. It is a practice of creating instructional experiences which make the acquisition of knowledge and skills more efficient, effective and appealing. It is basically about finding out what the current state and needs of a learner are, defining the end goal and planning an intervention13. Most of the instructional designs include five phases: analysis, design, development, implementation and evaluation. For each phase, a number of different tools have been developed based on theories on how individuals learn or how different levels of knowledge or skills are acquired. Medical education is progressing In medical education, training methods have long been traditional with insufficient attention to what needs to be learned, how is can best be learned and if it has been learned. In recent years, many initiatives have been made to use instructional design and its tools to improve medical education. These include evidence-based training, problem- based training or competencies based education. Several books have been published on improving medical education. Nevertheless, also in undergraduate, graduate and continuing education of health professionals on global health, the science of learning and instructional design know-how is essential. In this evaluation of the Global Health Training Programme, we have examined the trainings against this background of learning and instructional design. However, the main focus is on development cooperation criteria such as relevance, sustainability, effectiveness and others.

Finland’s Development Policy has four areas of emphasis:14 1. A democratic and accountable society that promotes human rights. 2. An inclusive green economy that promotes employment. 3. Sustainable natural resources management and environmental protection. 4. Human development. Three Crosscutting Objectives should be a part of all development activities: • Gender equality • Climate sustainability • Reduction of inequality Global health fits into the policy on “human development” as well as with the cross-cutting themes of gender equality and reduction of inequelity. NGOs Finland uses diverse forms of cooperation with an individual country, such as budget and sector support, as well as cooperation with civil society organisations (CSOs and NGOs). Cooperation carried out by NGOs plays an important role. Direct contact between the civil society and in hands-on promotion of the poorest population groups’ livelihoods are viewed as the strenghts of NGO development work. In 2012, Finland´s support for NGOs totalled EUR 103 million (about 10% of the total public development assistance). Over a half of this has been channelled through the so called Partner Organisations: eleven experienced Finnish non-governmental organisations, the largest ones being religion-based. About 300 Finnish civil society organisations take part in development cooperation activities either by implementing projects or by disseminating information about development policy and development issues. The Ministry for Foreign Affairs supports projects run by Finnish organisations in over one hundred developing countries. The objectives and principles of civil society organisations’ in development cooperation include the same as in the overall development policy with some specific objectives. The MFA has provided detailed rules and guidelines for NGO development cooperation in a manual.15

Figure 2. Disbursements for development cooperation by sector in 2013 16

CHAPTER 4 LOGIC of the PROJECT

We made an attempt to describe the project, as there were no ready-made descriptions or project plans, other than the funding applications on the MFA format. A detailed description is in Annex B. Here we present the objectives at different levels including the activities as the hierarchy and the logic of the objectives play a central role in an evaluation.

4.1 Objectives The project objectives at various levels have been phrased differently depending on the document. As such they have not been expressed as a logical, written hierarchy of objectives (or a results chain or a theory of change) in terms of potential impact, effectiveness and efficiency. The hierarchy of objectives provides a useful tool in planning and in implementation. Examining and re-examining the logic and rationale within the hierarchy helps in â&#x20AC;&#x153;doing the right thingsâ&#x20AC;? in a project. We have gathered the information on objectives from the project applications during the 14 years and attempted to organise a hierarchy of objectives. We wish to emphasise that the objectives at different levels have been defined and phrased by the project, not by us. We use these objectives phrased by the project as the basis for the evaluation.

DEVELOPMENT OBJECTIVE: Reduction of poverty, enhancing human rights and prevention of global environmental problems. OR Alleviation of global ill health and diminished regional variation in health status.

PURPOSE OF THE PROJECT: Trained health professionals able to adapt, design and implement health care interventions and target country level health priorities (2013).

RESULT 1 Raised awareness of Global Health: young health professionals and Finnish society.

Organise and fund GH training courses in Finland (9) and partner countries (9). Organise and fund half-day summits (2) to Finnish decision makers, politicians and lay people.

RESULT 2 Entry-point into international positions and projects for young health professionals.

RESULT 3 International network of students, teachers, and scientists interested in GH.

Organise and fund training courses in order to increase interest of health professionals to seek for and to get GH related work.

Invite participants (30) to each course from each partner country Invite a teacher/ facilitator to each course from each partner country

RESULT 4 Institutional capacity of partners in 4 countries to conduct training and research in GH. Invite core experts from partner organisations to courses in Finland Fund the GH courses organized by partner organisations Organise training of trainers

Design and fund eLearning courses (5) in Internet made available.

Figure 3: Hierarchy of objectives.

4.2 Activities of the project The project activities in different years are presented as a figure. Nine courses were organized in Finland in 20012013. Four project-funded courses were organized in Tanzania, 3 in the Philippines and one in Nepal. (In addition, the partners in the Philippines and Chile organised and funded courses of their own). Five eLearning courses were made available to the public. Four seminars were organized in Finland. For more information, please see Chapter 1.4 and Annex B.

1999

PLANNING STARTS

2001

1ST COURSE IN FINLAND

2002

2ND COURSE IN FINLAND

5TH COURSE IN FINLAND

2007

1ST COURSE IN TANZANIA

2008

6TH COURSE IN FINLAND 1ST COURSE IN THE PHILIPPINES

2009

2ND COURSE IN TANZANIA

2010

2012

2011

3RD COURSE IN TANZANIA

7TH COURSE IN FINLAND 2ND COURSE IN THE PHILIPPINES eLEARNING MODULE 1 (FINLAND)

SUMMIT & SEMINAR IN FINLAND TRAINING OF TRAINERS eLEARNING MODULE 2 (FINLAND)

2013

9TH COURSE IN FINLAND 1ST COURSE IN NEPAL eLEARNING MODULE 3 (THE PHILIPPINES) eLEARNING MODULE 4 (NEPAL)

2014 4TH COURSE IN TANZANIA 3RD COURSE IN THE PHILIPPINES SUMMIT & SEMINAR IN FINLAND TRAINING OF TRAINERS eLEARNING MODULE 5 (CHILE)

Figure 4. Timeline of the project activities.

CHAPTER 5 METHODS OF THE EVALUATION

tic information of the achievement of the objectives against the set indicators - P: monitoring and reporting system only on numbers of courses and trainees. In this project, the criteria for evaluability are not met, as described in Chapter 4. To be able to evaluate the project, we collected objectives from the project applications during the years and organised them into a hierarchy. We defined indicators and target values to the project defined objectives including results and aimed to measure them in the evaluation. When intended results, objectives and indicators and target values are not clearly defined, logical and monitored, the project implementation may suffer as plans may not be revised due to lack of information on “how far we are and what we have achieved” against the pre-set targets.

5.2 Evaluation criteria and matrix In Table 1 (next page) we summarise the evaluation criteria, main evaluations questions, indicators and methods of collecting information for the evaluation. Detailed information on the criteria and methods is available in Annex C. The survey methods and results are in Annex D.

OECD/DAC16 and EU Guidelines17 and especially the MFA Evaluation Manual18, in line with each other, form the basis for the evaluation process. Most importantly, they emphasise independency of the evaluators and the criteria to be applied against which to make the assessments. The starting point in the evaluation was to discuss and agree on the Terms of Reference of the evaluation (Annex G). The expectations and needs of the project team related to the evaluation and its use were also elaborated.

5.1 Evaluability is problematic To be evaluated a project it must be “evaluable”. The basis for an evaluation and evaluability is established already during a planning phase of a project. Compared to the criteria (1-4) that make the basis for evaluation sound, as defined in the MFA Evaluation Manual, there were some problems in this project (P): 1. Objectives are clearly defined at different levels - P: defined but do not form a logical hierarchy. 2. Objectives at different levels describe what will change - P: are too general, not specific and clear, do not describe a change. 3. Indicators and target values are defined for all objectives - P: indicators and target values not planned. 4. Regular, indicator-based monitoring and reporting system is in place in the project introducing systema-

5.4 About the methods There was no obstruction of a free and open evaluation process, which may have influenced the findings. Sharing of information and views was open. Besides matters related to the hierarchy and specificity of objectives, the main limitation in the evaluation is that the resources were not used in an optimal way. Much time was required to get and organise the documentation and facts of the project due to weaknesses in documentation and archiving during the past 14 years. Also, due to the problems of out-dated contact information, a big effort had to be made to get the response rate of the survey high enough to allow generalisation of the results. To verify the findings when using a method, we applied triangulation, i.e. checked the findings using another or two other methods. The main findings and issues came up clearly regardless of a method used. We made a special effort in the interviews to create an open atmosphere with discussion among colleagues rather than strict interviewer-interviewee relationship and question and answer technique. Having a member or members in the evaluation team from the partner institutions would have been valuable. For example, there was far too little opportunity to discuss with the partners from other countries than Finland and Nepal.

Table 1. The Evaluation Matrix. Sources of data and/ or method of collecting

Criteria

Evaluation questions

Indicators

RELEVANCE Are the results, purpose and overall objective of the project relevant and in line with the problems, needs and aspirations and priorities of the target group and the policies of the partner country and donor agency?

1. Whose sustainable development the project aims to promote, i.e. a) what kind of countries were selected and b) who benefit from the project? 2. Is the theme of Global Health as defined in the project relevant (important) to the five partner countries and institutes? a) Are the results, purpose and achievements of the project in line with the problems and challenges existing in the partner countries and institutes? b) Are the objectives and achievements of the project consistent with the policies of the partner countries and institutes? 3. What is the intervention logic to reduce poverty and inequality? Is it defined whose poverty and inequality? 4. Are the objectives and achievements of the project consistent with Finlandâ&#x20AC;&#x2122;s development policy, also including cross-cutting themes.

Extent of the need to deal with GH issues (eg. international trade negotiations, health inequalities) Logical hierarchy of project objectives Direct and final beneficiaries (explicit, implicit). Integration of crosscutting issues in project design and implementation.

IMPACT Progress made towards achieving the overall or development objective of the project? Overall effects it has made; intended and unintended, long term and short term, positive and negative.

1. To what extent it seems the project has made impact in the five countries in reducing poverty through multisectorial action for health? 2. Through what has the project had impact on the lives of the poor women and men (access to health improving services, empowerment)? 3. What are the overall impacts of the project intended and unintended, long term and short term, positive and negative?

Could not be evaluated as the development objective was too general and broad.

EFFECTIVENESS Progress in the project made towards achieving the purpose.

1. Are the purpose and the intended and/or achieved results making a contribution towards reducing poverty? 2. Are the results properly set and measurable?

Analysis of the achievement of the results one by one.

All interviews, project documentation, survey to Finnish trainees.

EFFICIENCY Can the costs of the project be justified by the achievements? How well the various activities have transformed the available resources into the intended outputs (sometimes referred to as a result), in terms of quantity, quality and time?

1. How well have the activities transformed the available resources into the intended outputs or results, in terms of quantity, quality and time? 2. What is the quality of the management of the project, including e.g. work planning, monitoring and reporting, resource and personnel management, cooperation and communication? 3. Have important assumptions or risks been identified? Have risks been appropriately managed?

Overall expenditure and their division. Unit costs. Savings made. Expenditures of a course per item.

Project documentation. Interviews with project personnel in Finland. Thorough analysis of the expenditures of the course held in Nepal.

SUSTAINABILITY Will the benefits / achievements produced by the project be maintained after the termination of external support?

1. Will the benefits produced by the programme be maintained after the termination of external support? 2. What are the possible factors that enhance or inhibit sustainability, including ownership/commitment, economic/financial, institutional, technical, socio-cultural and environmental sustainability aspects? 3. Has the phasing out of external support been planned, and will the plan ensure sustainability?

Analysis of relevant country information. Search of literature. Project funding applications Project documents, all interviews.

All interviews, project documentation, application for the phase out period.

CHAPTER 6 FINDINGS Worldwide, the Global Health Training Programme is one of very few projects that has included organisations in the developing countries aiming to strengthen their capacities in training in global health.

6.1 Relevance The need for knowledge on global health among health professionals is without any question. This is the case in the developed world, and even more in the developing countries, which are influenced greatly by globalisation - in positive or negative way. Health professionals need not only to have awareness and knowledge, but also to build skills, networks and channels to advocate global health issues for concrete action and decisions. This project is one of the few that has included organisations in the developing countries aiming to strengthen their capacities in training in global health. Global health today is largely a discipline of the North:

Global health has become a lapdog of the geopolitically powerfulâ&#x20AC;Ś Global health has evolved as an instrument for a new era of scientific, programmatic, and policy imperialism. A discipline in which those who claim the right to study, speak, argue, publish, perform, and judge, are part of an apparatus of power, self-interest, and control that denies justice and dignity to billions of people worldwide. 19

(please read the Literature note No 19)

In this perspective, the projectÂ´s approach has been important and exceptional.

â&#x20AC;?

Dialogue to be enhanced The lessons to consider are related to dialogue and sustainability/dependency. For the GH training to influence the global health concerns in a country, it needs to focus and address issues important and relevant to the country and in its context and reality. The same holds to building competencies in advocacy, political influence and networking. Therefore, much dialogue is needed. The courses held in the partner countries funded by the project have followed mainly the courses in Finland with some changes for local needs. On the basis of the studies we made, in our view:

1. In the project, there could be more dialogue and exchange of views among all the partners on contents and priorities in global health training. 2. Competencies could be built even more to the needs of a country, as has been the case by the partners in the Philippines, who have developed their own courses according to their specific needs. 3. The management structure, as discussed later, could be redesigned to promote dialogue even more than has been the case this far. Relevance could be improved through sustainability The other point is about sustainability. As long as

there is full dependency on donor funds to carry out activities, as is the case here, there is a difficulty of having equal partnership. Independence is gained through self-sustainability. This is needed to enhancing also the relevance of the project.

6.2 Effectiveness Purpose to be measurable As said previously in the report, the achievement of the purpose of this project cannot be evaluated, The stated purpose (for example in the 2013 documents) is “Trained health professionals able to adapt, design and implement health care interventions and target country level health priorities” does not match with the stated results to be achieved. In other words, the results are not in line with and do not lead into the purpose. A purpose should define what the project promises to achieve through its activities leading to outputs and then to results with all these forming a logical hierarchy. Hence, we have focused on evaluating the achievement of the results: 1. Gap filled in the Finnish medical education. 2. Raised awareness of Global Health: health professionals and Finnish society. 3. Entry-point into international positions and projects for mainly young health professionals. 4. International network of students, teachers, and scientists interested in GH. 5. Strengthened institutional capacity of partners in the five countries to conduct training and research in GH.

Result 1. Gaps have been filled in the Finnish medical education - but is not cost - efficient. The gap is evident This Result appears in the project documentation especially in its early years as a key concern and the main reason to have the project. In 1999, when the idea of the course was born, the small group of “enthusiasts” realised that the graduating Finnish medical doctors lack understanding of global picture of illness and health, and of relationships of health with wider issues in a society and with globalisation. This is the case even today as graduates have very little, if any, knowledge on how ill health and diseases are linked with an individual´s, a family´s, a nation´s or the globe´s poverty, socio-economic status, occupational conditions and rights, climate change, environmental degradation, pollution, gender equality, war, fragility of a state, movements of people, economic and political

or international players, and many more. This is unfortunate as in many other countries global health or similar topics have been included in the education of health professionals.20 Contents are relevant The analyses of the contents of the training showed that the project has been dynamic in changing the contents of the GH courses in Finland as more has been learned in the globe on globalisation and health and new issues have come up. The development has been from emphasis on infectious diseases to international health and lately more towards examining the relationship between health and globalisation and towards multi-dimensional and multi-sectorial approach. The project has offered to 240 Finnish medical graduates or medical doctors (including a few dentists and veterinarians) an opportunity to learn about global health and to experience reality in a developing country and its health care. The courses have narrowed the gap in education since 2001 until today. The gap cannot be closed this way forever However, the gap in the undergraduate training is much wider and impossible for any later training to close as can be shown by some counting. The number of 600 new medical students every year (from 2012, 750) adds up to 7 800 in the period of 2001-2013. Hence, only 0.5% of students (or even less as not all were students) had the opportunity to learn on GH, while it should be provided to everyone. Also, as organised only for a small proportion of medical (and some dental and veterinary) students the costs are relatively high per student compared to having ordinary undergraduate training. The revised model should be adopted Yet, the real value of the project trainings is not so much in the numbers, but in the project keeping the “candle lit” during the universities being in sleep about what is happening in the globe related to health. Also, the training courses have been important to the large number of teachers from different parts of Finland, whose main employment has mostly not been on global health. (An indication of this is that many have not accepted any lecture fees offered.) The project has developed throughout the years a functioning “model” course that could be applied with some modifications to the under-graduate education in all five Finnish medical faculties. While the established institutions have not done it, a few visionary professionals with hard and mostly volunteer work, have. However, the courses for Finns cannot be funded permanently from the development co-operation funds. It is worth noticing that in the

survey among the Finnish participants of any GH course, all had the opinion that global health should be included in the undergraduate training. However, 50% preferred elective courses while the other half was for compulsory courses. Nevertheless, students could form a pressure group to promote efforts to integrate global health in medical undergraduate education, as has been the case in many countries.

Result 2. Awareness of global health has been raised - but needs to be sustained Difficult to measure As the results is phrased so vaguely and in general terms, it is difficult to assess to what extent it has been achieved. In the following chapter, we discuss the need for more specific planning of training so that competencies are pre-defined and their attainment measured. Changes in mind-sets It is evident in the feedback gathered by the project of the courses organised in Finland and in the four partner countries that the participants have appreciated the course greatly. All nationalities have reported that in the theoretical part the main aspect learned was on the relationships of health and illness with a wider society, for the Finns also in Finland. (It is interesting this is not learnt in the undergraduate training in a country, Finland, where a main health policy priority is to reduce inequalities in health between socioeconomic groups.) In the survey among the Finnish trainees throughout the years, 56% of the respondents reported that the course had changed their thinking on global health much or extremely much (31% reported “somewhat”). In the feedback collected after each course and in our interviews it was also evident that the Finns had perceived the visit to a developing country very important. In the survey among the Finns, as many as 80% of the respondents considered the visit to a low or middle income country as one of the three options that had most benefitted them, as such a visit was the first for most of the trainees. The participants from the partner countries have reported the visits part in Finland having given hope that things can be changed and done differently than in their own country. Raise awareness beyond the courses though eLearning Five eLearning modules or courses, of one to two hours of duration, were developed and made publicly available. They were intended to raise awareness on global health among health professionals in general. Developing them is an achievement as such (although

the teams in partner countries perhaps did not get competencies in design of eLearning). Unfortunately, due to technical reasons, their use cannot be monitored on the website of Duodecim. “Introduction to Global Health” course has been opened about 600 times since its publication three years ago. Trainees did not use the eLearning modules The survey results were not so encouraging as only 15 of the Finnish respondents reported that they had taken an eLearning course. Of them, 10 persons had taken the Introduction course while hardly anyone had studied the other courses. (There was no information available on use of the eLearning courses in general due to the website system). IT IS RECOMMENDED that in the Phasing Out period, a better use of the eLearning courses could be made in the partner institutions and elsewhere. The courses could also be designed to be interactive and international with group discussions and joint assignments done via the Internet. The Result 1 on raising awareness also aims to inform on global health in the Finnish society, specified in the project documentation as decision makers, politicians and lay people. The activities to achieve this audience were to organise two Summits: a half-day seminars with presentations from two top-level experts in Europe or the world (the project management has been able to get them), and some key figures in Finland such as cabinet ministers, arranged in nice premises in Helsinki. Here, also, the feedback gathered in the project is positive. Although politicians and decision-makers were not reached as audience, the participants felt the meeting was worthwhile, and external relations important to the project were strengthened. The project has a good idea to advocate global health in the Finnish society. We recommend that it could be worked on further: we suggest organising and funding it in the future in partnership with other organisations, to do much stronger marketing and base the programme on clear pre-determined objectives. The two two-day seminars organised in Helsinki in 2012 and in Tampere in 2014 have served mainly the project teams from different countries and some alumni and have been important in up-dating their knowledge on GH. Some concerns… The survey and interviews with participants on GH courses showed that among the majority of them the awareness on global health does not seem to last and it does not result in action. Among the Finnish respondents in the survey, only

15% followed global health information systematically, for example following a website or a journal or had an RSS feed for new contents or editions. One-third read GH topics if seen while surfing in Internet. It is recommended that in the future it is ensured that trainees know the most important sources of GH information in order to keep up-to-date on issues. About 40% had participated in further training related to global health after the course. Of those who had, almost all participated in a rather similar national evening courses organised by the medical faculties and other organisations. The results on networking indicate that the Finnish trainees have not been active in networking with other participants. The motivation and interest that the courses have evidently been able to create are not easy to keep up. We recommend more attention is paid to provide giving more practical information on opportunities to participate in volunteer work and NGO activities. Also, the alumni could be engaged more in functions of the global health units established in the partner universities, as has been positively experienced in Chile and the Philippines. The next chapter “Issues” includes recommendations for better effectiveness related to the planning methods, contents and structure of the training, if and when integrated in undergraduate training in Finland or other countries or continued in another way.

Result 3. Entry-point into international positions - for some More opportunities exist to study It has been assumed in the project that the GH course would be a merit when applying for international work. There are, however, increasingly also other opportunities to study global health through the Internet achieving a certificate and also take such courses abroad as part of student exchange programmes. More chances exist to visit developing countries In Finland, there is no system of junior experts to serve in development co-operation projects as in many other countries (as there are no bilateral health projects by Finland). The majority of the Finnish alumni have viewed the two weeks visits part as the most valuable in the course. As it was the first time in a low or middle income country to the majority, it gave basic knowledge and a culture shock. Tailor-made visiting programmes of the project have evidently offered more than any ordinary visit. However, concerning Finnish students there are today

several other opportunities to gain the first experience of a developing country, for example through student exchange and volunteer programmes. Furthermore, global health is much wider than issues related to developing countries - it is about the globe and globalisation. Interest on international work rose Another assumption in the project has been that through the course the participants get motivated to seek international work. The project must be congratulated for undertaking a careful selection of participants; many may have had the motivation before the course. In our survey among the Finnish alumni, 40% reported that they started to seek for international jobs abroad or in Finland or started to seek GH related work or modify their work so that GH was part of it. This is a good outcome for the project. Only 13% has had any GH related work after the course, which may indicate the difficulty to get such a job. Result 4. International network of students, teachers, and scientists - has much more potential

The fourth result is about networks. The project has managed to identify motivated and serious institutional and individual partners. It has not been easy as can be seen on the first partners dropping out from the project for various reasons. Using individual contacts and setting clear criteria for a potential partner institution and teams have been crucial and successful. Also, having partners from different continents has been a great idea enhancing “global” aspects in the project. Networking among the partner institutions seems to us to have focused on organising the courses, not so much on global health topics or information itself. Meetings among the teams could have been targeted for the core teams with real dialogue. There seems to be no formal or informal network structure as such among the partners. Also, networking should take place more widely than among the partners, and the project could assists in this. The network among the participants in the courses has naturally taken place during the courses and exchanging emails immediately after it. However, there is no networking structure as indicated in our survey: about half of the email addresses of the Finns did not work and the project has not kept them updated. Facebook groups have been started up with alumni after several courses and they include some discussion on GH. On the other hand, it cannot be expected that alumni keep in close touch for many years. We recommend that the project would link the

trainees into wider networks internationally. We make some recommendations later in the report concerning networking among the teams, teachers and students.

Result 5. Institutional capacity of partners - needs much more strengthening Waking up One of the five results to be achieved, the most important one from the perspective of development co-operation, is to strengthen the capacities of the partner universities to carry out training and research in global health. In the interviews, team members in Chile, Nepal and the Philippines reported that the need for global health training at their universities was recognised as a result of 1-2 key persons of the faculty having attended a GH course in Finland. This did motivate the faculties to be involved in the co-operation: to send participants to Finland and to receive Finns for the visits part. Pools of experts In this way, the involved departments were able to develop at least a small pool of experts, closely or less closely attached with the department, to work on global health. This took place in Chile, where several alumni are involved in GH as faculty members. In the Philippines, justice and equality issues in health were already a major focus of the department. Due to participating in the GH course, more knowledge was gained on global health and integrated in the training and also the department started their own courses on GH and have run them since. In Nepal, a GH Unit was established. It has renovated the premises and named the key experts. The activities are yet to be developed as well as the human resources assigned by the Faculty. In the Philippines, Nepal and Tanzania, the faculties have organised the project GH courses to Finns and their nationals, and most recently from all partner countries. Of course, they did already have the competencies to arrange courses but the challenge was to identify local trainers and to somewhat modify the main programme coming from the Finnish course. This also assisted in engaging possible trainers for the courses of their own in the future. In order to enhance the capacities to carry out GH training courses, the project has organised only two two-day sessions for training of trainers during the years. In the recent one, in 2014, an important issue of modern learning methods was discussed and it gave a good start to emphasise more the defining of clearer and concrete objectives and competencies to be

achieved rather than merely raising awareness in GH courses. The latter part of the Result, on building capacities in research on GH, has no planned activities in the project documentation. It is an accomplishment that motivated partners are engaged in the project and that GH trainings have been organised by all of them although fully funded by the project. With the motivation at the early years of the project, Chile and the Philippines have started their own courses with their own funds besides the project courses. The key challenge is how to ensure that GH training/ learning will continue after the exit of the project and its funds. There is a clear need to improve sustainability. Improving sustainability requires capacity building based on jointly identified needs of the partners. The capacity building should not be one-way traffic but the competencies that each partner team already has should be utilised: for example, the Nepalese team and the Tanzanian new team know much about competencies based learning design, the team in the Philippines has a lot of experiences in advocacy and concrete action and the team in Chile is gaining experience in regional co-operation in organising GH training in Latin America. Also, the work on basing the learning on objectives and competencies needs to continue further. These aspects are discussed in the following chapter â&#x20AC;?Issuesâ&#x20AC;?.

6.3 Efficiency Use of funds The funds allocated by the MFA have been used to the purposes given. However, funds could have been used also for activities that support sustainability rather than taking care of more or less all of the costs of training courses. With proper management, the project has been able to make savings in the budget. Especially, the project director and other staff have revised processes in such a way that funds have been saved (e.g. early booking of flights as international travel is a considerable part of the budget). The team in Nepal did not consider they needed one-third of the funds allocated to the course organised in Nepal. The team in the Philippines saved also some money, which they then used to partly finance their own courses offered by the faculty. In Tanzania, in the course in 2012 the budget was exceeded. (Chile has not received any funding for organising courses as it is not eligible for development aid.) The large parts of expenditure are salaries of the

Finnish personnel in the project and travel and accommodation of the trainees between countries. From the environmental and sustainability point of view, the extensive travel of trainees to partner countries are questionable. It seems that there is no need to build capacities in financial management among the partners as planned for the Exit phase as their systems in the faculties seem to be fine. Donor harmonisation intends to reduce the extra work of reporting to the donors and rather utilise the existing systems, concerning also financial management and reporting. Project management The overall management of the project seems to work well. The staff and volunteers in Finland are very motivated and do perform their work efficiently. The engagement is really worth of noticing.

important in its development policy. Integrating crosscutting objectives into all development cooperation is required and it is one of the ways to effectively implement Finlandâ&#x20AC;&#x2122;s human rights-based approach to development. The MFA has an official guideline for the integration of cross-cutting objectives into development policy and cooperation issued in 2012.21 Global health as such is much about gender equality and reduction of inequalities of all kinds concerning health and health care and their determinants. The relation between climate sustainability and health is gaining importance as the world has been learning more on the multifaceted relationship.

Small challenges have been experienced at times on unclear division of responsibilities and tasks as reported by some members. These, however, have not caused any major problems. The Finnish consortium of partners is large. Only very few members participate in any other way than paying their part of the self-financing. Only very few attend the Steering Committee meetings and activities. This issue was raised in the evaluation report in 2006: Tom Sundell recommended the specific interest of the partners to be identified and some activities to be built on that basis. Activating the Finnish partners is a difficult challenge and could perhaps be more thought in the Exit phase.

Based on our observations we recommend that the project could in the Exit phase revise the management to be less led and managed by the team in Finland. The teams in the partner universities could be more involved in planning, monitoring, evaluating and revising the objectives and activities. This would improve the relevance and sustainability.

6.4 Cross-Cutting Objectives The cross-cutting objectives of Finlandâ&#x20AC;&#x2122;s development policy and cooperation are: gender equality, reduction of inequality of all kinds, and climate sustainability. They are international commitments, which the Government of Finland considers particularly

CHAPTER 7 ISSUES Evaluation of sustainability focuses on the likely continuation of achievements. The questions are: Will the benefits produced by the project be maintained after the termination of external support? What are the possible factors that enhance or inhibit sustainability, including ownership/commitment, economic/financial, institutional, technical, socio-cultural and environmental sustainability aspects? Has the phasing out of external support been planned, and will the plan ensure sustainability?

7.1 Sustainability As sustainability is a major concern in development co-operation and the major concern especially when the project is at its phasing out period, sustainability is discussed in detail in this chapter. From the very beginning every project should plan and implement systematic actions to gradually ensure that the results will sustain and/or activities will continue after the donor has pulled out. Mere capacity building of individuals seldom results in sustainability, but a variety of actions are needed to ensure commitment, integration into existing systems and functions as well as human and financial resources. To assess sustainability, we have taken that the sustainability in this project means, at the minimum: 1. High likelihood that training and capacity building in global health will continue in the five partner institutions, after the project pulls out and 2. Awareness on global health issues of those attending in the training will continue as a continuous interest in following GH matters in the Internet and journals and attending further training. As the partner institutions function in quite different realities, we discuss each partner and country separately. We also make here the partner-specific recommendations to enhance sustainability. Sustainability in the partner organisations

Partners in China, India, Nigeria and South Africa were involved in the first courses in Finland sending students and receiving Finns for the practical part of the courses. From the sustainability point of view, there would have been too many partners had they continued.

Chile Chile leads Latin American nations in rankings of human development, income per capita, and low perception of corruption and has since 2013 been classified as a developed country, not eligible to development aid. Hence, the partner institution in Chile has received less financial support in the project than the other partners. The Faculty of Medicine, the School of Public Health has benefited from the project especially in increasing understanding of global health issues after the closed era of the dictatorship in the country until 1990, as reported in the interviews we held with the Chilean team members. The Chilean team sees an indirect contribution of the project increasing interest in public health among medical students, and raising capacity in public health. A key person, presently the Dean of the Public Health School, attended the course in Finland in 2006. Since then, the School has organized the field visits part of

the GH Finnish course of Finnish participants and has systematically identified and sent Chilean students in the GH courses in Finland, funded by the project. The School started a Global Health Unit in 2010 with focus on research employing several alumni of the Finnish courses. In 2014, the School of Public Health is organizing an elective 80-hour course on global health in the undergraduate curricula of health professions being part of the duties of the School (with no external funding). The Unit also participates in the regional initiative of the Latin America Global Health Alliance with Brazil, Cuba, Costa Rica, Mexico, Peru and Colombia. A Diploma course of 5 months is being planned with eLearning and student fees. The impact of the project on Chilean partnering institution has been in initiating the interest in GH and basic capacity building of the faculty. Chile, as a developed nation, may not have been the best choice, in principle, for a development co-operation project to represent Latin America. However, it has proven to be an active partner.

Recommendations - Chile In our view, the Chilean partner, Global Health Unit, could benefit in the Phasing Out period on joint planning of the strategies and functions of the Unit and the identifying the support needed. (While funding of Chilean trainees to different countries to attend courses would not improve sustainability). The other partners would benefit in sharing of the Chilean experiences in developing their global health unit, building regional co-operation in global health, the eLearning courses being designed and information on the design and contents of the global health undergraduate courses in Chile.

the philippines The Philippines is a lower-middle-income country, eligible for development aid according to the list of OECD, and active member of G-24 and regional collaboration. The project was in its early years central in inspiring the key individuals from the partner university in Manila through attendance on GH courses in Finland. Since in 2002 the partner has organized the field visits part for Finns with funding from the project. They have also chosen and sent Filipino participants on the courses in Finland paid by the project thus building a pool of trainers. At the university, they have held several GH courses

on their own since 2004 linking it with community health service in villages. Other universities in the Philippines have picked up the model since. The project-funded GH courses with Finnish and Filipino participants were organized in 2009 and 2011. With the funds saved they could partly fund their own courses. The view on what is global health is broader in the Filipino institute than in the Finnish course emphasising positive and negative impacts of globalisation on health and healthcare of people, role of politics and proactive advocacy work with international and national NGOs.

recommendations - the philippines It is quite evident that the partner university will be able to organize GH courses for the local needs without project funds, and has already done so. It will most likely not have funds to invite participants from Chile, Nepal and Tanzania as has been possible in 2014 with project funds. The Phasing Out of the project should focus on strengthening the sustainability of GH training through assisting the partner in exploring and utilising opportunities for regional collaboration as the country is an active participant in Southeast Asian collaboration. The other partners could learn from sharing the wide and proactive approach on global health issues from the Filipino partner, the experiences on how to promote global health training to be included in the other universities in the country and how to integrate parts of global health in community activities.

Nepal Nepal is one of the six long-term partner countries along with Ethiopia, Kenya, Mozambique, Tanzania and Zambia on which Finlandâ&#x20AC;&#x2122;s development cooperation centres primarily. Taking its political instability and wide spread corruption, including Nepal in the project could have been a challenge, and a potential failure. Quite the opposite has taken place as the Nepalese partner university has rapidly shown serious commitment on GH training and has dealt with the project funds in an exemplary manner. The recently started Global Health Unit has 3-4 active individuals with their main tasks elsewhere. They have chosen and sent participants in the Finnish courses since 2010. They organized a project funded course in 2013 in Nepal and have paved way to integration of GH in the undergraduate curricula.

However, the human resource base is thin and vulnerable. The alumni of the courses in Finland have not yet been tied with the GH Unit as resource persons. There is enthusiasm and ideas on the role and functions of the global health unit including making efforts to integrate GH in curricula and opening these courses to regional, fee-based participation.

Recommendations - Nepal Being a rather new member in the project, the Nepalese partner university does not have yet sustainable results or activities in global health. In our view, it has the potential to build GH functions on its own but requires assistance in developing regional collaboration. The mode of assistance should be thoroughly discussed jointly and could include planning of strategies and activities and financing of the Unit, sharing experiences from Chile and the Philippines and marketing expertise and financial support from the project to employ staff for the first year of the Phasing Out, then with cost-sharing with the University and finally self-financed. The other partners could benefit from sharing the expertise of the Nepalese partners among others on competencies based learning design.

Tanzania Tanzania is also one of the six long-term partner countries and focus of the Finlandâ&#x20AC;&#x2122;s development cooperation. The partner institution has been the School of Dentistry at the University of Dar es Salaam. Contact with it was established through previous development collaboration with Finland. It has since 2004 organized field visits as part of the Finnish courses to the Finns and recruited Tanzanian participants to the courses in Finland. Sokoine University of Agriculture has been involved as well. Since the death of the key contact person in the Dental School, there have been some problems on continuation. Since early 2014, the new partner is School of Public Health, from the same university as the previous partner came in with considerable motivation. The course in Tanzania in 2014 was well organised.

Recommendations - Tanzania The Phasing Out period would need to be carefully and jointly planned to maximise the results and sustainability. The partner is new and its strengths, competencies and capacities are to be identified.

The other partners would benefit from sharing challenges in global health in a poor country and the experiences of the partner to introduce competencies based learning at the university and the international cooperation and networking.

Finland Finland, to which the project has focused the most and where the most courses have been held since 2001, is perhaps the most difficult of the five partners as to the sustainability of training in Global Health. The faculties are far behind the most Western schools in including globalisation and health in curricula, even as an introductory or elective course. The excuses given are that medical, dental and veterinary undergraduate curricula are packed with continuous pressures from several established clinical areas to expand their shares, that there are not enough experts to do the training or that funds are very tight especially during the following couple of years do the financing reforms in the Finnish universities. Defining the role of medical doctors in particular in the present world with everincreasing effects of and needs to act also proactively on globalization seems to be lost. A major reform in curricula is needed, as preliminary discussions at the University of Helsinki also indicate. In the evaluation in 2005, Tom Sundell strongly recommended for the project to lobby for the inclusion of Global Health teaching in the medical curricula in Finland. He also suggested that such a short basic course could then be used as a selection criterion for the applicants for the project GH course.

Recommendations - Finland During the Phasing Out period of the next three years, the project should strengthen its efforts to advocate for introductory course in GH to be included in undergraduate curricula in medicine, dentistry and veterinary studies in all the faculties. The project should initiate a movement joining forces with powerful organizations and individual opinion leaders and decisions makers, experts and students and spreading information on international developments in GH and in its training. A careful joint planning of such a model course taking into account the articulated and non-articulated obstacles (or excuses) is possible in the Phasing Out provided that the current plans are revised and funds re-focused. Planning of sustainability To consider sustainability, it starts already in the planning of a project and goes through-out the project.

The culmination is a Phasing Out period, which has its own purpose, results and activities and when the project funding and activities gradually diminish like tail. As the project is approaching its end, there is a time to critically look at the sustainability. The plans made for the Phasing Out period are good as such but do not enhance sustainability sufficiently. In the following chapter we give overall recommendations for the Phasing Out period and its planning.

7.2 Effectiveness and learning objectives In training, it is often the case that courses and events are organised, but it is not clear why, to whom and to learn what and how. One reason for general weakness may be lack of accountability: organisers may not need to report to someone if learning did take place in some issues and what were the reasons for it and what needs to be done to improve learning outcomes. The consequences of improper planning of training and learning are waste of money and time of organisers and learners. Knowledge and skills to be learned In the project, the major activity has been conducting courses in global health. Contents have been following the development in global health and new themes have been prioritised as needed. Feedback has been collected after each session, day and course. The problem is that planning of the contents and the feedback requested has not related to learning and improving competencies. In the project, better results and effectiveness could be achieved through applying a systematic approach to learning such as problem-based learning, resultsbased learning and competencies-based learning. An independent commission of 20 academic leaders from around the world recommended comprehensive reform in the training of healthcare professionals, in a major report published in The Lancet on 4 December 2010. The report called for competencybased curricula, creative use of information technology, transformative learning, and interprofessional teamwork, as well as a systems approach to institutional reforms. In the following four years, a follow-up dissemination and advocacy initiative was coordinated by the Commission’s co-chairs — Dean Julio Frenk of the Harvard School of Public Health and Dr. Lincoln Chen, president of the China Medical Board. The systematic approaches cover learning objectives and results, identification of appropriate learning methods to each session, briefing of trainers/ facilitators and leaners, assessment of achieving the

learning objectives and feeding experiences to the planning. Previous evaluation recommended systematic planning Already in the previous evaluation of the project published in 2006, Tom Sundell found it difficult to assess impacts or achievements of the project due mainly to the lack of goals (i.e. higher level objectives) and specific learning objectives. Hence, Sundell had to limit the evaluation of achievements on studying whether personal and institutional networking was taking place and whether knowledge and awareness had increased, basing his findings on personal views of course participants, partners and stakeholders. Consequently, one of the recommendations in the previous evaluation was that a new needs assessment should be made including all stakeholders to better meet their needs. Then, goals should be set including also longer-term goals and their measurement. Learning objectives were suggested to be defined more specifically: “Developing a means of measuring the learning and professional development of the participants during the course (and the field assignment) would give more concrete tools for evaluating the achievements of the course”. This seems not to have been taken into consideration in the project and the learning objectives and measuring their achievement have been vague and insufficient. As the Nepal partners have studied extensively systematic learning design and the new Tanzanian partner is implementing such system with the assistance of the Harvard School of Public Health, there is a great opportunity for the project to develop learning further during the Phasing Out period so that the models as products of the project will be most modern and based on evidence to be implemented in the partner institutions and disseminated for wider use.

CHAPTER 8 RECOMMENDATIONS On the basis of the strengths of the project and the evaluation findings and conclusions, we make the following main recommendations. Specific recommendations have been given in the previous chapters.

Recommendation 1 The Ministry for Foreign Affairs of Finland is srongly recommended to grant funds for the Phasing Out period 2015-2017 of the project, provided that the preconditions defined below are met. The MFA has funded the project for 13 years and with 2 million euros. The duration and the amount are exceptional and indicate that the MFA has viewed the project as important. The project has submitted in May 2014 an application for funding of the “Exit Phase” for the years 2015-2017, a total of one million euro. The project is about to become a bright star on the sky of the NGO development co-operation of Finland. The precondition for such a status is that the Exit period of a project will considerably improve achievement of the results and sustainability expected from a project of this magnitude. The plan submitted to MFA for the period needs some changes: there should be a clear exit strategy and activities that support better achievement of the results and sustainability. In other words, there is a risk that at the end of the project and its funding, the training and advocacy activities in Finland and the four other countries will collapse, networks do not function and experts and trainees lose their interest in global health. It is recommended that the MFA requests for changes to the project Exit period plan. As activities of an Exit period often need to be revised during the period, it is recommended that after the decision on the grant, the MFA releases funds on the basis of achievement of pre-set milestones and reporting (see the Table 2 on page 33).

Recommendation 2 The project is recommended to make changes in the plan for the Exit period to considerably enhance sustainability and achieve its results even better. Expertise in the five countries has been developed through participation in the Global Health courses as students or teachers/facilitators and through each partner organising project GH courses, but with covering the costs and with little focus on how partners could continue after the project. The Exit period offers a great opportunity to make the final difference on results and sustainability. It is recommended that these main aspects are used in making some changes in the Exit plan and in making specific plans at different points of time during the three years period for payments of the funds (see also the Table 2 on page 33). • Involvement of the partners in Chile, Nepal, the Philippines and Tanzania intensively in the planning process. Changes in the present management structure are needed to ensure continuation of dialogue and joint decision-making. • Designing jointly the objectives and especially the results and activities with milestones to be specific to the Exit period that by definition requires a specific approach. • Designing jointly sustainable and realistic strategic and action plans on global health functions for each partner institution (e.g. the GH unit in Nepal) that go beyond organising a GH course for undergraduates.

The functions and activities could be designed to achieve such results as: »» Undergraduate health professionals attain predefined competencies in GH and on themes most relevant to each country, and skills to keep updated on GH and to network globally, regionally and locally. »» Events and activities on global health are organised and attract internal (university) and external audiences. »» Global health courses alumni are engaged in the GH functions and activities of the institution. »» Active advocacy and cooperation takes place with decision-makers, politicians and NGOs on global health issues. »» The faculty is aware of and actively participates in and cooperates with the main global and regional networks and associations/consortia on global health education, research and developments and international and regional organisations and agencies. • Designing jointly a financing plan, including human resources, for the planned functions and activities as above. The plan should have strategies and activities to gradual self-financing and sustainability.

and financing as needed in close cooperation with the MFA.

Recommendation 3 The Ministry for Foreign Affairs in Finland to consider revising the NGO funding application forms (This comment does not relate to only this evaluation and project.) It is understood that forms are not easy to design for such a large variation of NGOs. There are two parts in the forms that are very important when assessing a plan or evaluating a project. These parts are even more important in running a project successfully. 1. Objectives and their hierarchy and logic with each other would need to be more visible in the form. In the part on Results, it would be important to emphasise that they are not outputs and activities and that they are one of the most important part in the plan. 2. Sustainability requires attention and planning from the very beginning of the project. In the form, it seems to be easy to bypass and omit.

»» Seeking for partnerships, regional co-operation, attracting paying regional students, engaging volunteers/alumni, using eLearning and videoconferencing, etc. Utilising the know-how and experience in the partner institutions. Gradual selffinancing (university) of human resources. • Assessing jointly competencies and human and financial capacities to carry out the above plans. Designing a plan for the Phasing Out period on how to support their strengthening. • Designing jointly a financial plan with milestones for the Phasing Out period to enhance sustainability on the basis of the above rather than funding participation in GH courses and their organisation. Re-planning of human resources in the project is essential, as the recommended Phasing Out plan requires human resources different from the ones in the present application. • In Finland, planning the sustainability, i.e. how to integrate global health in medical undergraduate education and how to involve the large Finnish consortium of partners in the efforts. • In Finland, seeking partnerships to organise, fund and market the planned events (Summit and seminar in F.) on global health issues to politicians and decision- makers. • Joint close monitoring of the progress with indicators and milestones during the Phasing Out period and making changes in strategies, activities

Table 2. Suggested schedule for submitting plans and reports for the funding of the Phasing Out period in 2015-2017 of the project.

Phase out 2015 - 2017 Point in time Type of plan

Contents

Budget plan

MFA to

Milestone

Fall 2014, deadline Revised application to be agreed with for funding. MFA Application of LFA, results-based or another justified tool.

Overall plan with results and main activities.

Overall tentative budget.

Approval of the overall application. Approval of the plan for the 1st year and release the funds for the first year (about 25% of the overall budget).

Application submitted in time, and according to the Evaluation recommendations.

By the end of the 1st year, 2015

Report on the 1st year. Detailed plan for the 2nd year.

Detailed plan for Detailed budget the 2nd year (and for the 2nd year tentatively for 3rd year) on results and activities and milestones based on the overall plan and the work done in the 1st year (see the milestones)

Approval of the plan for the 2nd year and release the funds for the 2nd year (about 50% of the overall budget).

Strategic, action and financing plans on global health functions for each partner institution designed and agreed. Joint assessment of competencies and human and financial capacities in each institution made.

By the end of the 2nd year, 2016

Report on the 2nd year. Detailed plan for the 3rd year.

Detailed plan for the Detailed budget 3rd year on results for the 3rd year and activities and milestones based on the overall plan and the work done in the 1st and 2nd years.

Approval of the plan for the 3rd year and release the funds for the 3rd year (about 25% of the overall budget).

Progress of the 2nd year plan analysed and final activities planned on that basis.

By the end of the third year, 2017

Ordinary final reports.

Detailed plan for the Detailed budget 1st year on results for the 1st year and activities and milestones based on the overall plan.

CHAPTER 9 LESSONS LEARNED In the evaluation, we identified several lessons that could be useful to other projects or training/education activities. These are, however, quite basic and generic and are aspects that have been emphasised in development co-operation and in its manuals as well as recently in education design. Objectives are important The importance of having an intervention logic or chain of change or a logical framework - whatever tool one may prefer- could not be overemphasised. The hierarchy of objectives from development objective and project purpose to results and activities -and vice versa- is essential. It should have also an internal logic and rationale. If a project had such a simple hierarchy, it could be much more effective and could produce better results. The Ministry for Foreign Affairs in Finland emphasises effectiveness and could enhance it through having a longer funding period (here only 3 years at a time) and requiring as a minimum a clear hierarchy of objectives (that actually can be expressed in a half a page). A project having a weak basic hierarchy and logic of objectives can lead also for an evaluation to assess results and activities which may not be relevant to start with unless the hierarchy and logic are also evaluated. Sustainability must be planned Development co-operation aims for sustainability. Activities for it need to be planned already at the initial phase and revised if needed during the course of the project. Here again, the MFA could be more clear in its applications forms and more strict in its requirements. It should be more emphasised in training on development co-operation.

Results need to be specific and measurable

Achievement of results that are vague and planned in general terms cannot be monitored during the project and hence no correcting action may not be taken. Also for evaluation such results are essential. Indicators of results, if missing unfortunately, can be developed to

some extent during the evaluation. Learning events should result in learning Today more attention than previously is being given to design of learning or training. Efforts need to lead in specific learning outcomes; otherwise they may be waste of time and resources of learners and trainers or facilitators. Several tools have been developed to define outcomes in terms of knowledge and skills or competencies. If used, they assist also in measuring the extent learning actually takes place. They are also useful in planning relevant learning methods to be used. Efficiency depends on implementers The evaluated project is a good example on saving funds within a budget line (to do what is planned with as low costs as possible) due to the mind-set of those working in the project. This holds also to partners in low and middle income countries especially when incentives are used to do so as in this project (saved funds could be used for the partner institutions´ own courses). Another side of effectiveness is to plan the “right things” i.e. the activities in such a way that most costeffective options are chosen. International co-operation today does not necessarily require people flying but technology can be utilised as least partly. Relevance is part of project logic If a simple hierarchy of objectives is properly designed, a project is more likely to be relevant through serious consideration and design of a development objective and its clear and strong logic with the lower level objectives. At least, it is then possible to assess relevance. Enthusiasm and commitment This project shows that a lot of activities can be implemented when there is enthusiasm. Commitment has lasted for over 14 years. Volunteer work is in many ways rewarding and needed especially in NGO projects, among all the partners. Evaluation of NGO projects Applying internationally accepted guidelines and conducting a “full” evaluation of an NGO project can reveal positive and negative aspects to be considered by the project itself and by the funding agency. To what extent evaluation results will be used depends mostly on how they are disseminated to those who could benefit from them. NGOs themselves including partner NGOs could use such findings in training in development co-operation if promoted by umbrella organisations of NGOs and by the MFA. Finally, it should be mentioned that an evaluation process is a huge learning opportunity for evaluators improving their competencies in project cycle management.

ANNEX A THE EVALUATION TEAM Anneli Milén

Anni Leppänen

Dr Anneli Milén is a Leading expert in health systems in the National Institute for Welfare and Health, THL in Finland. She has over 20 years of experience in development co-operation. She has worked in bilateral projects, by NGOs as well as in EU-funded projects in various capacities as planner, designer, leader, manager and evaluator. She has served in over 20 countries for long and short-term. She has designed the large multilateral Northern Dimension Programme on Health and Well-being, and worked in WHO in Geneva. Dr Milén has also an extensive background as a researcher being a docent (adjunct professor) on Community Medicine, a PhD and MSc from the University of London. Her career includes planning and conducting a large number of undergraduate, postgraduate and continuous educations for health professionals. She has a wide knowledge on development cooperation policies, modes and funding in general and in health in particular.

Anni Leppänen has a double BA from School of Oriental and African Studies, University of London. She is currently working on her Masters studies at Aalto University. Anni’s expertise lies in development studies and development collaboration. She has a diverse work history in project management, and in bilateral and multilateral collaboration in the fields of development, business and science. Her research has included graduate employment in China, and innovation systems for developing countries. Anni worked in the evaluation as a Junior Expert. She was assigned a total of 38 working days for the evaluation, including the 7-day mission in Nepal. Anni also designed the layout and figures for this report.

Dr Milén´s role in the evaluation was as the leading expert. She was assigned a total of 26 working days for the evaluation including the 10-day mission in Nepal.

ANNEX B Detailed description of the project The objectives and the activities of the project are presented in the figures on the pages 17 and 18. Here we describe the project in detail.

Activities Primary focus on Finns Four young Finnish medical doctors in 1999 decided to design a course to fill the gap in the Finnish medical education, namely to learn at least some basics on health in other parts of the world. They felt also that a recognised course would help young doctors to enter working abroad in international organisations or projects in developing countries. Adding developing countries â&#x20AC;&#x153;International health should not be studied among only Finnsâ&#x20AC;?, the four initiators concluded. Also, having some participants from developing countries was seen as an opportunity to get funding from the Finnish development co-operation funds. Through the contacts of Finnish colleagues they developed links with universities in China, South Africa, Nigeria, Chile, the Philippines and India (an NGO). The participants of all the courses have been 50% Finns and 50% from abroad since the beginning of the courses until the past year. The team of initiators with others joining designed the training course and established a kind of an exchange programme with the partner universities abroad. Funding was applied and received from the Finnish Development Co-operation of the MFA.

First courses in Finland, practical parts abroad The first course was held in 2001, followed by courses in 2002, 2004 and 2006. Of the 30-35 participants, half were selected by partner institutes abroad. All their costs were covered. Finnish final year medical students were selected and most of their expenses were paid. A four-week period of lectures, group work and evening discussions were organised as an internat. Numerous motivated Finnish trainers gave their time. After the theoretical part, the participants from other countries remained in Finland for one more month conducting a specific assignment, for example comparing maternal health care in Finland and their country, together with an assigned tutor. Costs were covered. Since 2006, practical part has been changed: the assignments were omitted and the programme consisted of visits to institutes, health clinics and hospitals. Finnish participants were sent for a month to the partner countries (China, South Africa, Nigeria, Chile, the Philippines and India), a few to each country. Practical arrangement and tutoring the assignments were the responsibility of the partners, with a modest pay. The Finns paid the flights and a small course fee while the project covered the rest of the costs. Partnerships stabilising After the first years, the partner institutes in China, Nigeria, South Africa and India dropped out for various reasons while closer partnerships were developed with Chile, the Philippines and Tanzania and later with Nepal. Courses in Finland continued in 2007, 2009, 2010, 2011 and 2013, but shortened from 8 weeks to 4-5 weeks. The practical part was for the international participants in Finland and for the Finns in the partner countries by omitting the written assignments and including visits to institutes, health centres and hospitals. Independent courses in two partner countries During the early years, the partner universities in Chile and the Philippines and later also in Nepal, began to systematically send staff members or potential resource persons to the courses in Finland. Their roles varied from teachers/trainers to participants. With a pool motivated young experts, the Philippines and Chile developed very early global health training functions of their own to their nationals, with no funding from the Finnish GH project. Project courses also abroad A new challenge was taken: to have full courses organised abroad by the partner contacts and funded by the project. Half of the participants were to be Finns and the other half of nationals. All expenses

were covered by the project, including payments to several local organisers. Finnish participants paid their flights and a small course fee. In Tanzania, a committed professor led the organising of the courses in 2008 and 2010. After his death, it was difficult to identify a replacement, an indication of the necessity to have several persons closely involved. However, a course was organised in 2012. The partners in the Philippines with several experts organised courses in 2009 and 2011. These were in addition to their independent courses, with some of these funded from the money saved in the official project courses. Chile was not eligible for receiving development funding for organising a course in Chile and hence did not organise project- funded courses. Cross-participation in courses abroad and in Finland The next move was to put all nationals together also in a course abroad. Nepal organised a course in 2013 having participants from Nepal (12) and Finland (12) and three from each of the other partner countries: Chile, the Philippines and Tanzania. This was the first GH course that the Nepalese partners organised. In the second half of 2014, a similar mix will take place in the courses in the Philippines, and in Tanzania with a new partner Department and contact persons. All partner universities have organised “project” courses of 4 weeks in their institute, with local trainers and visit programmes, have made the arrangements (except flights and insurances) as well as reported on the financial accounts. The partners have also selected participants from their countries to attend the courses abroad. A Finnish trainer has attended a part of each course. eLearning As there were some funds saved, the Finnish core team decided to expand training to be openly available online. Finnish experts developed two modules in 2011 and 2012. A Nepalese expert and the team in Philippines and Chile wrote the material to one module each. A brief training course with experts from partner countries was organised in Chile for writing the contents. The Finnish experts carried out the design, construction and IT to put into the website. Each module takes 3-5 hours to study and the topics are: • Introduction to global health by Finland, 2011 • Literature search skill by Finland, 2012 • Migration of Human Health Resources by the Philippines, 2013 • Transition in Health by Nepal, 2013 • Introduction to Social Security and Health Systems by Chile, 2014

Summits and update seminars In 2012 and 2014, the Finnish core team of the project organised a half-day summit in Finland to reach decision-makers and lay people in Finland. This was followed by two-day seminars with presentations on current GH topics.

Financial and human resources The project has been funded mainly by the MFA Development Co-operation from the funds dedicated for NGO co-operation. The total amount during 20012014 is about two million euros. In addition to this sum, the self-financing part, as required by MFA, has been 15% of the total budget, of which 7% in kind (volunteer work). In the past three years, the total funding was close to one million euro (Table 3, next page). The project has had a core team in Finland with 0.70 person years paid from the project funds. About 0.2 person years including the Project Director have been allocated by the leading partner Duodecim as part of the self-financing.

Finnish partners The official responsibility of the project has changed during the years. In 1999-2002 it was with the Finnish Medical Association, in 2003-2005 with the Finnish Society for International Health (David Livingstone Society) and since 2006 the Finnish Medical Society Duodecim. The project has had several partners: Finnish NGO’s, governmental institutions and universities. Some changes have taken place during the years. The following parties belong to the Finnish GH consortium and some of them provide voluntary work. All of them except medical faculties contribute funds for self-financing part of the project (7%), the main payers being the first two members. Project partners: • The Finnish Medical Society Duodecim • Finnish Medical Association • Finnish Dental Association • Finnish Dental Society Apollonia and Odontologiska Samfundet i Finland • The Finnish Veterinary Association • Medical Faculties at Universities of Helsinki, Kuopio, Oulu, Tampere and Turku • Dental Faculties, Uni. of Helsinki and Turku • Veterinary Faculty at Helsinki University • Finnish Medical Students’ International Committee • Physicians for Social Responsibility • The Finnish Society for International Health

Table 3. Funds in the project in 2012-2014 Year

Development cooperation funds MFA

Self-financing required (in cash and in kind)

Self-financing required (in cash)

TOTAL

2012 2013 2014

120,000 300,000 380,000

24,000 59,000 59,000

(12,400) (20,800) (20,800)

144,000 359,000 439,000

2012 -2014 total

800,000 85% of total

142,000 14% of total

(63,000) (7% of total)

932,000

• Finnish Family Federation, National Institute for Health and Welfare • International partners China (University of Beijing), India (Institute for Indian Mother and Child), Nigeria (University of Ibadan) and South Africa (University of Witwatersrand, Johannesburg) were involved in the project during the first years. Co-operation with organisations in developing countries has gradually focused on the following four universities: • Chile: Universidad de Chile, Facultad de Medicina, Escuela de Medicina and Escuela de Salud Pública. • Nepal, Tribhuvan University, Institute of Medicine • The Philippines: University of the Philippines, College of Medicine / Community Medicine • Tanzania: until 2012 School of Dentistry, since 2014 School of Public Health, both at the Muhimbili University of Health and Allied Sciences, in cooperation with Sokoine University of Agriculture. In each partner organisation, at least one senior expert has been involved. The role of the partner organisations has been to: • select participants from their country to the GH courses in Finland • select 1-2 experts as teacher/facilitator for the courses in Finland and in the other partner countries • select 2-3 experts or managers in the institution to participate in the Summit and update seminar in Finland • organise visits in their countries for the Finnish participants of GH courses held in Finland (to 3-4 persons) • organise a GH course based on the Finnish course, in their country including the theoretical and practical part. Report on the use of funds received.

Management of the project The project has had an Executive Board, which has been functioning as a team meeting. It has consisted

of the team members at the Tampere University, the Project Director from Duodecim participating mainly via video conferencing and a medical student representative and representatives of dentists and veterinarians. The meetings deal with daily management issues such as progress on tasks at hand, division of work and feedback on the organized courses. The Steering Committee has included representatives of the numerous Finnish partner organisations. Meetings have been held twice a year with rather low participation rates of the other members besides University of Tampere and Duodecim. The agenda has covered annual plans, decisions on applications to the MFA or changes in the previous plans. Dr Juha Pekka Turunen in Duodecim in Helsinki as the Chairperson of the Steering Committee and has carried the overall responsibility of the project to the MFA and the Finnish partner organisations, and has made the final decisions related to the use of funds. He has been involved throughout the lifetime of the project. Professor Per Ashorn, the previous chair of the Executive Board and Course Director has been responsible on the quality of the GH training courses, the summits and the seminars. The project has two education advisors. Dr Anna Pulakka with 20% of input has been responsible mainly for tasks related to curriculum and teaching, and selection of students. The other advisor Dr Ulla Ashorn (20%) has been responsible for communication with the contact persons in Chile, Nepal, the Philippines and Tanzania. Dr KirsiMaarit Lehto (50%), the coordinator has taken care of the practical tasks such as travel arrangements and insurances (for Finns and all others, regardless of where the course had been held), accommodation and financial accounting of the project. She has also made arrangements for the two update seminars and two partner meetings in Tampere. Dr Ulla Harjunmaa, the previous education advisor has functioned recently as the Chairperson of the Executive Board. She has also organised the two-day update seminars in Tampere.

Annex C Methods in the evaluation Criteria We have applied the internationally accepted criteria for evaluation of development co-operation. Six main criteria have been used.

Relevance Is the project relevant to the development of the five partner countries, i.e. does it focus on main problems and policy priorities in these countries? Is the use of resources for this project justified on the perspective of the development needs of the country and compared to the use of the resources on something else?

Impact To what extent has the project made progress towards contributing to achievement of the (assumed) overall/ development objective of the project? Is it likely that the training in GH will result in changes in the development of the country, sooner or later? Does the project result in something worthwhile for the countries involved?

Effectiveness Is the immediate objective of the project (or purpose, the reason why it exists) achieved? Are the predefined results of the activities with the inputs (resources) being achieved?

Efficiency Can the costs/expenditures of the project be justified by the achievements? Are the results of training courses and of the other activities worth of the funds and human resources used?

Sustainability What is the likelihood that the effects made and the results achieved will remain and activities continue after the project has ended? What has been done to enhance the likelihood?

Cross-cutting issues How have the cross-cutting issues defined for the Finnish development co-operation been taken into account in all functions of the project? As stated by MFA22, the cross-cutting objectives, gender equality, reduction of inequalities of all kinds, and climate sustainability play a very significant role in poverty reduction and in the promotion of sustainable development and the human rights based approach (to development). They are to be promoted in all development co-operation regardless of the instrument used.

Evaluation questions and indicators For each criterion, we defined specific questions to be studied. These were followed by indicators as well as sources for information.

Sources of information We used several sources to collect data and information: 1. Project documents Funding applications and progress reports with annexes submitted to MFA during the over 14 years, every 2-3 years, since 1999 have been the main documentation in the project, and hence important sources for information to the evaluation. However, it is cumbersome to get information on what was planned to be done and to result in what due to the format of the application forms. We studied the programmes and descriptions of all the courses, theoretical and field visit parts, held in Finland and/or in partner countries as well as their lists of participants. The project has faithfully collected feedback from the course participants on both parts of the course, which provided us information on how the participants viewed the course. 2. Specific studies in the evaluation The international evaluation criteria are based on objectives of a project. In this project, assessing impact is not possible as the objective (Reduction of poverty

or Alleviation of ill health) is far too general. Also, the achievement of the project purpose in this case is also not possible to be evaluated, as it is too broad and general (Trained health professionals able to adapt, design and implement health care interventions and target country level health priorities). The project effectiveness can be evaluated by examining to what extent the project results are being achieved: 1. Raised awareness of Global Health: health professionals and Finnish society. 2. Entry-point into international positions and projects for health professionals. 3. International network of students, teachers, and scientists interested in GH. 4. Institutional capacity of partners in 4 countries to conduct training and research in GH. 3. Thematic interviews We held thematic, partly structured interviews with a large number of persons (Annex E), including also the team members of different partner organisations abroad while they visited Finland, and the team members in Finland. The themes included the evaluation criteria, namely relevance, impact, sustainability, etc. Each intended result was discussed in detail, in particular result number 3 on networks and 4 on institutional capacity. 4. Field mission in Nepal Visiting the Nepalese partner organisation and the persons involved in the project and in the Faculty management was of great value for the evaluation (see the visit programme in Annex E). Open discussions were held on challenges in the local environment and how to go about them, and on the needs and opportunities for improved sustainability of the results and outputs. We also studied and discussed the financial accounts of the project-funded course held in Nepal.

5. Survey Previously in 2009, the project conducted a survey to find out to what extent results had been achieved among the previous course participants in Finland (Finns and internationals). Due to the low (30%) response rate it was not useful for this evaluation. Hence, we decided to conduct a new survey. We included all 195 Finnish participants in the courses in Finland and abroad since 2001 up to 2013. The contact information gathered in the project was mostly not valid anymore. Hence, we made special efforts to get a reasonable response rate of 61%. The survey questions were related to the intended results of the project. For the details of the methods and results of the survey, please see Annex D. 6. Observations We had the chance to make observations as we participated in the Summit and the update seminar in Finland in June 2014. Anneli MilĂŠn has also functioned as a trainer in courses in Tampere, Finland and could use this experience in the evaluation. 7. Meetings and specific discussions Communication with the Finnish members in the project was numerous as were the requests for pieces of information. In the three meetings with the team we discussed fundamental issues of the evaluation. 8. Literature searches To find out information on the multifaceted context of the project a search for literature on global health, global health training and Finnish development cooperation was made on the Internet.

In Nepal, the partners had organised us to meet with some of the Nepalese participants of the most recent course in Finland and the one in Nepal (see Annex D). These interviews were very useful and we are grateful for the intervieweesâ&#x20AC;&#x2122; efforts to travel even from afar. The Ambassador of Finland to Nepal, Mr Asko Luukkainen gave insight on the development cooperation in Nepal and in particular called for results on NGO co-operation.

ANNEX D Survey methods and results This chapter includes more detailed information on the survey methods, the survey questionnaire and the results of the survey. For information to assess achievement of the project results, it would have been valuable to study all the trainees, Finnish and other nationals. However, on the experience of the survey made in the project in 2009, it was clear that we would be able to reach mainly Finnish participants. Therefore, only Finnish participants in the Global Health courses were chosen for this survey, yet they consisted of around 50% of all trainees during 2001-2013.

Description of the survey methodology Subjects of the study Finnish participants in all Global Health courses in Finland or in the partner countries during 2001-2013 were chosen as the study subjects. There were a total of nine courses held in Finland between 20012013. The number of Finnish participants in these courses was 156. A total of five courses were held in the partner countries during 2001-2012. The total number of Finnish trainees in these courses was 53. Designing the survey The evaluation team constructed the questionnaire in order to find out results of the courses, especially regarding impact and effectiveness, and to map some basic information about the alumni population. In the questionnaire, there were 20 questions on 9 pages. Questions were divided into categories

according to theme. The survey included statements and questions on the following topics: • • • • • •

GH training in the university curriculum Further GH training GH awareness GH course feedback Engagement in GH related activities Career in GH

The questionnaire was moved online using the Survey Monkey platform so that it could be easily filled on the Internet. Contacting subjects To reach the subjects and request them to fill the questionnaire, their contact information was searched for. Unfortunately, no complete and upto-date email list or contact details were available in the project. With the help of Google search and Population Register Centre of Finland, we were able to find an up-to-date email address or postal address to about 80% of the people. The remaining 20% either had insufficient personal details for our search (could not be identified in the Population Register) or had requested their contact details to be confidential. Respondents were invited by email, social media and postal mail to take part in the survey online. As many as three reminders were sent during the twoweek period the survey was open. The survey access Subjects could enter the survey through a URL address provided in the invitation. Project and evaluation description were included in the invitation, as well as note on anonymity of the survey results. In the survey, questions were divided into nine categories. After completing a category, a respondent would continue on the next page. The approximate time to complete the survey was 10 minutes. The survey was active from 28 July to 18 August. Final subjects Of the total of 194 Finnish alumni in this subject population, 118 responded to the questionnaire (61%). Out of the respondents, 101 were women (86%), and only 17 were men. This is mostly due to more women taking part in the courses from Finland. Age of respondents varied from those born in the 1950s to alumni under 30 years old. Up to 58 respondents (49%) were born in the 1980s. Only 18 respondents (15%) were born before 1975. This is mostly because course trainees were to large extent chosen from students or those recently graduated. Most of the respondents (and overall course trainees) were

trained in medicine, and only a small portion of them was veterinarians or dentists. Limitations of the study At the end, a total of 39% of the alumni were with no response: around 20% could not be contacted at all, and 19% did not respond. These 39% were mainly alumni from courses organised in Finland before 2009. Fortunately, each course and year still had around 50% respondents. Therefore, it seems that there is not a large bias but the responses represent relatively well all the subjects. Hence, the results can be considered as representative of all the Finnish trainees with caution.

The reliability of the responses was most likely compromised by non-precise wording of the questions or options given. Pilot testing of the questionnaire would have been valuable but due to time limit it was not done. However, the responses in general made sense and were in line with other information from the other sources. As a majority of the most recent (2012-2013) alumni are still students or recent graduates, they would have spent a very short time in any employment. Therefore, it is less likely for these alumni to have found employment in global health related positions.

It is possible that the responses were too positive or favourable to the course. We did not use triangulation within the survey itself, but did compare the results with the interviews held.

results and conclusions GH for undergraduate education About half of the Finnish respondents suggested having a GH course integrated in medical undergraduate education as an elective, and the other half as a compulsory training, for 2-4 weeks. Some suggested a combined course with 2-4 days compulsory to every student followed by different options of electives. It was also pointed out that the elective course on medicine in developing countries (kehitysmaalääketiede in Finnish) presently available in all medical faculties is not sufficient and as relevant as “global health”. The alumni could be engaged with putting pressure on medical faculties to offer at least an elective course on GH to students. They could also assist in planning and implementing such a course. Views on the GH course Over half of the Finnish respondents said the course had changed their thinking on the importance of GH issues much or extremely much. (There is a possible bias here as those who had not changed their thinking probably viewed GH important already before the course). See figure 1 on page 51. When asked to choose on options on themes that one benefitted the most in the course, 80% reported the experience of visiting a developing country and 60%

of discussions with trainees from other countries among the most important ones. Key themes that the Finnish alumni reported to have learnt in the GH course included a large variety; all of them are relevant GH topics. The most frequently mentioned as a GH issue was poverty or inequalities in health. Maternal health was also reported (maybe as in Finland it is not an issue). See more on the back cover of this report, page 52. Studies after the course Only 10 of the Finnish respondents had studied at least one of the five project-made eLearning courses in GH. Of the Finnish respondents, 40 % had participated on some other global health training after the GH course. Several of them had taken more than one course. Practically all had taken a course “Health in developing countries” - an open evening class during several months, organised by all medical faculties with rather similar contents as the GH course. Eighteen had studied a short course in Finland or abroad, with a wide selection. Eleven had studied or were studying a master-level course with GH included, many held abroad, such as in Edinburgh, Sweden, London, Copenhagen and Mumbai. It is concluded that there is a definite need to make a better use of the five eLearning course that the project experts have designed and made publicly available. It could be included also as part of a course in undergraduate and graduate education in Finland and in the partner institutions. Seeking for further training on global health was used here to indicate (at least to some extent) sustained

interest in GH. It is concluded that most (60%) had not shown further interest in such a way. On the other hand, there seems to be a considerable proportion of the alumni who are actively educating themselves through several courses, also abroad and also at masters´ level. This positive finding may be due to the GH course or the selection of the trainees.

”

I follow multiple GH organisations and people via Twitter and Instagram: WHO, Red cross, Doctors without borders, GH US, Anthony Costello etc.

- A Finnish alumni

Follow up of GH issues Another indicator of sustained interest on GH used in the survey was on how an alumni followed GH news and updated one´s knowledge on the web or through other resources. Of the Finnish respondents, half reported they occasionally sought for GH information when they surfed on the web. Fifteen percent said they systematically followed certain GH sources (websites, journals, others). The most frequently mentioned were Lancet (journal), the WHO website and blogs. See figure 2 on page 51. The findings indicate that some alumni still have some interest on GH. Yet, one would expect that a GH course would result in systematic follow-up through for example RSS feeds and other methods. In the future, the need for updating oneself and the tools to use could be emphasised in the training. Networking One third of the respondents have kept in touch with other participants after the course. Nevertheless, it is quite expected that most have not kept in touch throughout the years. A more relevant indicator of networking could have been frequency of networking with other GH interested persons and experts. Six alumni reported that their contact with the others had resulted in a GH project. Paid or volunteer work in GH It seems that the GH course influenced professional careers or at least contents of work of the alumni. Over half reported that they were clearly aware of different things in their work than without the course. About 40% said they started to seek for international jobs, or GH related work or to modify one´s work so that GH issues were part of it. See figure 3 on page 51. Eighteen out of 118 respondents had worked in a global health related paid job, varying from 3 months to 4 years and mostly abroad in countries such as Haiti, Nepal, Peru, Burundi and Italy. Ten had done

such work for over one year at a time. One third (about 30%) reported having worked as volunteers in GH issues. The organisations included NGOs in Finland or abroad or International NGOs or NGOs of another country. Countries included India, Chile, Indonesia, Nepal, Tanzania, Zamibia, Ghana etc. The alumni worked as a medical doctors, dentist, consultants, project coordinators, board members, secretaries, etc.

I am a Global Clinic activist providing free heath care for illegal immigrants in Finland. I worked as a volunteer for 4 weeks in Tanzania at a hospital.

” ”

- Two Finnish alumni Other overall comments that the respondents wished to make:

One important thing which I learnt for myself, was ”Hakuna matata”, take it easy. I had quite a stressful job in Finland that time, and there in Africa I was really happy to learn toslow down: you can be effective enough even if you can not be faster, and sometimes ”to do something” is enough, you do not need to do it perfectly every time... We need to learn realistic view to our own work too, and I learnt plenty of good attitude in Tanzania.

The best course! Changed my views profoundly about GH issues. E.g. recently on a honeymoon in Beijing, we asked the guide to take us to an organized visit in a hospital, an elderly home and an orphanage to see how global health issues are organised there and to discuss with the professionals. I do think that it´s of great benefit for occupational health physisicians to know GH-issues. - Three Finnish Alumni

ANNEX E LIST OF MEETINGS AND PERSONS INTERVIEWED List of meetings and evaluation schedule

Meetings and interviews during the field mission to Nepal:

List of persons interviewed

20 January 2014 Kick-off meeting

28 April Evaluation briefing with Nepal core team

Nepal:

2 February 2014 TOR draft submitted 11 February 2014 TOR accepted 24 February 2014 Inception discussion 25 April 10 May Field mission to Nepal 2 August 2014 Global Health Summit, Helsinki 3-4 August 2014 Global Health Update Seminar, Tampere 5-6 August 2014 ToT, Tampere 2-6 August 2014 Interviews with partner country core teams 23 June 2014 Evaluation progress meeting

28 April Group interview with 6 Nepal GH course alumni 29 April Discussion with Dean of IOM 29-30 April Nepal core team interviews 1 May Alumni case study meeting at CHEERS 2 May Group interview with 7 Nepalese Finland course alumni 2 May Discussion with Senior Officer, Dr Babu Ram Marasin, Ministry of Health, Nepal 9 May Discussion with The Ambassador of Finland, Mr Asko Luukkainen

Dr Babu Ram Marasin, MD, Head of Epidemiology, Ministry of Health, Nepal Prof Rakesh Prasad Shrivastav, The Dean of IOM, Tribuvhan University Prof Dr Jagdish Prasad Agrawal, Executive Director, National Centre for Health Professions Education (NCHPE) (Head of Nepal core team) Dr Pradip Gyawali, Co-ordinator, Global Health Unit, NCHPE, IOM Dr Archana Amatya, Global Health Unit, NCHPE, IOM (Co-ordinator of Nepal core team) Prof Mandira Shahi, Global Health Unit, NCHPE, IOM, GH alumni, GH core team Dr Niraj Bam, Global Health Unit, NCHPE, IOM Dr Santosh Timalsina, Global Health Unit, NCHPE, IOM, GH core team, GH alumni

1-17 August 2014 Online survey

Ms Shakun Sharma, GH alumni

21 August 2014 First evaluation report draft submitted to partners

Mr Prakash Aryal, GH alumni

Mr Pramod Kattel, GH alumni Mr Dilip Kumar Yadav, GH alumni

26 August 2014 Review meeting for first draft report

Ms Durga Gautam, GH alumni

30 September Final evaluation report submitted, seminar

Mr Bishnu P. Choulagai, GH alumni

Mr Poojan Shrestha, GH alumni Mr Prem Basel, GH alumni

Mr Ojbindra KC, GH alumni

Dr Elia John Mmbaga, Tanzania

Dr Anish Mudvari, GH alumni

Dr Germana Henry Leyna, Tanzania

Dr Sadichhya Lohani, GH alumni Mr Bhojkumar Basnet, Account Office, IOM Mr Sureshkumar Shrestha, Accountant, IOM Mr Tahal Bahadur, Chief Accountant, IOM Mr Shiva Raj Mishra, People’s Health Movement, Nepal Dr Prasanta Poudyal, GH alumni Dr Madan Upadhyay, Director of CHEER children’s hospital Prof Dr Bimal Kumar Sinha, Asst. Dean Planning, IOM

Finland core team interviews: Prof Dr Per Ashorn, University of Tampere Dr Ulla Ashorn, University of Tampere Dr Juha Pekka Turunen, Duodecim Ms Anna Pulakka, University of Tampere Dr Kirsi-Maarit Lehto, University of Tampere Dr Ulla Harjunmaa, University of Tampere

Other interviews:

ANNEX F LIST OF MAIN DOCUMENTS REVIEWED Some of the key documents reviewed for this evaluation: Funding applications to MFA for all these years: 2001, 2002, 2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014 Annual reports to MFA for all these years: 2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013 List of participants, trainers and organisers for all years and all courses

Prof Dr Rajendra Raj Wagle, HOD Community Medicine and Public Health, Tribuvhan University

Dr Taneli Puumalainen, Leading doctor, Ministry of Social Affairs and Health, Finland

Financial accounts for: 2006-2013

Mr Asko Luukkainen, The Ambassador of Finland to Nepal

Dr Tom Sundell, MD, author of project evaluation in 2005

Interviews during the GH week 2-6 August 2014 - partner countries, core teams:

Survey respondents:

Course evaluation materials: 2008 Tanzania, 2009 Philippines, 2010 Finland, 2010 Tanzania, 2011 Finland, 2011 Philippines, 2012 Tanzania, GH Summit and Seminar 2012, 2013 Finland, 2013 Nepal

Dr Cristian Rebolledo, Chile Dr Alejandra Fuentes García, Chile Dr Louricha Opina Tan, The Philippines Dr Gene Alzona Nisperos, The Philippines

120 Finnish GH alumni (courses 2001-2013)

Curricula and programmes for all the courses held in 2006-2013

2005 Evaluation report by Dr Sundell 2009 Email survey to alumni Training materials for ToT 2014 Memos from the Steering Committee meetings: 2011-2014

Dr Edelina Padilla Dela Paz, The Philippines

ANNEX G TERMS OF REFERENCE Evaluation, March 2013 Revised 23.6.2014 (in cursive) UHA 2011-006027

1.2. The Global Health Training Project The project to be evaluated trains health professionals who can look beyond the immediate causes of diseases and recognize the importance of poverty and other social determinants on health. 1.3. The purpose, scope and objectives The project documentation defines the project objectives as follows. The overall objective is to alleviate global ill health and to diminish regional inequalities in health. “The project addresses uneven global distribution of health. Successful progress in global health development, e.g. achievement of the health related UN Millennium Development Goals requires that responsible professionals know worldwide variation in health, health systems and health interventions and share a positive vision for global health development. Reaching this target is, however, hindered as health professionals all over the world are mainly trained to tackle health from a local or national perspective.” The immediate (direct) objectives are, as in the project documentation:

1. Background 1.1. Global health Global health is the health of populations in a global context and transcends the perspectives and concerns of individual nations ((Brown et al 2006). In global health, problems that transcend national borders or have a global political and economic impact are often emphasized (Global Health Initiative 2008). Global health has been defined as “the area of study, research and practice that places a priority on improving health and achieving equity in health for all people worldwide” (Koplan at al 2009). Thus, global health is about worldwide improvement of health, reduction of disparities, and protection against global threats that disregard national borders. With expanding globalization, any nation is closely connected to and interdependent of the others also in the sphere of health and health care. Acute and chronic emergencies, health threats from other countries, global economy, international trade agreements, development aid, climate change, poverty, mobility of people, global balance of power, political trends and many more influence a nation directly or indirectly. This sets a huge and increasing need for competencies of those working in health field at any level, from clinical work to high politics. They must have knowledge on global health and be able to apply the knowledge at one´s work and responsibilities.

• “to provide current and future health care practitioners, teachers and decision makers in Finland and in the Partner countries with knowledge, skills and motivation to implement successful health sector interventions and capacity building activities in their own countries and as a part of international collaboration and, • to increase the institutional capacities of all the collaborating countries, especially in the South.” 1.4. Partners The Finnish Medical Society Duodecim is the officially responsible organization of the project. Duodecim has subcontracted the content production (various courses, events and collaboration with Partner countries) to the University of Tampere. The project brings together a large number of Finnish NGO’s, governmental institutions and universities as the Finnish GH Consortium: • Medical Faculties at the Universities of Helsinki, Kuopio, Oulu, Tampere and Turku • Dental faculties at the Universities of Helsinki and Turku, Finnish Medical Association • Finnish Medical Students’ International Committee • Finnish Dental Association, Finnish Dental Society Apollonia • The Finnish Veterinary Association, Physicians for Social Responsibility • The Finnish Society for International Health • National Institute for Health and Welfare

• Odontologiska Samfundet i Finland and Veterinary Faculty at Helsinki University The role of the above listed partners is to provide education planning expertise and voluntary work and collect funds for self-financed part of the project. The international Partner institutions are: • Universidad de Chile, Facultad de Medicina, Escuela de Medicina, Santiago, Chile • Tribhuvan University, Institute of Medicine, Kathmandu, Nepal • College of Medicine, University of the Philippines, Manila, the Philippines • Sokoine University of Agriculture, Tanzania • Muhimbili University, College of Health Sciences, Tanzania. The role of the Partner institutions is: 1) to select students to attend the courses and to send one faculty per course to teach, and 2) to organise GH training in their countries in order to build capacity for their nations and to 3) to send faculty members for Global Health training week in Finland. International Partners have also participated actively in creating e-learning courses. 1.5. Activities of the Project The Project has evolved during its lifetime according to the progress and needs. The activities since 2001 have been: 1. The Finnish Certificate Course on Global Health has been arranged in Finland annually since 2001 (except in 2003, 2005, 2012 and 2014). It has had participants from Finland and the current four Partner countries and during the first courses from South-Africa, China, Nigeria, and India. . 2. Since 2008, a GH courses have been held also in the Partner countries; Tanzania in 2008, 2010, and 2012, the Philippines in 2009, 2011 and Nepal in 2013. In the Philippines there have been additional local courses that were partly supported with funding from Finland. Up to date, approximately 300 medical, dental and veterinary students and doctors from Finland, Chile, China, India, Nepal, the Philippines, Malawi, Nigeria, and Tanzania have taken the GH course held in Finland. GH course in partner countries has gathered together almost 200 students. The faculty has so far included approximately 60 senior researchers/ professors/lecturers from the Partner countries and 60 European global health experts. Together, the students and faculty now form a network of more than 600 global health minded professionals. 3. The first e-learning course on GH was developed and made available in 2011 and new modules to the course have been added in 2012 and 2013.

4. An intensive training of trainers (ToT) workshop in Finland with the Partner Institutions in Tanzania, the Philippines, Chile and Nepal was organized in 2012 and is planned for 2014 to enhance running of the GH courses in the Partner Institutions. 5. A high profile summit on topical global health issues was organised in 2012 and is planned for 2014 as well. 2. Purpose and objectives of the evaluation The evaluation shall provide an overall assessment of the performance and impact of GH Training project, focusing on the evaluation questions listed below. In addition to identifying key successes to date, attention shall be paid to areas where project implementation, execution and delivery have been challenging. The results of the evaluation will primarily be used in the context of the GH Steering Committee and Executive Board as well as meetings of the Finnish and international Partners to provide information on lessons learned and strategic guidance for future organisation of Global Health capacity building in the countries involved and elsewhere. 3. Scope of the evaluation The evaluation shall cover all its components as implemented in different years. The evaluation shall focus mainly on the period 2006-2013 (partly 2014). The early period of the project (2000-2005) should be discussed regarding reasons and needs for starting such a project, and in particular on developing networks within Finland and between countries. The evaluation shall also examine enhancement of the South-South co-operation. Progress in each one of the four Partner countries shall be assessed in the context of the project. The evaluation shall also assess the operations of the oversight and governance structures of GH project. The time period under evaluation is from the start of the project in 2001 to date. 4. Evaluation issues The main issues will be studied against the evaluation criteria. Relevance Relevance concerns whether the results, purpose and overall objectives of the intervention are in line with the needs and aspirations of the beneficiaries, and with the policy environment of the intervention. It will be further inquired if the intervention is consistent with the needs and priorities of the target groups and the policies of the partner country and donor agencies. Additionally, has the situation changed since the approval of the intervention (e.g. programme/project) document?

Efficiency The evaluation will include management performance. How well the activities (training, ToT, eLearning and summit) have transformed the available resources into the intended outputs or results in terms of quantity, quality and time. Can the costs of the intervention be justified by the results? Effectiveness How well the results have furthered the attainment of the purpose of the GH training project? Has the GH project achieved its objectives? Impact Has progress been made towards achieving the overall objective of the GH training project? What are the overall effects of the GH training project- intended and unintended, long term and short term, positive and negative? Sustainability To what degree the benefits produced by the GH training project are likely to continue after the external support has come to an end. Will the benefits produced by the Project be maintained after the termination of external support? 5. Cross-cutting issues Evaluation should examine all the cross-cutting issues of the Finnish development cooperation: human rights, reduction of inequalities of kinds, promoting gender equality and strengthening of climate sustainability. Such questions include: have the cross-cutting issues been included in the contents of the GH training? Is gender balance been considered in selection and recruitment of students, staff and teachers? How is sustainable development been enhanced and promoted in the Project? 6. Methodology The methodology will be developed and proposed by the consultant. The detailed evaluation methodology shall be presented in the inception report. It will be finalised after discussions with the Executive Board and prior to starting the field phase. It is foreseen that the methodology will consist of background studies and analysis, data collection in a structured form and in a form of various interviews and workshops/ seminars in the field. Both qualitative and quantitative methodologies should be used. Validation of results will be done through multiple sources. The interviews will include policy, institution and individual-levels, and they should reflect the holistic contents of Global Health. The evaluator/s shall develop an evaluation matrix outlining the evaluation criteria, evaluation questions, indicators for each criteria and methodology and/or

source of data or verification. The result of the evaluation exercise will be well analysed and findings and recommendations justified. The consultant shall follow the guidelines presented in the MFA Evaluation Manual 2013, including checklists for evaluation quality and evaluation report templates. The link to the manual can be found in the end of this document. 7. Time Schedule and Reporting The evaluation shall commence immediately after the Terms of Reference have been agreed in the Executive Board, preferably in February-March 2014, provided that the process is approved by the Ministry for Foreign Affairs desk officer. The assignment will begin with briefing meeting with the Executive Board members and selected key actors of the GH Project followed by provision of all relevant material by the project staff to the evaluator. The Consultant will conduct a thorough desk study of the existing material and documentation. On the basis of the desk study the Consultant will prepare an inception report including a work plan, evaluation matrix, detailed methodology for the field visit/s and the list of issues to be studied. Field mission will be conducted in one or two partner countries. The Consultant will be assisted by the Project Staff and Partner representatives. The meetings should cover as wide range of project stakeholders as deemed necessary. To complete the field visit the consultant will provide a debriefing for the key stakeholders towards the end of the field visit. On the basis of the above, the consultant will prepare a draft evaluation report. The partners will be given time to provide comments on the draft report. The final evaluation report is to be submitted after having received feedback and comments from the stakeholders. An indicative work plan and schedule is: 20.1.2014 Discussions on the need and wishes for the evaluation, focus and implementation with project Executive Board members: Per Ashorn, Ulla Ashorn, Anna Pulakka, Ulla Harjunmaa, Juha Turunen, Anneli Milen. by 3.2. Delivery of the documentation to the consultant (Ulla Ashorn) Draft Terms of Reference to be commented. Contract. Discussion with the International Partners on the evaluation. (Ulla Ashorn). February Inception discussion with the Executive Board.

April-May Field visit/s June Meetings, discussions and interviews with partner institution reps in connection with the Training of Trainers Workshop in Tampere. by 28.8 Draft Final Report and discussion with the Steering Committee. by 30.9. Final Report The final evaluation report shall include an analytical description of the results of the evaluation, focusing on the purpose and objectives of the evaluation and the evaluation questions. It should present the major results, quantitative and qualitative, achievements, successes and failures in a clear and concise manner. The final report shall include also a section for clear conclusions and recommendations based on verifiable evidence. It shall also indicate identified needs and recommendations. The total length of the main body of the final report should remain below 30 pages plus annexes. Outside the main body of the text, immediately after the contents and acronyms should be a brief executive summary both in Finnish and in English. The report may contain graphs, drawings, tables and boxes and also photographs. Additional, relevant information can be presented in Annexes. The language of the report shall be fluent English and the report edited strictly according to the guidelines of the Ministry.

• Programme/project evaluation, including areas such as Project Cycle Management, Results Based Management and Logical Framework Approach, development and usage of baseline and disaggregated data (gender, income group, etc.) in project design, monitoring and evaluation. • Experience and knowledge demonstrated in implementation and mainstreaming of the cross cutting objectives of Finnish development cooperation: human rights, reduction of inequalities, gender equality and climate sustainability. To support the training of new evaluation professionals, the team may additionally include a Junior Expert. The junior expert must have at least a Master’s level degree, be a resident of Finland and have working knowledge of English. 9. Mandate During the assignment the consultant is entitled and expected to discuss with the pertinent persons/ organisations any matters related to the assignment. However the consultant is not authorised to make any commitments or statements concerning further activities on behalf of the project/client. Annexes Annex A.1: Link to the evaluation manual: http://formin.finland.fi/public/default.aspx?contentid=2 88455&contentlan=2&culture=en-US

8. Expertise required The composition of the consultant team is not predetermined, but shall include a minimum of one member. One person shall be nominated as the Team Leader. One member could come from a Partner country. The consultant/ team must have excellent command of English. The Consultant team shall ensure solid knowledge and experience in the following fields: • Global Health and its topics and aspects • Competencies needed on Global Health issues in developing-country contexts • Capacity building and competencies development; theory, methods and practice. Human resources development and institutional development strategies. • University education systems and approaches in health

literature notes Chapter 1 1. http://www.formin.fi/Public/default. aspx?nodeid=43862

14. Finlandâ&#x20AC;&#x2122;s Development Policy Programme. Government Decision-in-Principle 16 February 2012. Ministry for Foreign Affairs, Finland.

2. Sundell Tom: 1st, 2nd and 3rd Finnish diploma course on global health. Final Report and Evaluation. Publications of the Public Health Institute, Helsinki. B23/2005. Evaluation. Publications of the Public Health Institute, Helsinki. B23/2005.

15. Development Cooperation of the Civil Society Organisations â&#x20AC;&#x201C; Project Guidelines. Ministry for Foreign Affairs, 2012.

Chapter 3 3. Fried, Bentley, Buekens et al. Global health is public health. Lancet 2010; 375:535-537. 4. Koplan, Bond. Merson et al. Towards a common definition of global health. Lancet 2009; 373: 1993-95. 5. Brown, Cueryo, Fee. The WHO and the transition from international health to global public health. Am J Public Health 2006; 96:62-72. 6. Frenk, Gomez-Dantes, Moon. From sovereignty to solidarity: a renewed concept of global health from an era of complex interdependence. Lancet 2014; 383: 94-97. 7. Beaglehole, Bonita. What is global health? Global Health Action 2010; 3: 5142 8. De Ceukelaire, Botenga. On global health: stick to sovereignty. Lancet 2014; 383:951-2. 9. Rowson, Smith,Hughes et al. The evolution of global health teaching in undergraduate medical curricula. Globalization and Health 2012; 8:35. http:// www.globalizationandhealth.com/content/8/1/35 10. Peluso, Forrestel, Hafler, Rohrbaugh. Structured global health programs in U.S. medical schools: A web-based review of certificates, tracks, and concentrations. Academic Medicine 2013;88:124-130. 11. Association of American Medical Colleges. GH Program Evaluation Survey. All Schools Summary Report Final. Washington, DC: Association of American Medical Colleges, 2012.

Chapter 5 16. http://www.oecd.org/dac/evaluation/ dcdndep/45438179.pdf and http://www.oecd.org/dac/ evaluation/dcdndep/41612905.pdf 17. http://ec.europa.eu/europeaid/how/evaluation/ methodology/index_en.htm 18. http://formin.finland.fi/public/default.aspx?conten tid=288455&contentlan=2&culture=en-US Chapter 6 19. R. Horton: Offline: The case against global health. Lancet: 383: 1705, 2014. In this editorial of a leading medical journal Lancet, the chief editor quotes the debate in the 2014 Conference of the Consortium of Universities for Global Health. His point is that it is important to argue deliberately and aggressively against an idea you passionately believe in as there may be a truth in it. See also: R. Norton: Offline: Is global health neo-colonialist? Lancet 382: 1690, 2014. 20. Rowson et al.: The evolution of global health teaching in undergraduate medical curricula. Globalization and Health 2012 8:35. 21. Cross-cutting objectives in the Development Policy Programme of the Government of Finland; Guidelines. Annex C 22. http://www.formin.finland.fi/public/default.aspx? contentid=258009&contentlan=2&culture=en-US

12. Battat et al. Global health competencies and approaches in medical education: a literature review. BMC Medical Education 2010, 10:94 13. http://enwikipedia.org/wiki/Instirtuinal_design

Much 48%

Somewhat 31%

Not at all 1.7 % Extremely much 8%

Slightly 12%

Figure 1. To what extent the course changed your thinking about GH issues in the work of health professionals?

53% Occasionally, I surf on the web and seek for global health information.

32 % I do not actively seek or follow global health information, but may read if I see something.

15% I systematically follow certain global health information sources (websites, journal, others).

Figure 2. How much do you follow global health news and update your knowledge independently on the web and other resources?

20% Not particularly.

52 % I am clearly aware of different things in my work than without the course.

18% I started to seek international jobs in Finland or abroad.

20% I started to seek global health related work or modify my work in such a way that global health themes are part of it.

Figure 3. How has the Global Health course influenced your professional career?

MATERNAL AND CHILD HEALTH ISSUES, POVERTY AND ITS CAUSES TO HEALTH, WEATHER AND ITS RELATION TO HEALTH, CULTULAR HABITS AND BELIEVES, INEQUITY, DISASTERS, MATERNAL HEALTH, DETERMINANTS OF POPULATION GROWTH, PROBLEMS RELATING TO THE TANZANIAN HEALTH CARE SYSTEM, THIRD WORLD DISEASES (AIDS, MALARIA ETC), WATER CONTAMINATION, HEALTH/WELFARE INDICATORS, MAKING DIFFERENCE IN GLOBAL HEALTH INVOLVES MORE POLITICS THAN MEDICAL PRACTISE, HEALTH INEQUALITIES ARE HUGE- BUT PEOPLE STILL CAN BE SATISFIED WITH THEIR LIFE, HEALTH IS A HUMAN RIGHT AND SHOULD BE EQUAL TO ALL, THE IMPORTANCE OF WOMEN’S EDUCATION, THE ROLE OF NGO’S, STRUCTURE OF DIFFERENT HEALTH SYSTEMS, GLOBAL WARMING, CHILD MORTALITY, THE IMPORTANCE AND ACCESSIBILITY OF VACCINES, BREASTFEEDING ETC, SAFE BIRTH, THE SIGNIFICANCE OF GENDER EQUALITY IN GLOBAL HEALTH ISSUES, MENTAL HEALTH ISSUES - CULTURAL ASPECT, ENVIRONMENTAL ISSUES, FUNDING, HEALTH POLICIES, BIRTH CONTROL, MATERNITY AND CHILD HEALTH, NUTRITION, INFECTIOUS DISEASES, DIFFERENCIES IN HEALTH CARE SYSTEMS IN DIFFERENT COUNTRIES, THE EFFECT OF GDP TO HEALTH CARE IN DIFFENRENT COUNTRIES, THE VARIETY OF HEALTH PROBLEMS IN DIFFERENT COUNTRIES (IE. OBESITY VS. MALNUTRITION), PUBLIC VS PRIVATE HEALTH CARE SYSTEMS, IMPLEMENTATIONS, ADVANTAGES AND DISADVANTAGES), GLOBAL BURDEN OF DISEASE, MANAGING AN EFFECTIVE HEALTH PROMOTING INTERVENTION, EVALUATING DIFFERENT HEALTH SYSTEMS, COMPOSITION OF THE GLOBAL COMMUNITIES AND ACTORS IN GLOBAL HEALTH, HEALTH CARE MODELS, MILLENNIUM GOALS, GLOBAL BURDEN OF DISEASE, HEALTH CARE INDICATORS, CHILD HEALTH, INFECTIONS, CHRONIC DISEASES GLOBALLY, MATERNAL HEALTH, BASIC HEALTH CARE DIVERSITIES, HEALTH CARE POLITICS AT GRASSROOT LEVEL, INFECTIOUS DISEASES & GLOBAL HEALTH, VETERINARY & GLOBAL HEALTH, PREVENTION, CORRUPTION, BUREAUCRACY, CO-OPERATION, MALNUTRITION, SOCIOECONOMIC PROBLEMS AFFECT HEALTH, EFFECTS OF CLIMATE CHANGE ON HEALTH IN DEVELOPING COUNTRIES, MILLENIUM TARGETS, WHO’S ROLE, INEQUITY OF MEDICAL AND DENTAL CARE INSIDE A DEVELOPING COUTRY, IMPORTANCE OF EDUCATION IN DEVELOPING COUNTRIES, THE EFFECT OF HUMAN RESOURCES ON HEALTH CARE, DISTRIBUTION OF MEDICAL SERVICES, THE UNDERLYING FACTORS OF CHILDHOOD MORTALITY, AN UP TO DATE INTRODUCTION TO MAIN INFECTIOUS DISEASE PROBLEMS IN DEVELOPING COUNTRIES, INTRODUCTION TO MAIN MATERNAL AND CHILD HEALTH CHALLENGES IN THE GLOBAL HEALTH FIELD, INTRODUCTION TO DIFFERENT HEALTH CARE PROVIDING SYSTEMS AND THEIR CHALLENGES, INTRODUCTION TO DIFFERENT (TYPES OF) PLAYERS/ORGANISATIONS IN THE GLOBAL HEALTH FIELD, CULTURE AND HEALTH, BURDEN OF DISEASE, EDUCATION AND HEALTH, WORKING TOGETHER TO IMPROVE GLOBAL HEALTH, HEALTH INEQUALITY IN MANY PERPECTIVES, KNOWLEDGE AND KNOW HOW OF HEALTH PERSONNELS GLOBALLY, HOW HEALT DEPENDS MANY STAGES OF SOCIETY, DISEASES AND BIGGEST PROBLEMS WITH HEALTH IN DIFFERENT PART OF PLANET, DIFFERENT HEALTH ISSUES GLOBALLY, MANY HEALTH ISSUES IN DEVELOPING COUNTRIES RELATED IN POOR EDUCATION AND POVERTY, ROLE OF INTERNATIONAL ORGANISATIONS, NON-EXISTING EQUITY, EDUCATION, HEALTH, EQUITY AND SO ON ALL GO HAND IN HAND, UN MILLENNIUM DEVELOPMENT GOALS AND THAT THEY WERE NOT REACHED, INFECTIOUS DISEASES ARE NOT THE MAIN THING, SOCIAL DETERMINANTS OF HEALTH AND THEIR IMPORTANCE, BASIC INDICATORS OF HEALTH (EG UNDER-FIVE MORTALITY RATE) AND THE MILLENNIUM GOALS, THE WAY TO EFFECTIVELY WORK FOR THE BENEFIT OF A DEVELOPING COMMUNITY - VERTICAL AID/ HORIZONTAL AID, MULTIPLE AID ORGANIZATIONS IN THE WORLD, COLLABORATION WITH THE LOCAL GOVERNMENT IMPORTANT, DIFFERENT KINDS OF HEALTH SYSTEMS, CLEAN WATER IS REALLY A BIG ISSUE GLOBALLY, TANZANIAN WORKING ETHICHS, GLOBALLY THE HEALTH ISSUES AND CHALLENGES ARE QUITE SIMILAR IN DEVELOPING COUNTRIES BUT DIFFER A LOT BASIC HEALTH ISSUES COMPARED TO INDUSTRIAL COUNTRIES, AT A MILITANCY ENVIROMENT HEALTH SITUATION CAN SUDDENLY TURN INTO DISASTROUS EVEN THE SITUATION EARLIER WAS SATISFYING, MALNUTRION OF CHILDREN IS ONE OF THE LARGEST PROBLEMS, POLITIC, BUSINESS AND TRADIOTIONAL HABITS HAVE AN LARGE IMPACT ON GLOBAL HEALTH, EQUALITY IN HEALTH, POVERTY AND HEALTH, INTERNATIONAL HEALTH AGENDAS, HEALTH AND HUMAN RIGHTS, HEALTH THREATS ARE MORE AND MORE SHARED AROUND THE GLOBE, INEQUALITY IN RESOURCES AND HEALTH, A POTENTIAL FOR CONFLICTS IN THE FUTURE, MIGRATION RELATED HEALTH PROBLEMS, IMPACT OF ENVIRONMENTAL CHANGES AND GLOBAL WARMING ON HEALTH, IGNORANCE CAUSES POVERTY, POVERTY CAUSES GREED, GREED MAKES EVERYTHING HARDER, POWERLESSNESS IS MAIN CAUSE FOR WOMENS SUFFERING, MALARIA, NUTRITION, MATERNAL HEALTH, MATERNAL HEALTH, PERINATAL HEALTH, HUMAN RIGHTS, ENVIRONMENT, ACCESS TO MEDICINE, HUMAN RIGHTS IN HEALH, GENDER, CULTURE, SEXUALITY AND HEALTH, NON-MEDICAL DETERMINANTS OF HEALTH, ESSENTIAL MEDICINES AND ACCESS TO THEM, EMPOWERING WOMEN, FAMILY PLANNING, HEALTH INDICATORS, POVERTY REDUCTION, IMPORTANCE OF CO-WORKING, HEALTH SYSTEMS, UNIVERSAL HEALTH CARE, NON-COMMUNICABLE DISEASES What is Global Health? - Responses by Finnish alumni of the Global Health courses

global health training programme evaluation report 2014