FOCUS ON THE RESULTS Measuring the outcomes of internationalisation – from IMPI to EIS Uwe Brandenburg, CHE Consult Bogotá, November 25, 2016
www.che-consult.de
Structure of the presentation Who is CHE Consult Background on Internationalisation Reality and wishful thinking A first start: the IMPI project The memo© approach Results from the Erasmus Impact Study Other impact studies of CHE Consult: EVS and Inhope
2
Who/what is CHE Consult? History:
founded 2001 as a spin-off of CHE (known e.g. for the Ranking in Die Zeit and the U-Multirank)
Currently owned by Christian Berthold and Uwe Brandenburg
What does it do?
Consulting: usually universities, small to medium size, consulting on all aspects of universities (such as mindset change, organisational change, change management, etc), mainly in Germany and also abroad (Spain, Japan, Italy…)
Applied Research: usually EC/ministries/networks/foundations, large scale, multiple years, international but also for the BMBF, usually around internationalisation, employability and impact assessment
Lots of publications on internationalisation, e.g. Erasmus Impact Study or Change in Chinese Higher Education (so far 95,000 downloads) 3
ď€ Background on Internationalisation
4
Internationalisation Buzzword of at least the last decade and a half Started off in political science and governmental relations, entered higher education in 1980s Last two decades: concept of the internationalisation moved from fringe to core of institutional agenda Effects: tends to become a conditio sine qua non creates an atmosphere of high risk of lip service and “give to the emperor…” 6
Core assumptions and statements Internationalisation No goal in itself but effective instrument to achieve other goals As IMPI said 2009: can enhance education, research, civic engagement etc. Measuring Internationalisation So far too many assumptions made, to few things proved: we claim but we do not know! Satisfaction surveys dominate next to input measurement Outcome (impact) is not measured yet largely… 7
ï€ Reality and wishful thinking
8
Rationale for internationalisation
quick
Time
low
slow Equilibrium high
Costs Quality low
high
University culture
Rationale for internationalisation - an example of intelligent internationalisation I2 Goals
The public
Can vary by actor
Can be contradictory between layers
Can be clear on some layers but not on others
National strategy Decentralised strategies
HE system HEI HEI system Faculties
Institutional strategy
11
Achieving and accountability: measure what matters… Input
Output
Defined as the resources invested in internationalisation
Defined as the direct results of internationalisation
Examples: staff in the IRO, partnerships, stipends, website
Examples: number/percentage of exchange students, percentage of staff from abroad
Accountability: easy to measure, low relevance, high prevalence
Accountability: not difficult to measure, midlevel relevance, medium-level prevalence
Outcome
Defined as the effects and impact of an internationalisation activity
Examples: change in personality, increase in competences, increase in employability, increase in institutional visibility
Accountability: hard to measure but possible, high relevance, low to very low prevalence 12
ï€ The IMPI project
13
What was the IMPI project?
A comprehensive approach to input and output of internationalisation
Partners • Core:
ACA, CampusFrance, CHE Consult (coordinator), NUFFIC, Perspektywy Foundation, SIU
• Associated: Networks of universities (national & European), individual universitieis, NA (DAAD)
Principles: • No ranking tool • No ultimate set of key indicators • Tool for tailored-made profiling and self-assessment, inter-institutional comparison • Inclusion of very broad set of relevant indicators • Changes due to interactivity with user (rating relevance, no. of users) • Globally relevant as based on different global projects & used globally! 14
What was the IMPI project?
15
What was the IMPI project?
16
Geographic distribution of IMPI users by country of HEI
750+ users 58 countries
17
Interest in goal dimensions of internationalisation
0% 10% 20% 30% 40% 50% 60%
1. to enhance the quality of education
53.0%
2. to enhance the quality of research
34.5%
3. to well-prepare students for life and work in an intercultural and globalising world
43.5%
4. to enhance the international reputation and visibility of the unit 5. to provide service to society and community social engagement
45.8%
18.5% 18
The memo© approach
19
The concept of the self CORE PERSONALITY: most difficult to develop
SKILLS
SURFACE: most easily developed
SELFCONCEPT Impact of mobility on the person
TRAIT, MOTIVE
ATTITUDES, VALUES KNOWLEDGE
Impact of change in traits on the person
SOURCE: „The Iceberg Model and Central and Surface Competencies“, Spencer & Spencer, 1993
20
The six memoŠ factors used in EIS
Curiosity
Decisiveness Vigour
Confidence
Serenity
Tolerance of ambiguity
This is what each student saw‌
Relation between memo© and job reality 95% 90%
Fulfilling career
85% 80% 75% 70% 65% 60%
Prosperous career
55% 50% 45% 40%
memo© values
35% Fulfilling career
Prosperous career
individual memo© employability value is closely correlated to one’s job characteristics. the higher the memo© employability, the more satisfied the alumni with aspects such as salary, job security, responsibility, position or creative and challenging tasks.
Principal Component Analysis revealed job attributes to pay together to two larger factors Prosperous career • • • • •
Good career prospects Social recognition and status High income Job security Clear and well-ordered tasks
Fulfilling career • • • • • • • • •
Opportunity of pursuing one´s own ideas Opportunity for creativity and innovation Challenging tasks Opportunity of pursuing continuous learning Largely independent disposition of work Possibilities of using acquired knowledge and skills Co-ordinating and management tasks Chances of doing something useful for the society Opportunity of undertaking scientific/scholarly work
ď€ Results from the Erasmus Impact Study
25
Achieving and accountability: Internationalisation and employability – the Erasmus Impact Study
56 733
18 618
652
964
4 986
higher mobile and mobile and mobile and employers education non-mobile non-mobile non-mobile (mainly institutions staff (academic alumni students SMEs) and nonacademic)
78 891 individual responses in total the largest sample ever
26
Why to measure mobility impact?
Do people change? 74% 72%
Statistically significant gain through mobility
70%
Difference before even going abroad
68% 66% 64% 62% 21
No development without mobility 22 Erasmus PRE
23
24 Erasmus POST
25 Non-mobile
26
Change of personality - perception vs. reality
Students tend to overestimate their learning outcomes: 81% of students perceived an improvement in their personality traits
while only 52% actually attained higher memo factors values
Mobility influences private relationships Life partner of different nationality
32% of mobile alumni and 33% of Erasmus alumni had a life partner of a different nationality. This was nearly three times more than among non-mobile alumni.
Life partner met abroad 24% of mobile alumni and 27% of Erasmus alumni met their life partner while abroad.
ï€ Other impact studies of CHE Consult
31
InHoPe study for the German Federal Ministry (BMBF): The impact of internationalisation of administrative staff
Three year research project app. 1 Mio Euros budget In first two rounds 10,000+ participants Mix of online surveys and webinars / panel
The model
32
Achieving and accountability: The impact of internationalisation of administrative staff ď€ Some major findings so far‌
Major ity is inte (62 %) rested
Minor ity (11 partic %) ipated at curren t HEI Major ity feels b (58 %) ad inform ly ed
33
Achieving and accountability: The impact of internationalisation of administrative staff Major findings so far…
Staff recruitment: • Previous experience abroad and migration background are essential for international mindset • Personality trait (memo©) values are strongly correlated with international attitudes • Thus for positions with substantial international relevance and strong exposure to international clientele, it is more efficient and effective to recruit staff with the dvanced mindset
Staff development: • Very useful for those with less international work environment, lower education and less self-responsibility, development measures can substantially boost an international mind-set • Strongest effect with mobility, followed by intercultural trainings and language courses
Unfortunate reality: • Those who don’t need it, get it; those who need it don’t get and don’t know about it!
European Volunteering Service Evaluation
Assessing the impact of EVS on employability, career, personality, social responsibility, citizenship and European attitudes
Large scale study across Europe, so far approximately 8,000 participants (volunteers and institutions) and growing
Current project, so results cannot be unleashed
35
What really matters?
Impact on personality traits! Look at employability!
36
Thank you for your attention!
If you want to contact me: uwe.brandenburg@che-consult.de
37