The external review of the Evaluation System for Public Higher Education Institutions in Iceland Reflections and introductory remarks Professor Agneta Yngve, chair of the External Commission
The task and the Commission ď ľ
To evaluate how well the Icelandic ES serves its original purpose to encourage high research productivity and performance, in accordance with international development and the university strategies
ď ľ
To investigate whether the system can be improved and/or extended to better capture the three pillars of universities
ď ľ
To evaluate whether separate frameworks need to be established to capture all three pillars
One American, two Swedes and a Norwegian ‌ Different disciplines, nationalities, experiences
The input
We did our own bibliometric analyses
One for all of Iceland
One for University of Iceland
We analysed the vision/mission statements
We read about QEF2
We met with stakeholders
We discussed the ES from these perspectives
Individual research vs collegiality
Academic freedom
Leadership
Stress and mobility
Qualification portfolios and attractive career paths
Teaching vs Research
Bibliometrics
More scholars publishing over time in international journals
Especially women!
Reduced productivity after 2010 …
Field-adjusted productivity going down
Can have several reasons
Excellent gender balance
Focus on journal quality … but the system does not fully measure quality, so quantity of papers is going up but quality needs to be addressed
The lowest point scores are for recently hired faculty
Journal Impact Factor not a good tool
Why JIF is not a good tool
Top journals will direct research into specific directions
Top journals are not always providing the best audience for all specialities
Top journals often designed to fit the JIF counting method
High cost for authors, long time for peer review – accommodation to specific research agendas and accommodate conventions in science – downplaying innovative work.
Field-adjusted productivity 2006-2014, WoS and all of Iceland
UI based on personnel list
Citation score and top 10% publications
And for university of Iceland ‌
Other comments on bibliometry
The number of publishing staff obviously affects the number of publications
The number and type of publications makes a difference to the number of citations – obviously
Citations increase with time after publication, for a number of years
The number of PhD students obviously influences the number of papers in a system based on published paper dissertations – and also publications increase w time after start
The probability of high-cited papers increases with productivity (Sandström 2016)
An Australian case shows an increased publication rate with a research evaluation system but also increased research quality (Besselaar, 2017)
Base research funding seems to be of fundamental importance for productivity according to a new analysis (Sandström et al 2017, under review), not necessarily competitive project funding …
The report
Recommendations/Scenarios for revising the ES
Additional pictures of the ES
Portfolios are important tools for international careers
In a portfolio, all aspects of performance is noted over time and in line with academic career
In a year-by-year research assessment, this is a separate way of reporting only research
Development for the individual researcher, the department or for the university?
The only way to be assessed?
Influence so immense from this system …
International affairs should probably also be rewarded, not only publications but also from the full portfolio perspective
Level of internationalization can also be assessed, on university level – internationalization index (www.stint.se)