WORKSHOP Rankings, Bibliometrics and their Contribution to Higher Education Management Rectorate Universidade NOVA de Lisboa June 29th 2012 https://www.unl.pt/nova-2011-2012/rankings-and-bibliometrics
P R O G R A M Rankings, Bibliometrics and their Contribution to Higher Education Management 09.15h Opening Prof. João Crespo, Vice-Rector, UNL
09.30h Multidimensional rankings. The U-Multirank project Dr. Gero Federkeil, CHE Centre for Higher Education, Germanyl
10.00h Measuring UNL research: the use and interpretation of bibliometric indicators Dr. Martijn Visser, Center for Science and Technology Studies, Leiden University, The Netherlands
10.30h The scientific production of UNL: evolvement and prospects Prof. João Crespo, Vice-Rector, UNL
11.00h Bibliometric analysis in Social Sciences & Humanities: paradigm shift and mutual challenges Prof. João Costa, Vice-Dean, Faculdade de Ciências Sociais e Humanas, UNL
11.30h Bibliometric analysis and its contribution for university governance Prof. Carlos Costa, former Dean, Faculdade de Engenharia, Universidade do Porto
12.00h Open Discussion 13.00h Closing Prof. António Rendas, Rector, UNL
B i ographical notes of speakers Gero Federkeil is Project Manager at the Centre for Higher Education (CHE), Guetersloh, Germany, and Vice-President of IREG Observatory on Academic Rankings and Excellence since October 2009. At CHE he is responsible for international ranking activities, including the U-Multirank project funded by the European Union. He is an internationally recognized expert in the field of rankings and his publications focus on rankings, performance indicators, benchmarking, quality assurance and employability/university – labour market relations. He is a member of the German Association of Evaluation and a team member of the CHERPA Network. Before joining the CHE in 2000 he worked 7 years for the German Council of Science and Humanities in the field of higher education policy, investments in higher education, evaluation and university medicine. He holds a Master Degree in Sociology (1989) from Bielefeld University. Martijn Visser holds a MA in History from Leiden University. Since 1996 he is affiliated with the Centre of Science and Technology Studies (CWTS) at Leiden University. His main activities at CWTS concerned managing, research assessment and benchmarking studies. In conjunction with that he has been responsible for developing and maintaining the data infrastructure and software routines to facilitate these studies. In the past ten years, he (co-) authored around 20 publications mainly on the development and application of indicators for the measurement research performance. Recently, he changed his role within CWTS and embarked on a research project to study the limitations and potentialities of new data sources for bibliometric studies. João Costa is a Professor of Linguistics at Faculdade de Ciências Sociais e Humanas – UNL, where he currently is Vice-Dean for Research and Human Resources. He received his PhD degree in Linguistics from Leiden University in 1998, and his undergraduate degree from the University of Lisbon in 1994. He was a visiting scholar at MIT, and taught as a visiting professor in universities in Brazil, Macau, The Netherlands, and Spain. His research focuses on syntax and language acquisition in typical and atypical development. He has authored 7 books and around 100 papers (in journals, book chapters and proceedings), integrates the editorial board of 11 journals and book series and he is currently involved in 5 research projects and 3 international networks on syntax, language acquisition and parsing. He coordinated projects related to curriculum and teacher training for the Portuguese Ministry of Education. Carlos Costa retired in 2010 as Professor of Chemical Engineering at Faculdade de Engenharia da Universidade do Porto (FEUP) and assumed the Presidency of Tâmega Entrepreneurship Institute and become non-executive administrator of RIMA, S.A. In 2011 he also assumed the direction of TER (Association for Innovation and Development of Renewable Energies). During 9 years (2001-2010) he was FEUP director and member of the General Councils of the UPorto Business School and of the Gomes Teixeira Foundation. Before (1995–2001) he was director of FEUP Chemical Engineering Department and founder and coordinator of LEPAE (Laboratory for Process, Environment and Energy Engineering) at FEUP. In all these management activities he always integrated the use of quantitative methods and feedback control to help better decision making and performance.
F O R E W O R D The goal of this workshop is to promote the discussion about the role of bibliometrics and university rankings in higher education management and university governance. Universities are increasingly confronted by ranking and classification initiatives – both at the national and international level. Despite all objections about the methodologies and criteria of rankings, there is a growing recognition that such initiatives are here to stay. A particular focus will be given to U-Multirank, the EU sponsored project for the development of a ranking that considers not only research and bibliometrics but also other dimensions of higher education institutions, from teaching/learning to international profile and knowledge transfer.
This workshop will also present and discuss the results of the third bibliometric study produced for NOVA by the Centre for Science and Technology Studies of Leiden University. This study analyses the scientific production of NOVA in different areas and subfields and compares the impact and output of NOVA with reference universities.
This study shows that the normalized impact of publications of NOVA, indexed to Web of Science, has improved consistently since 2000 and reached a value similar to the international average. The number of highly-cited papers has also increased significantly. This study identifies scientific areas with significant production and impact close to, or above, international average such as Biological Sciences - Animals & Plants, Chemistry, Geosciences, Applied Physics & Chemistry, Clinical Medicine, Biological Sciences – Humans, and Engineering. As far as scientific subfields are concerned, the impact of NOVA is above the international average in Applied Physics, Materials Science Ceramics, Food Science & Technology, Plant Sciences, Physical Chemistry, Electrical & Electronic Engineering and Multidisciplinar Chemistry. A group of 35 universities (from Portugal or other European countries) was selected for a benchmarking study. The impact of NOVA is significantly above the impact of Portuguese universities in the following main scientific areas: Biological Sciences Animals & Plants, Chemistry, Economics, Psychology, Psychiatry & Behavioral Sci and Humanities & Arts. Concerning subfields, the impact of NOVA is higher than that of the 35 benchmark universities in Applied Physics, Materials Science Ceramics, Chemical Engineering and Electrical & Electronic Engineering. This event is designed to be an informal workshop, open not only to NOVA but also to the national scientific community at large, aiming at discussing the role of rankings (bibliometric and multidimensional) in the management of universities. 13.00h
Update of the bibliometric study of NOVA (publications indexed to Web of Science 2004-2010)
RESULTS
Benchmark NOVA with other Portuguese and European Universities
Identify areas of excellence, strengths, and weaknesses
Assess the impact of the publications (articles, letters, reviews) in journals indexed to WoS and compare with the results of the previous studies (2000-06 & 2002-08)
Main objectives:
Solid conclusions can be withdrawn for the majority of the Natural and Exact Sciences, where the coverage of Web of Science is excellent and the publication in periodicals of international circulation is the most common way of dissemination of results
Study carried out by the Centre for Science and Technology Studies (CWTS), Leiden University, the Netherlands
Bibliometric analysis
3350 3995 (+19%)* 4671 (+39%)*
2000-2006
2002-2008
2004-2010
0.98 (+13%)*
0.95 (+9%)*
0.87
Impact CPP/FCSm
1.02 (+7%)*
1.0 (+5%)*
0.95
Impact MNCS
If CPP/FCSm or MNCS values are between 0.8-1.2 the impact is similar to the international average. A value >1.2 means that the impact is above average
MNCS, Mean Normalized Citation Score – New indicator that replaced CPP/FCSm in 2010. It is also a normalized indicator, but the calculation method is different
citations excluding self-citations (CPP) obtained by a research unit to the international reference value (FCSm, the mean field(s) citation score). It is a normalized indicator that corrects for differences in “citation habits” among scientific areas
CPP/FCSm, Field Normalized Citation Impact - this indicator compares the average number of
* percentages are relative to 2000-2006
Production
Publication period
Production and normalized impact
0.66
0.72
0.75
0.82
0.82 0.80
0.88 0.87
0.89
1.00
1.03
1.05
The impact of NOVA publications has been increasing and reached 1.05 in 2007-2010
0
0.2
0.4
0.6
0.8
1
1.2
Evolution of Impact
Normalized impact (CPP/FCSm)
0
100
200
300
400
500
600
154
Top20%
559
57
Top10%
28 Top5%
133
3 Top1%
35
2004-2008**
2002-2005
2001-2004
2000-2003
1999-2002
1998-2001
1997-2000
** 2004-2008 papers counted in 5 years; all other data corresponds to 4 years
283
Highly-cited papers (2004-2008)
Collaboration
Arrows identify areas with significant output (>200) and impact close / above international average: Biological Sciences-Animals & Plants (1.51), Chemistry (1.17), Geosciences (1.07), Applied Physics & Chemistry (1.03), Clinical Medicine (0.99), Biological Sciences-Humans (0.97), Engineering (0.94)
Scientific areas of highest impact and production
Subfields of highest impact and production
“ Chemical Engineering”, “Microbiology” and several areas of “Chemistry” have both high impact and significant output
The impact is above international average (>1.2) for 8 subfields: Applied Physics; Material Sci Ceramics; Chemical Eng; Food Sci & Technol; Plant Sci; Physical Chemistry; Elect & Electr Eng; Multidisc Chem
Approximately 50% of publications result from international collaboration
The number of highly cited papers has been increasing consistently
The field normalized citation impact of NOVA, CPP/FCSm, increased 13% from 0.87 in 2000-2006 to 0.98 in 2004-2010. The Mean Normalized Citation Score, MNCS, increased from 0.95 to 1.02
The number of publications increased 39% from 3350 in 2000-2006 to 4671 in 2004-2010
Conclusions
Benchmarking
Austria Technical Univ Wien
Germany Stuttgart Technical Univ Dresden
Belgium Katholieque Univ Leuven Libre Bruxelles Switzerland ETH Zurich
Portugal Porto Lisboa Técnica de Lisboa Coimbra Aveiro Minho Algarve
Ireland Dublin City Univ United kingdom Bath Southampton Leeds
Italy Perugia Poland Krakov
France Toulouse 3 Paul Sabatier Paris 5 René Descartes
Spain Granada Sevilha Autónoma Barcelona
Greece Aristotle Univ Thessaloniki
Benchmark universities
Czech Republic Masaryk Univ Brno
Norway Bergen Norwegian Univ of Sci & Technol Trondheim
The Netherlands Utrecht Twente
Denmark Aarhus Univ of Southern Denmark
Finland Helsinki Turku Abo Akademi Univ
NOVA vs 35 benchmark universities: subfields with highest impact at NOVA
* Differences in impact higher than 0.15
Applied Physics Mat Sci, Ceramics Chemical Eng Elec & Electr Eng Physical Chemistry
The impact of UNL is significantly above* the benchmark universities in:
* Differences in impact higher than 0.15
Applied Physics Applied Physics Mat Sci, Ceramics Chemical Eng Elec & Electr Eng Plant Sciences Physical Chemistry Multidisc Chemistry Parasitology Microbiology Organic Chemistry Applied Mathematics
The impact of UNL is significantly above* the impact of Portugal in:
NOVA vs 35 benchmark universities: subfields with highest impact at NOVA
universities that produce more than 700 publications per year indexed to Web of Science
Position of NOVA among 365 top world universities
Ranking
↑ ↑ ↑
Engineering - 265
Biol Sci Humans - 269
↑ ↓
Multidisciplinary Sci - 259 ↑
Applied Physics & Chemistry - 249
Social Sci Medicine - 242
Other Social Sci - 235
Geosciences - 226
Chemistry - 221
↑
Economics - Ranked 189
World Top176-270 in 9 areas:
↑
World Top126-150 Humanities & Arts - Ranked 140
World Top51-75 Biol Sci Animals & Plants - Ranked 66
Rank in 2004-2010 and change in position compared to 2000-2006
Ranking based on impact (2004-2010) and comparison with 2000-2006