For more information about our SciVal suite of products, please contact your nearest Elsevier Regional Sales Office. North, Central and South America
Europe, Middle East and Africa
Asia and Australasia
E-Customer Service Department 3251 Riverport Lane Maryland Heights, MO 63043
E-Customer Service P.O. Box 211 1000 AE Amsterdam, The Netherlands
E-Customer Service 3 Killiney Road #08-01 Winsland House 1 Singapore 239519
Tel: +31 20 485 3767 Fax: +31 20 485 3432 Email: nlinfo@spotlight.scival.com
Tel: +65 6 349 0222 Fax: +65 6 733 1050 Email: sginfo@spotlight.scival.com
Email: usinfo@funding.scival.com Tel: 1 888 615 4500 (+1 314 523 4900, if calling from outside the USA and Canada) South America
Japan
E-Customer Service Rua Sete de Setembro, 111/16 Andar Rio de Janeiro – RJ – 20050-006 – Brazil
E-Customer Service 1-9-15 Higashi-Azabu, Minato-ku Tokyo 106-0044, Japan
Tel: +55 21 3970 9300 Fax: +55 21 2507 1991 Email: brinfo@spotlight.scival.com
© 2010 Elsevier B.V. All Rights Reserved.
Alternative Energy Leadership Study Measuring Performance Through a Multidisciplinary Lens Dr. Kevin W. Boyack Senior Development Advisor and Study Analyst for Academic & Government Products Elsevier B.V.
February 2010
Tel: +81 3 5561 5034 Fax: +81 3 5561 5047 Email: jpinfo@spotlight.scival.com
www.scival.com
www.scival.com
Alternative Energy Leadership Study
Introduction The world relies predominantly on carbon-based energy sources such as oil, gas, and coal to meet its energy needs. Given that these resources are finite and a major cause of pollution, scientists and engineers have sought for decades to develop alternative energy sources. Drawing an accurate picture of how universities and countries are performing with respect to alternative energy-related research is critical for understanding the potential solutions emerging from this science. Without proper insight, academic and government bodies cannot make appropriate funding decisions or develop strategic blueprints that will lead them to the scientific breakthroughs crucial to long-term alternative energy solutions and economic success. Solving today’s most pressing scientific challenges increasingly requires a multidisciplinary approach. Yet, the traditional methods for measuring output no longer capture the reality of how research is being conducted. This study examines alternative energy research from a multidisciplinary perspective, identifying leading institutions (primarily universities) using a new model of science and illustrating how leadership can be masked under current performance measurement systems.
References 1. SciTech Strategies, Inc. Working Paper 2008-01. Klavans, R., & Boyack, K. W. Identifying distinctive competencies in science. 2. Klavans, R., & Boyack, K. W. (in press). Toward an objective, reliable, and accurate method for measuring research leadership. Scientometrics. 3. KLAVANS, R. Taxonomies; International Comparisons & Policy Applications. Visualization Workshop at National Science Foundation (2008).
Dr. Kevin W. Boyack Senior Development Advisor and Study Analyst for Academic & Government Products Elsevier B.V. Kevin Boyack, is a Senior Development Advisor for Elsevier and a recognized authority on multidisciplinary research. His work includes “Visualizing Knowledge Domains www.cs.indiana.edu/~bmarkine/oral/visualizing-knowledge.pdf” published in the Annual Review of Information Science & Technology and his expertise spans information visualization, knowledge domains, science mapping with associated metrics and indicators, network analysis, and the integration and analysis of multiple data types.
Highlights of the proof-of-principle study were shared in a recent webcast:
“ Research Leadership Redefined… Measuring Performance in a Multidisciplinary Landscape.” Over one thousand academic and government executives, researchers, and librarians joined live or viewed the recorded version to learn more about the methodology behind the study which demonstrates a new way to determine leadership.
page_1
page_14
ics
84k paradigms
H Brain h Svc Healt
M e di c al Sp ec ase
page_13
Phy s
is e nf. D h I Tec Bi
Digging beyond total paper counts and breaking out of the traditional discipline boundaries, output in distinctive competencies reveals the degree to which institutions have constructed sub-disciplinary or cross-disciplinary networks within their organization, focused on achieving specific breakthroughs. Measuring output by distinctive competencies more accurately reflects the reality of today’s multidisciplinary scientific landscape, offering a more precise way of determining leadership.
Each of these paradigms contains many research papers, and is centered on a separate and highly specific topic, (e.g. first-order kinetic effects in iron oxide fuel cells) in science. Taken as a whole, these clusters of papers (and thus the SciVal Spotlight model of science) cover almost all of the science that is being published by universities, industry, and government labs today.
Math /
Bi ol
It is also interesting to note that fuel cell research worldwide is much less likely to be conducted in depth with less than a third of total papers (29.7%) in this topic group falling into distinctive competencies as compared to 48.9% of worldwide solar/photovoltaic research. This may signal that countries in general view solar/photovoltaic research in a much more strategic manner than they do fuel cell research and are supporting this research in a way that leads to core competencies in institutions rather than isolated research. At the same time, an even smaller percent (27%) of environmentally-related research is conducted in distinctive competencies.
Responding to the need for a more insightful, multidisciplinary perspective on research performance, Elsevier developed SciVal Spotlight, a performance measurement tool based on a more detailed model of the current structure of science. The model was developed using Scopus, Elsevier’s abstract and citation database of peer reviewed literature and the world’s largest of its kind. The model includes 5.6 million separate research papers published between 2003 and 2007, along with another 2 million of the reference papers that these publications cite heavily. This content was then divided into about 84,000 clusters or paradigms (see Figure 1).
Eng Elect Sci/
Ea rth
In fuel cells and solar/photovoltaic, U.S. leadership is much more diffused with Germany, and to a lesser extent China, emerging as significant players in these two groups. In fact, while Germany’s total number of papers remains lower than the U.S., its percent of papers in distinctive competencies in both solar/photovoltaic and fuel cells is higher. This indicates Germany is at minimum a formidable competitor and possibly an emerging leader in these areas (particularly in solar/ photovoltaic where Germany has 335 papers in DCs and the U.S. has 454).
es nc ie
mp Co
y
In today’s world, research output is often evaluated based on the journal in which it is published. Each scientific journal is classified into a broad field despite the fact that journals are progressively covering a wide array of topics that are not necessarily reflected in their field classification. By its nature, this system only allows for a simplistic view of current research initiatives leaving significant work unaccounted for and overlooked.
og
While the U.S. has the greatest number of institutions ranked in the top 25 based on alternative energy papers in distinctive competencies and is the most prolific in all of the topic groups, the only area in which it has an overwhelming leadership position is in environmentally-related research.
Engineerin g
Background
y istr em Ch
Summary
So c ia lS c
Alternative Energy Leadership Study
Figure 1. The Circle of Science. Approximately 84,000 paradigms are ordered around the perimeter of a circle (for visualization purposes). The model can also be used to identify areas where an institution is a research leader. Researchers at an institution tend to focus their work within a unique set of related paradigms. For any given institution, these paradigms form natural clusters that are based on the networks within the institution. These clusters formulate the unique core competencies or “distinctive competencies (DCs)” of the institution (see Figure 2 overleaf). These are the areas in which the institution is a research leader. One of the unique features of this method is that it can identify those distinctive competencies within an institution that are highly multidisciplinary. Research output mapped toward the center of the circle indicates that it is multidisciplinary and output mapped closer to the perimeter denotes more field-specific work.
page_2
Alternative Energy Leadership Study
Distinctive Competencies (DCs)
New Methodology Uncovers Hidden Leadership – Germany Revealed
indicates significant depth to the work being conducted making it a leader in this area.
The methodology illustrated overleaf can be used at a national level in addition to being used at the institution level. Figure 7 shows the results of an analysis of German competencies using the Spotlight methodology. The Figure indicates a highly diverse set of research strengths in Germany.
In fact, Germany’s total number of papers from all three topic groups is 538 with 467.8 of them published within their DCs, resulting in the country ranking first in all three energy research groupings when measured by the percentage of research in distinctive competencies. This may indicate that Germany has a much more coherent and institutionally focused alternative energy policy as compared to other countries, which makes them a formidable competitor and unrecognized leader. In contrast, Japan’s total number of papers in the three groupings is 1202.31 with only 242.7 of those published in DCs. This substantiates the idea that Japan is opting for breadth over depth in its alternative energy research approach. Again, work in breadth is not necessarily of lesser quality, however major advances tend to come from depth which is indicated by research performed in an institution’s distinctive competencies.
It identifies many distinctive competencies in areas associated with physics, chemistry, and engineering, where much of their alternative energy work is being conducted. On an institutional level, for example, Germany’s Helmholtz Centre Berlin for Materials and Energy is a leader in solar /photovoltaic research. Although the U.S. is the most prolific across all three topic groups, Germany is ahead of the U.S. in solar/photovoltaic research with respect to the percentage of papers published in its distinctive competencies. As mentioned earlier, this
Figure 2. Sample map visually illustrating a university’s distinctive competencies (DCs) within the Circle of Science (see Figure 1 overleaf). The DC highlighted incorporates work from several disciplines including medicine (noted in red), chemistry (noted in blue) and biotechnology (noted in green).
Figure 7: Mapping Germany’s Research Strengths
* Please note that when a country has 0% of its papers in DCs it does not indicate they are not publishing papers in the area, but that they are being developed as isolated pieces rather than part of a larger effort.
page_3
page_12
Alternative Energy Leadership Study
Environmentally-Related Energy Research Leadership North America, led by the U.S., dominates in environmentally -related alternative energy research. The U.S. has close to 2000 total papers in this topic group. While less than half (40%) of U.S. papers are in distinctive competencies, the only other country with any meaningful portion of its research in DCs is Hong Kong at 26.9% and China at 17.7%. However, the total output of these countries is significantly lower (93.9 and 424.6 papers respectively) putting them well behind the U.S. in terms of leadership.
Country
Number of Institutions in the Top 25
This suggests that research into energy efficiency, security, and biomass related science has a low strategic priority in most of the world compared to research based in physics and chemistry. An interesting hypothesis is that because the U.S. consumes the largest percentage of the world’s natural resources, it also takes more responsibility in making breakthroughs in this area. Studies have shown that countries with a lower gross domestic product (GDP) perform more research in the physical sciences, but research tends to shift toward the social sciences (quality of life issues) as GDP increases.
Total Papers
Papers in DCs
% in DCs
United States
17
1997
797
39.9%
China
2
425
75
17.7%
Japan
1
216
0%
S. Korea
1
164
0%
Canada
1
130
0%
Hong Kong
1
94
Switzerland
1
84
0%
72
0%
69
0%
Singapore
45
0%
Italy
44
0%
Great Britian
44
0%
Germany
23
23
100%
Taiwan Netherlands
1
25
26.9%
North America
18
2127
797
37.5%
Asia
5
1016
100
9.9%
Europe
2
264
23
8.6%
3407
920
27%
TOTAL
Study Methodology This proof-of-principle study identifies institutions that are research leaders in alternative energy-related science using the SciVal Spotlight tool. At the time the study analysis was conducted, an institution needed to have at least 100 Scopus-indexed papers per year to be included in the SciVal Spotlight tool. In addition, at the time of the study, only academic institutions, along with a few government laboratories, had been coded. The analysis was thus limited to these institutions. To be considered an alternative energy research leader in the study, an institution first must have substantial activity and impact (i.e. there needs to be a lot of quality research) in the topics associated with alternative energy research. Second, the alternative-energy related work must appear as part of an institution’s distinctive competencies. If the energy-related work does not appear as part of a distinctive competency that does not mean that it is not good work, but rather that it is isolated and not part of a larger network of activity.
Following are the specific steps used to identify the leading institutions (primarily universities) in alternative energy-related research: Step 1: Starting with the 84,000 overall research paradigms, specific search terms were used to identify those that are alternative energy-related. The search terms were culled from various websites dealing with alternative energy, including the White House energy page from the previous Bush administration and the energy-related platform for the current Obama administration. The list of terms used included those illustrated in the table below.
Alternative energy search terms
biodiesel biofuel biomass energy biomass burn biomass pyrolysis bioenergy cellulosic ethanol
energy security energy efficiency clean technology hybrid car hybrid automobile fuel economy lighting efficiency appliance efficiency
co-generation geothermal energy geothermal power hydroelectric power hydroelectric energy ocean thermal
fuel cell nuclear energy nuclear power
alternative fuel alternative energy renewable fuel renewable energy
solar cell solar energy solar power solar installation photovoltaic
wind energy wind power offshore wind
clean coal clean carbon carbon capture carbon sequestration
Figure 6*. Country and region statistics in environmentally-related energy research based on the published output of the top 25 ranked institutions (as listed in Figure 3).
* Please note that when a country has 0% of its papers in DCs it does not indicate they are not publishing papers in the area, but that they are being developed as isolated pieces rather than part of a larger effort.
page_11
page_4
Alternative Energy Leadership Study
Approximately 1,100 energy-related paradigms were identified in which one or more of these terms were highly ranked. These terms were then grouped into 16 topic areas as follows: fuel cell, solar/PV, biodiesel, biofuel, cogeneration, clean coal, efficiency, renewable, biomass, nuclear, wind, carbon, bioenergy, energy security, hydroelectric, and geothermal. Each paradigm was then labelled with one of these 16 topic areas. These topic areas were then grouped into three equally distributed topic groups as follows:
Additionally, as mentioned earlier, the tool only included institutions that publish more than 100 research papers per year (on any topic) as indexed by Scopus. Using only the 1,100 paradigms from the three topic groups listed above (solar/PV, fuel cells, environmentally-related) the numbers of alternative energy research papers for each institution were counted and the institutions were ranked according to output. The top 50 institutions were then taken for further analysis around country and region output in each of the three topic groups.
Topic Group 1: Solar/Photovoltaic Topic Group 2: Fuel Cells Topic Group 3: Environmentally-related (efficiency + renewable + biomass + biodiesel + biofuel + nuclear + wind + cogeneration + clean coal + carbon + bioenergy + security + hydroelectric + geothermal). Step 2: A list was developed including over 3,000 institutions from around the world, most of which are universities and some of which are government laboratories. The list was built using the SciVal Spotlight tool, which did not include industrial/commercial institutions or institutions with an industrial component at the time the analysis was conducted.
Fuel Cell Research Leadership While North America produces significantly more papers than Europe (1225.9 vs. 434.8 respectively), they are about equally as effective at placing their fuel cell research within distinctive competencies at 37% and 36.2% respectively. Asia once again has the largest volume of papers, but the smallest percent in DCs at 22.7%.
Country Step 3: Distinctive competencies for each of the top 50 institutions on the list were identified using the SciVal Spotlight tool. The number of alternative energy related papers that were in distinctive competencies for each university/laboratory were then counted to identify alternative energy leaders. Step 4: The information from the top 50 institutions was aggregated by country and region to generate country and region-specific findings regarding leaders in each of the three alternative energy topic groups.
Number of Institutions in the Top 25
The U.S. leads this topic group with over 1000 total papers as well as the largest number of papers in DCs at 377. However, when it comes to the percent of papers in DCs, Germany is the clear leader at 75.9%. Japan’s performance is surprising in this area as it ranks third in total papers with over 500, but ninth in percent of papers in DCs at 17.7%.
Total Papers
Papers in DCs
% in DCs
United States
6
1006
377
37.5%
China
5
574
157
27.4%
Japan
6
531
94
17.7%
S. Korea
2
296
76
25.6%
Canada
2
220
66
30.1%
Germany
1
145
110
75.9%
Great Britain
1
95
25
26.5%
Netherlands
1
88
25
28.6%
Taiwan Singapore
86 1
76
0% 31
41.3%
Italy
57
0%
Switzerland
49
0%
Hong Kong
18
0%
Asia
14
1581
358
22.7%
North America
8
1226
443
36.2%
Europe
3
435
161
37%
3242
962
29.7%
TOTAL
Figure 5*. Country and region statistics in fuel cell research based on the published output of the top 25 ranked institutions (as listed in Figure 3).
* Please note that when a country has 0% of its papers in DCs it does not indicate they are not publishing papers in the area, but that they are being developed as isolated pieces rather than part of a larger effort.
page_5
page_10
Alternative Energy Leadership Study
Solar/Photovoltaic Research Leadership While Europe has the smallest number of total papers in solar /photovoltaic research (687.6 total papers), it is the most effective at placing that research within distinctive competencies (499.2 papers in DCs representing 72.6% of its research in this topic group). By contrast, Asia has the largest number of total papers, but the smallest percent of papers in DCs (31.2%). Trailing closely behind Asia in total number of papers, North America (913.3) has somewhat greater depth with just under half (49.7%) of its solar /photovoltaic research in DCs.
Country
Number of Institutions in the Top 25
From a national perspective, Germany is a prime example of a country that is focused on depth. While Germany ranks third in total papers (369.9) published in this topic group, the majority (90.5%) of its research is in distinctive competencies. This indicates a leadership role for the country. Although Switzerland and Singapore do not produce a significant number of total papers (53.3 and 38.3 respectively), both publish over 80% of them in DCs. On the other side of the spectrum, China publishes more work in breadth than depth, producing 246.1 papers with only 24% in DCs.
Total Papers
Papers in DCs
% in DCs
United States
8
893
454
50.8%
Japan
5
455
149
32.7%
Germany
2
370
335
90.5%
China
4
246
58
23.7%
Taiwan
2
135
61
45.5%
Great Britain
1
117
53
45.5%
Netherlands
1
79
44
56.0%
S. Korea
76
0%
Italy
1
68
21
31.1%
Switzerland
1
53
46
85.8%
Singapore
38
32
84.7%
Canada
20
0%
Hong Kong
13
0%
Asia
14
963
301
31.2%
North America
8
913
454
49.7%
Europe
3
688
499
72.6%
2564
1254
48.9%
TOTAL
Ranking the Alternative Energy Leaders – Surprising Findings at the Institutional Level Distinctive competencies (DCs) represent expertise in specific research areas. They reveal the degree to which institutions have constructed multidisciplinary networks within their organization focused on achieving specific breakthroughs. Indicating that research within the university is not being done in isolated silos, examining output in distinctive competencies offers a more accurate way of determining leadership in a given area compared to traditional measurement methods. The study shows how distinctive competencies should be taken into account when one identifies research leadership in a specific field such as alternative energy. Using DCs, one can identify institutions with core competencies in alternative energy, as well as the related topics (or complementary research capabilities) that make the alternative energy research at particular institutions highly unique and valuable. The top ranked institutions in the study based on the total number of alternative energy research papers produced in distinctive competencies are listed in Figure 3 overleaf. The chart also includes the number of papers published in distinctive competencies within each of the three alternative energy topic groups.
Nine U.S. institutes are included among the top 25 with NASA’s Goddard Space Flight Center ranking number one. NASA Goddard is renowned for its solar work, yet it ranks first in environmentally-related research in the study. It achieves this ranking in large part because of its biomass work which centers on the environmental effects of burning biomass fuels, as opposed to research on the use of biomass as a fuel or energy source. While some countries view particular alternative energy topics from an energy standpoint, others look at those same topics from an environmental perspective – this suggests a strong link between energy and climate research. Other surprising findings from the study include the fact that the UK has only one institute ranked in the top 25, the Imperial College of London. In fact, this university placed fourteenth behind eight institutes from the U.S., two from Germany, two from China, and one from Japan suggesting that the UK may be lagging behind its major competitors in alternative energy research.
There is a common misperception that the most significant alternative energy-related research is being conducted at only a handful of top-ranked well known institutions. Looking beyond total paper counts affirms that there are institutional leaders in various areas within the alternative energy umbrella within all levels of the university rankings. Examining distinctive competencies shines a light on overlooked output, revealing interesting findings and exposing unrecognized leaders.
Figure 4*. Country and region statistics in solar/photovoltaic research based on the published output of the top 25 ranked institutions (as listed in Figure 3).
* Please note that when a country has 0% of its papers in DCs it does not indicate they are not publishing papers in the area, but that they are being developed as isolated pieces rather than part of a larger effort.
page_9
page_6
Alternative Energy Leadership Study Drilling Down – A Geographic and Topic Group Perspective Institution
Country
Total # of alternative energy papers in DC
Solar/ photovoltaic papers in DCs
Fuel Cell papers in DCs
NASA Goddard Space Flight Center
United States
309
96
National Renewable Energy Laboratory
United States
271
221
Helmholtz Centre Berlin for Materials and Energy
Germany
240
240
Forschungszentrum Julich
Germany
234
Pennsylvania State University
United States
168
National Oceanic and Atmospheric Administration
United States
121
University of California at Irvine
United States
101
Osaka University
Japan
97
97
California Institute of Technology
United States
97
25
Harvard University
United States
84
Tsinghua University
China
83
28
26
Shanghai Jiaotong University
China
81
31
50
University of Colorado
United States
80
Imperial College London
Great Britain
79
53
25
Tohoku University
Japan
71
Eindhoven University of Technology
Netherlands
69
National Research Council of Canada
Canada
66
Nanyang Technological University
Singapore
64
32
National Cheng Kung University
Taiwan
61
61
University of Illinois at Urbana-Champaign
United States
61
Korea Institute of Science and Technology
Korea
54
University of Science and Technology of China
China
54
Massachusetts Institute of Technology
United States
53
Los Alamos National Laboratory
United States
52
Kyoto University
Japan
52
Environmentally Related papers in DCs 213
20
30
94
110
30
37
131 121 101
71 84 29
80
71 44
25 66 31
61
The information from the top 50 institutions was aggregated by country and region to identify worldwide geographic trends in each of the three topic groups within alternative energy research. Summary statistics by country and region were generated for each of the topic groups which are shown overleaf in Figure 4 for solar/photovoltaic and Figure 5 for fuel cells, and on page 11, Figure 6 for environmentally-related. Data for each country and region is solely based on the aggregated research output of the top 50 institutions (as ranked in Figure 3) and includes total papers published in the topic group, number of papers published in distinctive competencies (DCs), and percent of papers in DCs. The total number of papers published indicates the combined breadth and depth of research conducted by the top-ranked institutions in each country, whereas the number and percentage of those papers published in distinctive competencies indicates how much of the overall research is in areas where the institutions have depth. It is natural that all institutions, and therefore the countries and regions in which they are located, have areas of breadth and depth. The typical publication ratio for an institution is approximately four parts breadth to one part depth. Breadth indicates that an institution is examining numerous approaches to achieving scientific breakthroughs, whereas depth (published content in DCs) suggests a more cohesive approach in which larger networks of researchers within the institution are focused on the same research goals.
Figure 3. Top 25 institutions (primarily universities) ranked by total number of alternative energy papers produced in distinctive competencies (DCs).
page_7
page_8