Excellence Mapping: Bibliometric study of the productivity and impact of scientific publications of JRC

Transcription

1 EUROPEAN COMMISSION JOINT RESEARCH CENTRE Policy Support Coordination Planning, Evaluation and Knowledge Management Unit Excellence Mapping: Bibliometric study of the productivity and impact of scientific publications of JRC Mapping of scientific areas and application areas Volume 2 Analysis of the JRC collaborations with world institutions Report prepared by: Mihaela Bobeica, Guy Bordin, Grazia Federico, Mayya Hristova and Vera Calenbuhr; JRC.A2 Planning, Evaluation and Knowledge Management October 2014 Page 1

2 Table of Contents 1 Executive summary Introduction Methodological notes Collaborations analysis JRC collaborations analysis using pre-defined lists of comparators Overall Scival statistics on joint publications produced by the JRC with world institutions, in the period Generic information on the rankings used for the analysis Analysis of the ranked institutions having formal agreements with the JRC Analysis of the ranked institutions having produced joint publications with the JRC in the period Conclusions Sources Literature Annex 1: Institutions included in the three analysed rankings Annex 2: General JRC joint publications information, by scientific area (level 2 and level 3) Annex 3: Methodological comparison of the 'Thomson Reuters'- report, the 'JRC collaborations with universities from EU-28 Member States'-report and the 'Excellence Mapping Vol.2' Annex 4: Main findings of the 'Thomson Reuters'- report, the 'JRC collaborations with universities from EU-28 Member States'-report and the 'Excellence Mapping Vol. 2' Annex 5: Analysis originally foreseen for analysing JRC collaborations with the "best" institutions in the world Page 2

3 1 Executive summary The present report is the second volume of the JRC's excellence mapping: while the first volume concentrated on the productivity and impact of JRC publications in general, the present volume analyses the sub-set of publications jointly produced with scientists from other organizations. It does so by means of 'publications-' and 'citations-' analysis in order to identify and map areas of excellence. Main findings include: 71% of all JRC publication published in have been co-authored with other world institutions; JRC scientists co-authored publications with institutions during ; JRC scientists co-authored publications with 87% of the institutions ranked Top 100 in three world university rankings. Over one third of these institutions are found amongst the Top 15 in the world in terms of numbers of citations in different scientific areas; The JRC has formal agreements with 45% of the institutions ranked Top 100 in three world university rankings. 96% of these institutions produced joint publications with the JRC; 38% of the institutions that have a formal agreement with the JRC and that have produced joint publications with the JRC are ranked amongst the Top 15 in terms of numbers of publications in at least one of the scientific areas where collaborations occur. Page 3

4 2 Introduction The present report is the second volume of the JRC's excellence mapping: while the first volume concentrated on the productivity and impact of JRC publications in general, the present volume analyses the subset of publications jointly produced with scientists from other organizations. It does so by means of 'publications-' and 'citations-' analysis in order to identify and map areas of excellence. The excellence mapping is part of a wider effort to produce the evidence base for the following purposes: - the ex-post evaluation of the Framework Programme (FP) 7 (both nuclear and non-nuclear); - the strategic work programme planning (e.g. input for ex-ante evaluation thereby closing the annual planning, reporting and evaluation cycle) ; - the design of a long term JRC scientific strategy The report complements an internal publication collaboration 1 study carried out in 2013 as well as a bibliometric study of JRC results carried out by Thomson Reuters 2 in 2014 and it analyses and benchmarks JRC collaborations in the context world-wide university rankings. The current analysis provides the first elements of the evidence base to answer the following questions: 1. Which are the "best" institutions, i.e. the institutions with which the JRC should develop collaborating strategies, partnerships etc. (cf. the DG mind map "World-class", section "Work with the best"). 2. Is the JRC collaborating with the "best" institutions? How do the current JRC collaborators rank compared to peer institutions in the world? To this end, the analysis will also correlate collaborations analysis data and JRC information on collaboration agreements. 3 Methodological notes The present excellence mapping volume 2 focuses on the scientific excellence dimensions of publication productivity of the scientific collaborations of the Joint Research Centre. In essence, the present collaboration analysis addresses the question to what degree the JRC works with the best organisations world-wide, in particular, in the chapters presented hereafter, with organisations that rank Top 100 in some world rankings. To this end we analyse scientific publications co-authored by JRC researchers and authors of other institutions world-wide. The principal data sources for the excellence mapping are Elsevier's Scopus database and the associated analytical tool SciVal. Scopus is the largest available citations and abstract database of peer-reviewed scientific literature. 1 JRC collaborations with universities from EU-28 Member States at the level of co-authored scientific peer reviewed articles..european Commission Joint Research Centre Evaluation of the Research Performance of the Joint Research Centre of the European Commission during the 7th Framework Programme ( ). Thomson Reuter, 2014 Page 4

5 In this report, the analysis is performed for the JRC publications as a whole, as well as broken down according to scientific areas of the 2nd and 3 rd levels. Further details can be found in the methodological notes of Volume 1 of the excellence mapping. In order to benchmark JRC's scientific collaboration, the joint publications of the JRC with other organizations are analysed in the context of three world-wide university rankings: 1. Times Higher Education Ranking (THES) 2. QS World University Ranking (QS) 3. Academic Ranking of World Universities (ARWU) The focus on these university rankings represents a fall back option for the analysis of JRC collaborations, since certain functionalities of Scopus/SciVal for creating sub-populations of publications were unavailable during several weeks, which forced the author team to conceive and pursue an alternative methodology. The methodology that was foreseen originally would involve benchmarking in the context of the Top-15 organizations regarding selected bibliometric citation indicators similar to the method applied in volume 1 of the excellence mapping. The fall back option also provides interesting insights. Nevertheless, in the interest of analysing and mapping JRC's scientific excellence according to a one homogeneous methodology, it would be desirable to complete the excellence mapping in general and volume 2 in particular using the original methodology, once the Scopus/SciVal delivers again all functions. The original methodology is described in Annex 5. The bibliometric study performed by Thomson Reuters (see introduction) used the Thomson Reuters database underlying the Thomson Reuters Web of Science research platform - one of the largest scientific databases. Most publications including the JRC's are present in both systems, i.e. Web of Science and Scopus/SciVal. Yet, the thematic structure of the information is different. The complementarities between the present excellence mapping and the Thomson Reuters report are discussed in Annex 3. This annex also contains a methodological comparison with the collaborations study in the context EU28. Moreover, Annex 4 presents the main findings of all three studies regarding collaborations. Data extraction: Data for the analysis have been extracted from Scopus database and SciVal (accessible from and during the months of June-September The analysis covers publications and citations during the period resulting in a total of almost 5000 publications. This time window covers five out of seven years of the duration of FP7, and its use is due to the fact that Elsevier's analytical tool SciVal provides the required citation information as well as certain statistical tools and indicators only for the abovementioned period. Further methodological information will be provided in the respective chapters below. 4 Collaborations analysis 4.1 JRC collaborations analysis using pre-defined lists of comparators This chapter presents results of the analysis of the collaborations, in particular joint publications, between the JRC and some world universities highly ranked in different world rankings. Page 5

6 In the context of this analysis, three different rankings have been used and the Top 100 institutions in each of them have been analysed, in order to find out: The existence of a formal agreement with the JRC The existence of publications co-authored by the JRC with these institutions The numbers and proportions of joint publications The Scopus scientific areas (level 2) corresponding to these joint publications The world rank of these collaborating institutions in terms of the overall number of citations and also their rank in terms of numbers of joint publications with the JRC. The analysis of the joint publications produced by the JRC in collaboration with institutions included in these three rankings is based on publication data extracted using the SciVal analytical tool. The publications considered cover the period Overall Scival statistics on joint publications produced by the JRC with world institutions, in the period This section presents some overall statistics on the publications produced by the JRC in the period and on the subset of publications jointly produced by the JRC with scientific institutions of the world. More generic information on JRC joint publications and collaborators, detailed by scientific area (level 2) and subarea (level 3), can be found in Annex 2. Total number of publications published by the JRC in the period , all scientific areas Total number of co-authored publications (all scientific areas) 3523 Total number of collaborating institutions 1328 Proportion of co-authored publications 71% Table 1. Overall Scival statistics on joint publications produced by the JRC with world institutions, in all scientific areas ( ) Generic information on the rankings used for the analysis Three different rankings have been used for the present analysis: 4. Times Higher Education Ranking (THES) 5. QS World University Ranking (QS) 6. Academic Ranking of World Universities (ARWU) 3 See Volume 1 of the "Excellence mapping" study for details about Scopus/SciVal methodology and scientific areas, the covered publication period, and publication types. 4 This value is slightly different from the value used in the Volume 1 of this study: this difference is explained by the fact that the preparation of the data, including its extraction from the source system (Scopus/SciVal) has been done over several months. In the meantime, the publication numbers corresponding to the JRC have been slightly changed in the source database. Page 6

7 Each of these three rankings has a generic version and a version by domain, ie. life sciences, engineering, social sciences etc. For the current analysis, only the generic version of each of the three rankings has been used and the Top 100 institutions in each of them have been scrutinized. It should be noted that 52 institutions are common to all three rankings. Some world renowned institutions appear amongst the Top 5 to Top 10 institutions in all three rankings (Table 2): Harvard University Institution name Ranking Ranking position Massachusetts Institute of Technology (MIT) Stanford University University of Cambridge University of Oxford Table 2. World institutions highly ranked in the three analysed world rankings (THES, QS and ARWU) - 43 institutions appear in two of the three analysed rankings; - 58 institutions appear in only one of the three rankings. ARWU 1 THES 2 QS 4 QS 1 ARWU 3 THES 5 ARWU 2 THES 4 QS 7 QS 2 ARWU 5 THES 7 THES 2 QS 5 ARWU 9 The three rankings together contain 153 unique institutions. The list of the institutions included in the three rankings and their respective ranking positions can be found in Annex Analysis of the ranked institutions having formal agreements with the JRC This section analyses the institutions included in the three rankings and which have concluded formal agreements with the JRC. Based on the data extracted from two JIPSY exports 5 (dated June and October 2014) listing scientific bodies with which the JRC has agreements, the formal agreements between the JRC and institutions included in the three rankings analysed in the context of the present report are of three types: - collaboration agreement 6 - competitive activity 7 - institutional network. 5 These exports do not provide clear information on the overall number of agreements that the JRC has concluded with world institutions. 6 A formal Collaboration Agreement (CA) shall be signed when the JRC wishes to undertake specific collaboration activities - such as joint activities and projects - with an external partner. The purpose of establishing a CA is to define the content and modalities of the intended activities or projects. 7 In the commission terminology," competitive activity" refers to administrative arrangements, FP7 indirect actions and to third party work. Page 7

8 45% of the institutions in the three rankings have a formal agreement with the JRC (68 institutions of 153 in total). Fifteen of 68 institutions have more than one type of formal agreement with the JRC. The number and share of institutions by type of formal agreement with the JRC are depicted in Figure 1. Figure 1. Number and share of institutions by type of agreement with the JRC 96% of the institutions that have a formal agreement with the JRC have produced joint publications with the JRC in at least one scientific area. Only three ranked institutions have formal agreements with the JRC but no joint publications yet (Ludwig-Maximilians-Universität München (Germany); Ruprecht-Karls- Universität Heidelberg (Germany) and Vrije Universiteit Amsterdam (The Netherlands)). 38% of the institutions having a formal agreement with the JRC and having produced joint publications with the JRC are ranked amongst the Top 15 in terms of numbers of publications in at least one of the scientific areas where collaborations occur Analysis of the ranked institutions having produced joint publications with the JRC in the period This section looks at the institutions included in the three analysed rankings and in particular the institutions that have co-published papers with the JRC in the reporting period ( ). The information related to numbers and shares of joint publications, numbers of ranked JRC collaborating institutions and their rankings in terms of numbers of citations are presented here below by scientific area, level Generic SciVal statistics on joint publications produced by the JRC with the institutions in the three analysed rankings Table 2 below gathers the generic Scopus/SciVal statistics on joint publications between the JRC and ranked institutions. Page 8

9 Raw sum of co-authored publications (ie. collaborations) with the 133 institutions in the 3 analysed rankings Estimated number of unique publications produced in collaboration with institutions in the three analysed rankings Total number of collaborating institutions in the three analysed rankings 133 Proportion of JRC publications co-authored with the 133 institutions in the three analysed rankings of the total number of publications produced by the JRC in the reporting period Proportion of JRC publications co-authored with the 133 institutions in the three analysed rankings of the total number of co-authored publications produced by the JRC in the reporting period Table 2. Overall SciVal statistics on joint publications produced by the JRC with the institutions in three analysed world rankings (THES, QS and ARWU) % 22% It is important to note that 87% of the institutions in the three rankings have produced joint publications with the JRC (133 institutions of 153 in total), whilst the vast majority (96%) of the 52 institutions that appear in all three rankings have produced joint publications with the JRC Analysis of collaborations by SciVal scientific area, level 2 Figure 2 displays the distribution of the JRC collaborators included in the three world rankings (THES, QS and ARWU), by Scopus/SciVal scientific area, level 2. A collaborator is an institution included in the three world rankings and which has collaboration with the JRC. 8 This estimation was calculated based on the average number of institutions per co-authored publication. This number has been calculated using JRC collaboration information (ie. numbers of joint publications and numbers of collaborating institutions), in all level 2 areas. Page 9

10 Figure 2. Number of JRC collaborators included in the three analysed world rankings (THES, QS and ARWU), by SciVal/Scopus scientific area, level 2 Table 3 below presents the JRC collaborators (included in the three analysed rankings) which have the highest numbers of joint publications with the JRC, by scientific area, level 2. Annex 1 presents the overall number of joint publications with the JRC, produced by each ranked institution, in all scientific areas taken together ( ). Page 10

11 General Physical sciences: Earth and Planetary Sciences Columbia University Lund University ETH Zurich (Swiss Federal Institute of Technology) Wageningen University and Research Center Harvard University Physical sciences: Energy McMaster University Delft University of Technology University of Edinburgh Physical sciences: Engineering Wageningen University and Research Center Imperial College London Health sciences: Health Professions Physical sciences: Environmental Science Johns Hopkins University ETH Zurich (Swiss Federal Institute of Technology) Health sciences: Medicine Wageningen University and Research Center Columbia University Physical sciences: Materials Science Harvard University Delft University of Technology Johns Hopkins University Imperial College London Lund University Social sciences: Arts and Humanities University of Munich Columbia University Wageningen University and Research Center Duke University Life sciences: Agricultural and Biological Sciences Monash University Lund University University of Bristol Wageningen University and Research Center University of Edinburgh Life sciences: Biochemistry, Genetics and Molecular Biology University of Leeds Ghent University University of Melbourne Lund University Social sciences: Business, Management and Accounting Wageningen University and Research Center University of Manchester Life sciences: Immunology and Microbiology Utrecht University Wageningen University and Research Center Social sciences: Decision Sciences Life sciences: Neuroscience Lund University Johns Hopkins University Social sciences: Economics, Econometrics and Finance University of Bonn Ecole Polytechnique University of Munich University of Copenhagen University of Zurich University of Groningen Life sciences: Pharmacology, Toxicology and Pharmaceutics University of Manchester Johns Hopkins University Wageningen University and Research Center Wageningen University and Research Center Social sciences: Social Sciences Physical sciences: Chemical Engineering Delft University of Technology Imperial College London University of Leeds Physical sciences: Computer Science Wageningen University and Research Center Wageningen University and Research Center Table 3. Top JRC collaborators included in the three analysed world rankings (THES, QS and ARWU), by scientific area, level 2 Over one third of the 133 collaborating institutions (in the three analysed rankings) are found amongst the Top 15 in the world in terms of numbers of citations in different scientific areas (Table 4 below). 38% of all collaborations are with the 52 institutions that appear in all three rankings. Page 11

12 SciVal scientific areas, level 2 Number of JRC collaborators ranked Top 15 in the world, in terms of number of citations Social sciences: Arts and Humanities 12 Physical sciences: Mathematics 11 Social sciences: Psychology 11 Social sciences: Social Sciences 11 General 10 Health sciences: Health Professions 10 Health sciences: Medicine 10 Life sciences: Neuroscience 10 Physical sciences: Engineering 10 Physical sciences: Materials Science 10 Social sciences: Business, Management and Accounting 10 Social sciences: Decision Sciences 10 Social sciences: Economics, Econometrics and Finance 10 Health sciences: Nursing 9 Life sciences: Biochemistry, Genetics and Molecular Biology 9 Life sciences: Immunology and Microbiology 9 Physical sciences: Chemical Engineering 9 Physical sciences: Computer Science 9 Life sciences: Agricultural and Biological Sciences 8 Physical sciences: Chemistry 8 Physical sciences: Physics and Astronomy 8 Health sciences: Veterinary 7 Life sciences: Pharmacology, Toxicology and Pharmaceutics 7 Physical sciences: Earth and Planetary Sciences 6 Physical sciences: Environmental Science 6 Physical sciences: Energy 5 Table 4. Number of JRC collaborators present in three analysed world rankings (THES, QS and ARWU) that are ranked Top 15 in the world, in terms of numbers of citations, by scientific area, level Conclusions The analysis presented in the sections above describes the very first elements to characterize the current scientific collaborations of the JRC. This analysis could be interestingly complemented by a wider study of current and potential JRC collaborators. A proposal for a complementary study of JRC collaborations has been shortly described in Annex 5 below. Page 12

13 5 Sources Scopus/SciVal database & analytic tools. 6 Literature -' JRC collaborations with universities from EU-28 Member States at the level of co-authored scientific peer reviewed articles'. European Commission Joint Research Centre 'Evaluation of the Research Performance of the Joint Research Centre of the European Commission during the 7th Framework Programme ( )'. Thomson Reuter, 'Excellence Mapping: Bibliometric study of the productivity and the impact of scientific publications of the JRC. Mapping of scientific areas and application areas - Volume 1 General analysis and benchmarking'. European Commission Joint Research Centre Page 13

14 7 Annex 1: Institutions included in the three analysed rankings Institution name THES ranking position QS ranking position ARWU ranking position Total number of co-authored publications (all areas) Source: SciVal Australian National University Boston University Brown University California Institute of Technology Carnegie Mellon University Columbia University Cornell University Duke University École Normale Supérieure ETH Zurich (Swiss Federal Institute of Technology) Harvard University Imperial College London Johns Hopkins University King's College London KU Leuven Kyoto University Leiden University Massachusetts Institute of Technology (MIT) McGill University New York University Northwestern University Princeton University Stanford University Technical University Munich University of Manchester University of Melbourne The University of Tokyo University College London University of Bristol University of British Columbia University of California, Berkeley University of California, Davis University of California, Los Angeles University of California, San Diego University of Cambridge University of Chicago University of Edinburgh University of Groningen University of Helsinki University of Illinois at Urbana Champaign University of Michigan University of North Carolina at Chapel Hill University of Oxford University of Pennsylvania University of Queensland University of Texas at Austin University of Toronto University of Washington University of Wisconsin-Madison Utrecht University Washington University in St. Louis Yale University Table 5. List of the 52 institutions common to the three analysed world rankings, their respective ranking position and the number of publications by scientific area Page 14

15 Institution name THES ranking position QS ranking position ARWU ranking position Total number of co-authored publications (all areas) Source: SciVal Aarhus University Delft University of Technology Durham University Ecole Polytechnique École Polytechnique Fédérale de Lausanne Erasmus University Rotterdam Georgia Institute of Technology Ghent University Heidelberg University KAIST - Korea Advanced Institute of Science & Technology Karolinska Institute London School of Economics and Political Science (LSE) Ludwig-Maximilians-Universität München McMaster University Monash University Nanyang Technological University National University of Singapore (NUS) Ohio State University Osaka University Peking University Pennsylvania State University Pohang University of Science and Technology (Postech) Purdue University Rice University Seoul National University The University of Western Australia Tsinghua University Université Pierre et Marie Curie University of Amsterdam University of Basel University of California, Irvine University of California, Santa Barbara University of Colorado at Boulder University of Copenhagen University of Geneva University of Hong Kong University of Minnesota University of Pittsburgh University of Rochester University of Southern California University of Zurich Uppsala University Vanderbilt University Table 6. List of the 43 institutions that appear in 2 of the three analysed world rankings, their respective ranking position and the number of publications by scientific area Page 15

16 Institution name THES ranking position QS ranking position ARWU ranking position Total number of co-authored publications (all areas) Source: SciVal Arizona State University 88 5 Case Western Reserve University 88 Emory University 80 2 Freie Universität Berlin 86 9 Fudan University 71 3 Georg-August-Universität Göttingen 63 Hebrew University of Jerusalem 70 8 Hong Kong University of Science and Technology 57 Humboldt-Universität zu Berlin 94 6 Lund University Maastricht University 98 3 Michigan State University Moscow State University 84 8 National Taiwan University (NTU) 76 2 Queen Mary University of London (QMUL) 96 5 Rockefeller University 33 Ruprecht-Karls-Universität Heidelberg 49 Rutgers, The State University of New Jersey - New Brunswick 52 7 Stockholm University Swiss Federal Institute of Technology Lausanne 96 Technion-Israel Institute of Technology 78 1 Texas A and M University 96 8 The Chinese University of Hong Kong 46 2 The Hong Kong University of Science and Technology The University of Adelaide 100 The University of Auckland 92 4 The University of New South Wales The University of Nottingham The University of Sheffield The University of Sydney 38 4 The University of Texas Southwestern Medical 45 Center at Dallas The University of Warwick 61 2 Table 7. List of the 58 institutions (1/2) that appear in only one of the three analysed world rankings, their respective ranking position and the number of publications by scientific area 40 Page 16

17 Institution name THES ranking position QS ranking position Tohoku University 73 ARWU ranking position Total number of co-authored publications (all areas) Source: SciVal Tokyo Institute of Technology 68 2 Trinity College Dublin 74 7 Tufts University 80 Université de Montréal 83 3 University of Alberta 84 7 University of Arizona University of Birmingham University of Bonn University of California, San Francisco University of California, Santa Cruz University of Florida University of Glasgow University of Leeds University of Maryland, College Park University of Munich 49 2 University of Notre Dame University of Oslo University of Paris Sud (Paris 11) University of Southampton 94 4 University of St Andrews 88 University of Strasbourg University of Sydney 72 4 University of Utah 87 VU University Amsterdam 100 Wageningen University and Research Center Table 7. List of the 58 institutions (2/2) that appear in only one of the three analysed world rankings, their respective ranking position and the number of publications by scientific area Page 17

18 8 Annex 2: General JRC joint publications information, by scientific area (level 2 and level 3) SciVal area (level 2) / sub-area (level 3) Environmental science Total publications Co-authored publications Proportion of co-authored publications Number of collaborators % 728 General environmental science % 401 Waste Management and Disposal % 147 Environmental chemistry % 291 Management, monitoring, policy and law % 163 Pollution % 206 Health, toxicology and mutagenesis % 199 Ecology % 217 Water Science and Technology % 135 Ecological modelling % 61 Physics and astronomy % 523 Nuclear and high energy physics % 230 Condensed matter physics % 281 Instrumentation % 146 Atomic and molecular physics, and optics % 177 Radiation % 113 Engineering % 427 Electrical and Electronic Engineering % 196 Safety, Risk, Reliability and Quality % 109 Mechanical Engineering % 119 Mechanics of Materials % 79 Industrial and Manufacturing Engineering % 54 Civil and Structural Engineering % 58 Earth and Planetary Sciences % 623 Atmospheric Science % 419 General Earth and Planetary Sciences % 218 Computers in Earth Sciences % 103 Earth and Planetary Sciences (miscellaneous) 74 Oceanography % 125 Geophysics % 143 Geology % 96 Space and Planetary Science % 132 Geotechnical Engineering and Engineering Geology % 34 Agricultural and Biological Sciences Energy % 538 Aquatic Science % 216 Ecology, Evolution, Behavior and Systematics % 259 Food Science % 77 Agronomy and Crop Science % 137 Soil Science % 163 Forestry % % 340 Nuclear Energy and Engineering % 172 Energy Engineering and Power Technology % 95 Renewable Energy, Sustainability and the Environment % 181 General Energy % 57 Fuel Technology % 65 Chemistry % 392 General Chemistry % 216 Analytical Chemistry % 171 Physical and Theoretical Chemistry % 102 Spectroscopy % 120 Organic Chemistry % 37 Inorganic Chemistry % 38 JRC Page 18

19 SciVal area (level 2) / sub-area (level 3) Computer Science Total publications Co-authored publications Decision Sciences Immunology and Microbiology Multidisciplinary Health Professions Psychology Arts and Humanities Neuroscience Veterinary Nursing Table 8. JRC joint publications information, by scientific area (level 2 and level 3) Proportion of co-authored publications Number of collaborators % 292 Computer Science Applications % 167 Computer Networks and Communications % 97 Software % 78 General Computer Science % 47 Information Systems % 52 Materials Science % 324 General Materials Science % 201 Electronic, Optical and Magnetic Materials % 164 Materials Chemistry % 87 Surfaces, Coatings and Films % 67 Social Sciences % 286 Geography, Planning and Development % 161 General Social Sciences % 60 Sociology and Political Science % 28 Development % 50 Law % 36 Library and Information Sciences % 17 Safety Research % 16 Biochemistry, Genetics and Molecular Biology % 392 Biochemistry % 127 General Biochemistry,Genetics and Molecular Biology % 163 Biotechnology % 63 Cancer Research % 15 Medicine % 478 Public Health, Environmental and Occupational Health % 207 General Medicine % 269 Radiology, Nuclear Medicine and Imaging % 71 Biochemistry (medical) % 41 Pharmacology, Toxicology and Pharmaceutics % 260 Toxicology % 213 Mathematics % 231 Applied Mathematics % 115 Modeling and Simulation % 65 Theoretical Computer Science % 31 Computational Mathematics % 26 Statistics and Probability % 24 Chemical Engineering % 152 General Chemical Engineering % 85 Bioengineering % 58 Economics, Econometrics and Finance % 103 Economics and Econometrics % 97 Business, Management and Accounting % 73 Strategy and Management % 21 Management of Technology and Innovation % 35 Business and International Management % 32 JRC % % % % % % % % % 2 Page 19

20 9 Annex 3: Methodological comparison of the 'Thomson Reuters'- report, the 'JRC collaborations with universities from EU-28 Member States'-report and the 'Excellence Mapping Vol.2' The table below aims at providing a brief overview of the sample analysed, coverage and indicators used in the Thomson Reuters report, 'JRC collaborations with universities from EU-28 Member States'-report and the Excellence Mapping vol.2 produced by JRC.A2. JRC collaborations with Thomson Reuters Report universities from EU-28 Member States Excellence Mapping Data source Thomson Reuters & Thomson Reuters Web PUBSY Scopus & SciVal of Science Time period Nr of JRC publications Some Publications: journals, conferences peer-reviewed journals, and books; peer-reviewed journals conference papers, books, partial focus on article, (Pubsy categories 1.4 and trade publications; article-proceedings 3.1) partial focus on articles paper, review Productivity analysis publication output publication output publication output Productivity Benchmarking Collaborations Impact analysis Impact benchmarking Comparators Indicators Analytical dimensions: Top 10 countries and top 10 institutions with which JRC co-authored citations; citation per publication; normalized citation impact; average impact factor; countries and institutions citing JRC; social media impact 10 organisations selected by the JRC bibliometric; patents; social media Yes Yes Yes 11 organizations having produced joint publications with the JRC and that are part of the top-50 organizations of three world-wide university rankings No 153 organisations included in three world rankings (THES; QS, ARWU) citations Yes No Yes No bibliometric 15 organisations that received the highest number of citations in 26 journal categories of level 2 bibliometric Page 20

21 Thomson Reuters Report JRC collaborations with universities from EU-28 Member States Excellence Mapping Areas analysed and indicators used to determine JRC excellence 20 journal categories & 20 custom subject categories: number of publications; citation impact No distinction according to scientific areas 26 journal categories 82 journal sub-categories The Excellence Mapping, the Thomson Reuters study and the 'JRC collaborations with universities from EU-28 Member States'-report are complementary. Yet, where comparable, the results of the studies are coherent, see Annex 4 below. Page 21

22 10 Annex 4: Main findings of the 'Thomson Reuters'- report, the 'JRC collaborations with universities from EU-28 Member States'-report and the 'Excellence Mapping Vol. 2' Conclusions of the Excellence Mapping report Vol. 2: 71% of all JRC publication published in have been co-authored with other world institutions; JRC co-authored publications with institutions during ; JRC co-authored publications with 87% of the institutions ranked Top 100 in three world university rankings. Over one third of these institutions are found amongst the Top 15 in the world in terms of numbers of citations in different scientific areas; JRC has formal agreements with 45% of the institutions ranked Top 100 in three world university rankings. 96% of these institutions produced joint publications with the JRC; 38% of the institutions that have a formal agreement with the JRC and that have produced joint publications with the JRC are ranked amongst the Top 15 in terms of numbers of publications in at least one of the scientific areas where collaborations occur. Conclusions from Thomson Reuters report regarding joint publications with other organizations: Top collaborative countries are from Europe, led by Germany, JRC publications in collaboration with the United States resulted in highest impact, Switzerland and Sweden collaborations with the JRC had relatively high impact, United States led the trend in citation impact among the top 10 countries, Impact of collaborations with Spain and Belgium consistently ranked ninth and tenth among the top 10 most productive collaborative countries, Seven of the top 10 institutions that collaborate with the JRC are located in Netherlands, France and the United States, Wageningen University and Research Centre had strong collaboration with the JRC, Publications resulting from collaboration between JRC and the French National Institute of Agronomic Research had the highest normalized impact, Citation impact for collaborative organizations with the JRC trended downward between 2007 and 2010 Four out of ten comparators publications included are from one or more of JRC s top 10 collaborative countries Publications that included JRC s top 10 collaborative countries had a higher normalized citation impact Conclusions from the 'JRC collaborations with universities from EU-28 Member States'-report: Large heterogeneities across EU Member States regarding the collaborations between JRC and universities, partly reflecting the different scientific systems in the various MS. For example, in Member States like France, most of the collaborations are with national research organisations, reflecting the weight of these organisations as compared to e.g. universities. In other countries, e.g. Poland, universities together with national and governmental laboratories have a more balanced share. During the reference period , there are in total 399 universities co-writing scientific articles with the JRC, corresponding to some 3400 collaborations with scientists in universities. 25 of these 399 universities represent roughly 1/3 of all university collaborations with the JRC. Germany, Italy and the Netherlands cover 45% of collaborations amongst the top 25 universities. Page 22

23 Analysis of the position of the collaborating universities in FP7 comparative studies. Eight of the top-25 universities collaborating with the JRC can be found in the top-25 group of the FP7 ranking, and 11 can be found in the top-50 of the FP7 ranking. Putting the JRC top-25 collaborating universities with various world-wide university rankings, typically four to five universities can be put in the top 100 of the world. Page 23

24 11 Annex 5: Analysis originally foreseen for analysing JRC collaborations with the "best" institutions in the world Two important questions need to be answered: 1. Which are the "best" institutions, ie. the institutions with which the JRC should develop collaborating strategies, partnerships etc (cf. the DG mind map "World-class", section "Work with the best"). 2. Is the JRC collaborating with the "best" institutions? How do the current JRC collaborators rank compared to peer institutions in the world? A pre-requisite for answering these questions would be to clearly define from the start what is meant by the "best" institutions. Given the previous analysis done in the context of the scientific excellence mapping, and for consistency reasons, the "best" institutions have been defined as those institutions that rank top in terms of numbers of citations. These institutions are called in what follows "Top 15 most cited institutions". Approach proposed: in order to answer the two questions above, the original approach foresaw to define/calculate "collaboration profiles": (1) firstly for the "Top 15 most cited institutions" (in order to answer the second question above) taken together; and (2) secondly, for the JRC (in order to answer the first question above). A "collaboration profile" is composed of two parts: - a table containing general information, ie. number of co-authored publications and their share of the total and number of current collaborating institutions, for each area (level 2) and sub-area (level 3) of relevance for the JRC. Area / sub-area Table 1. Sample general table for area "Environmental science" and associated sub-areas - a table containing: Total Co-authored publications publications JRC Proportion of co-authored publications a. the overlap between the "best" institutions and the current collaborators (of the JRC and of the Top 15 most cited institutions taken as a group). In other words, how many current collaborators are amongst the Top 15 ranked most-cited institutions; If easily feasible, it could be interesting to calculate the overlap also with the Top 50 ranked most-cited institutions; the Top 100 ranked most-cited institutions; (applicable only for JRC current collaborators) the institutions that rank higher than the JRC in each analysed area/sub-area (Top "JRC"); and the Top 500 ranked most-cited institutions; b. the raw proportion of publications co-authored (by the JRC and by the Top 15 most cited institutions taken as a group, respectively) with institutions which are amongst the Top 15 ranked most-cited institutions. And, if feasible, amongst the Top 50 ranked most- Page 24 Number of collaborators Environmental science % 728 General environmental science % 401 Waste Management and Disposal % 147 Environmental chemistry % 291 Management, monitoring, policy and law % 163 Pollution % 206 Health, toxicology and mutagenesis % 199 Ecology % 217 Water Science and Technology % 135 Ecological modelling % 61 Total publications Top 15 most cited institutions Co-authored publications Proportion of co-authored publications Number of collaborators

25 cited institutions; amongst the Top 100 ranked most-cited institutions; amongst the institutions that rank higher than the JRC in each analysed area/sub-area (Top "JRC"); and amongst the Top 500 ranked most-cited institutions; c. the weighted proportion of publications co-authored (by the JRC and by the Top 15 most cited institutions taken as a group, respectively) with institutions which are amongst the Top 15 ranked most-cited institutions. And, if feasible, amongst the Top 50 ranked most-cited institutions; amongst the Top 100 ranked most-cited institutions; amongst the institutions that rank higher than the JRC in each analysed area/sub-area (Top "JRC"); and amongst the Top 500 ranked most-cited institutions; Note: The "weighted proportion" is based on the association of some weighting factor to the number of co-authored publications, according to the ranking of the collaborators. The principle is: a paper co-authored with a highly ranked institution should "weigh" more (ie. be more important) than a paper co-authored with a lowranked institution. Eg. a publication co-authored with a Top 15 institution would rank twice as much as a publication ranked with a Top-500 (or lower) institution. Area / sub-area Environmental science Proportion of current JRC collaborato rs amongst the Top 15 Top 15 most cited institutions Top 50 most cited institutions Top "JRC" most cited institutions Raw proportion of publications co-authored with the Top 15 Weighted proportion of publications coauthored with the Top 15 Proportion of current JRC collaborators amongst the Top 50 Raw proportion of publications coauthored with the Top 50 Weighted proportion of publications coauthored with the Top 50 Proportion of current JRC collaborators amongst the institutions ranked higher than the JRC Raw proportion of publications coauthored with the institutions ranked higher than the JRC Table 2. Sample profile table for area "Environmental science" and associated sub-areas Weighted proportion of publications coauthored with the institutions ranked higher than the JRC 93% 18% 36% 94% 0% 0% 95% 0% 0% General environmental science 87% 25% 51% 82% 0% 0% 94% 0% 0% Waste Management and Disposal Environmental chemistry Management, monitoring, policy and law Pollution Health, toxicilogy and mutagenesis Ecology Water Science and Technology Ecological modelling Remarks on feasibility The creation of blocks of "Top 15 most cited institutions" by area / sub-area implies the manual creation, within SciVal, of "groups of institutions". These groups, given that they are "custom-made" based on the ranking of institutions in terms of citations numbers, have to be built and requested one by one (and subsequently computed by Elsevier). The computation of these groups by Elsevier takes up to 2 weeks and this functionality was recently exceptionally unavailable during several weeks. Page 25