Why web-based rankings of universities should not be neglected

Size: px

Start display at page:

Download "Why web-based rankings of universities should not be neglected"

Nora Thornton
6 years ago
Views:

1 World University Rankings Symposium Identifying Excellence and Diversity in International Education: Rankings and Beyond École Normale Supérieure (ENS), Paris. May 18th 2011 Why web-based rankings of universities should not be neglected Isidro F. Aguillo

The Cybermetrics Lab A rigorous scientific approach to the ranking of universities Scholars making scientific research Researchers belonging to the National Research Council (CSIC) The largest

2 The Cybermetrics Lab A rigorous scientific approach to the ranking of universities Scholars making scientific research Researchers belonging to the National Research Council (CSIC) The largest Spanish research public organization Recognised by our peers 15 years experience in quantitative analysis and evaluation of scholar communication and academic institutions Papers in referred scientific journals, contributions to international conferences, reports to governmental bodies Funded by public resources International cooperation projects funded by European Commission Research Agenda Promote Open Access initiatives Global coverage, including universities from developing countries Building Cybermetrics/Webometrics as an emerging discipline 2

suitable for students/scholars Experts driven Complex reports, difficult to understand Lack of global scenarios Focus on US research university

3 Why Global Rankings of Universities? A new powerful tool with a significant impact on Higher Education The problems of the Higher Education Science in the 90s Limited impact, not suitable for students/scholars Experts driven Complex reports, difficult to understand Lack of global scenarios Focus on US research university model International Rankings (ARWU, 2003) Huge impact, including governments Data driven Easy to understand, ONE composite indicator (rank) World and continental scenarios Excellence clearly defined Useful for both individual universities and national systems 3

4 How? A couple of models ACTIVITY 4 IMPACT

5 Customizing the model Most of the Global Rankings use an arbitrary weighting system, without an underlying model RANKING ARWU (Shanghai) 40% TIMES HE (2010) QS HEEACT (Taiwan) WR (Webometrics) 5 IMPACT ACTIVITY 60% 35% 65% 30% 70% 20% 80% 50% 50%

6 Feasibility It is not so easy, sometimes there is no data, or they are not trusted or they are not useful 6

Quality Quality can be measured by consensus or majority of experts evaluating the

Peers with global knowledge (not available) o Size: Medium (several thousands) Citation

Expertise: Peers o Size: Small (a few hundreds) Link analysis o Topic: Quality of

7 Quality Quality can be measured by consensus or majority of experts evaluating the targeted topic Survey o Topic: Perceived (subjective) academic reputation o Expertise: Peers with global knowledge (not available) o Size: Medium (several thousands) Citation analysis o Topic: Empirical (objective) scientific impact (formal communication) o Expertise: Peers o Size: Small (a few hundreds) Link analysis o Topic: Quality of academic and other web contents o Expertise: Web-editors (peers and non-peers) o Size: Huge (millions) 7

8 A critical review Surprisingly most of the Rankings are being accepted without a critical review of the authorship, the methodology and the results Ranking Missions Indicators Model Results Comments ARWU Research Scientific excellence No model (Top 200) Correlated World class universities (incomplete, mistakes, old data) WEBOMETRICS All Activity & Impact (Web) Impact factor based Correlated (relevant exceptions) World universities (extreme penalties for bad web policies, search engines biases) HEEACT Research Activity & Impact (Papers, Citations) Striking not explained model Correlated World research universities (variables not independent) LEIDEN Research Impact (Citations/Papers) Crown indicator Several rankings World research universities (questioned methodology) SCIMAGO Research Activity OR Impact (Papers, Citations) Ranked by size (!) as default Several rankings World research institutions (no composite ranking) THE All Activities, Prestige (Surveys) & Impact (Citations) No model Not valid Methodology applied incorrectly, unethical (non-cooperating universities are not identified) CHEPS All Multi-Rank Customizable by end-users! No Ranking A collection of colorful smileys 8

9 Power-law Distribution SCORE WR log-norm z-score QS ARWU HEEACT CWTS RANK 9

Webometrics Ranking The Ranking Web is one of the first World Rankings to be published and currently it is the one with the largest coverage Published since 2004, two editions per

universities according to web indicators The hypothesis is that in the 21st century the web reflects the organization, activities, research results, knowledge transfer, prestige,

10 Webometrics Ranking The Ranking Web is one of the first World Rankings to be published and currently it is the one with the largest coverage Published since 2004, two editions per year (January & July) The largest and most updated Directory with more than 20,000 Higher Education Institutions from all over the World The Ranking provides the Top 12,000 universities according to web indicators The hypothesis is that in the 21st century the web reflects the organization, activities, research results, knowledge transfer, prestige, and international visibility of the universities If the web performance is below the expected position could be due to lack of commitment to the electronic publication, or bad web practices 10

11 Geographical coverage (Top 1000) WR Jan

12 Digital Divide 12

13 Europe WR Jan (Top 500)

14 Universities in France 14

Comparative Analysis Different views are correlated but the position of French universities, also in the Web ranking, is below expectations Coverage o Rankings based on research performance where the

(distance) teaching in French or transfer of knowledge and technology, also in local language University roles o Research-based rankings describe World-class universities o Web ranking also identify

15 Comparative Analysis Different views are correlated but the position of French universities, also in the Web ranking, is below expectations Coverage o Rankings based on research performance where the English is the lingua franca of scholar communication are penalizing French language o Webometrics takes into consideration also other university missions including traditional and off campus (distance) teaching in French or transfer of knowledge and technology, also in local language University roles o Research-based rankings describe World-class universities o Web ranking also identify nation-building institutions, probably more important to the local communities (social commitment) and governments (economic and political involvement) Bad practices o Surprisingly for the 21st century, there are academic institutions (its leaders or scholars) that are ignoring the Web: No enough contents o Web policies are inadequate, incorrect or insufficient. Too many institutions, frequent web domain changes, too few repositories 15

16 Rankings should be useful French and Norwegian (and others) Higher Education Ministries visited Shanghai for advice!!. Suggestions provided here are tentative: ARWU oriented oriented Choose 2-3 universities as World-class candidates All the universities should reinforce their web presence Increase their budget at the expense of the other universities Budget should be available to web publication for Give legal capability for offering better salaries and grants Attract international & national best candidates to these universities Reinforce research capabilities, transferring elite institutes and hospitals Fire low performers & close not very productive or international disciplines (Humanities!) 16 Webometrics * Reflecting the institution s performance * Attracting international students & scholars * Disseminating the research results * Giving more visibility to neglected disciplines (Humanities!) * Improve the transfer of technology to industry and other companies * Increase the knowledge transfer and the community engagement

Web strategies Web should be the showcase of the University, the place for publishing its basic info & facts,

teaching supporting pages o Second mission: Research groups and (scientists) personal pages, Research projects

papers o Media (videos, webinars, objects) and software portals o Data repository / (Web) Archive o Social

17 Web strategies Web should be the showcase of the University, the place for publishing its basic info & facts, organisation, activities, policies, results Contents o First mission: Traditional and off campus (distance) teaching supporting pages o Second mission: Research groups and (scientists) personal pages, Research projects websites o Third mission: Hosting third parties sites Quality Contents o Portal of journals / Repository of papers o Media (videos, webinars, objects) and software portals o Data repository / (Web) Archive o Social commitments International Contents o Multilingual (also English) versions o Guides to foreign students/scholars o Super-sites 17

18 Learning management systems (LMS)

19 MIT Open CourseWare 19

20 Personal Pages

21 ENS (?) Repository Why not? 21

22 Thesis at USP 22

23 Web 2.0: Priorities Rank Facebook YouTube Social Bookmarking External Blogs Institutional Blogs 1 Campus Life Events Courses Faculty Campus Life 2 Sports Campus Life Projects, NonResearch Research, Physical Sciences Events 3 Technology Faculty Research, Physical Sciences Institution Overall Institution Overall 4 Product Services Courses Events Expert Commentary Institution SubGroups 5 Events Institution Overall Faculty Events Admissions

activities and the results of scholars and researchers o Web is the best channel for online and distance learning, the best showcase for attracting international talent (foreign students and

24 Summarizing Why the Web? o Web presence is an indicator. Web is reflecting overall performance and not only online activities o Web is already the most important scholarly communication tool o Web increases significantly the visibility and impact of the activities and the results of scholars and researchers o Web is the best channel for online and distance learning, the best showcase for attracting international talent (foreign students and professors), the meeting point with the society, the economic and industrial stakeholders Why the Ranking Web? o Webometrics is offering the larger coverage, ranking World-class universities but also institutions from emerging and developing countries o Discrepancies on the web ranking informs about problems with the governance and long-term strategies of the universities, obsolete or inadequate policies and bad web practices o US universities leadership in Webometrics is showing a concerning and a long term threat to the scientific and cultural presence of the Europeans (and their languages) in the Web 24

25 Open Forum Contact: Isidro F. Aguillo, HcPhD The Cybermetrics Lab. IPP-CCHS-CSIC Questions? Thank you! 25

EVALUATING POPULARITY OF COLLEGE S WEBSITE IN INDONESIA

International Conference on Internet Studies, August 16-17, 2014, Singapore EVALUATING POPULARITY OF COLLEGE S WEBSITE IN INDONESIA Widya Silfianti, Gunadarma University, Indonesia wsilfi@staff.gunadarma.ac.id