A Comparison of Academic Ranking Scales

Transcription

1 A Comparison of Academic Ranking Scales Alona Zharova Andrija Mihoci Wolfgang Karl Härdle Ladislaus von Bortkiewicz Chair of Statistics C.A.S.E. Center for Applied Statistics and Economics Collaborative Research Center 649: Economic Risk Humboldt Universität zu Berlin lvb.wiwi.hu-berlin.de case.hu-berlin.de sfb649.wiwi.hu-berlin.de

2 Motivation - Nature

3 Motivation -2 Nobel Prize in Economics Figure : Selected Nobel Prize winners (from 75), (link) Milton Friedman on BBI:

4 Motivation -3 Nobel Prize in Economics Figure 2: Selected Nobel Prize winners (from 75), Robert C. Merton on BBI:

5 Motivation -4 Rankings and Funds Asset Allocation at Universities Distribution scheme based on performance, e.g., HB ranking Recruitment process for sta Funding Organisations Federal Ministry of Education and Research in Germany Deutsche Forschungsgemeinschaft (DFG): mechanisms, CRCs

6 Motivation -5 Objectives (i) Treatment of incomplete ranking data Statistics and Scientometrics Handelsblatt (HB), RePEc (RP) and Google Scholar (GS) (ii) Ranking systems dependence Perspectives: ranks and dynamics Case study: age groups across ranks

7 Motivation -6 Example Research project leaders Collaborative Research Center 649: Economic Risk - 35 research projects, 38 principal investigators (PIs), Lifespan: I. (25 28); II. (29 22); III. (23 26) Comparison using Handelsblatt (HB), RePEc (RP) and Google Scholar (GS) ranking data

8 Motivation -7 Research Questions How to predict an academic rank for researchers which are not included in a particular ranking system? How are HB, RP and GS rankings related? How does age inuence academic ranking: ranking scores, dynamics, number of citations?

9 Outline. Motivation 2. Academic Rankings and Incomplete Data 3. Ranking Systems Dependence 4. Empirical Results 5. Conclusions

10 Academic Rankings and Incomplete Data 2- Academic Rankings Scientometrics - measuring of scientic performance Academic ranking systems HB 6 rankings VWL/BWL: Life's Work (LW), Current Researchers (CR) and Under 4 (U4), Wohlrabe (23) here: 726 researchers RP 37 rankings, here: 233 researchers GS proles

11 Academic Rankings and Incomplete Data 2-2 Data VWL BWL LW CR U4 LW CR U4 PI PI PI Table : Ranking scores of selected project's PI of CRC 649 within HB VWL (23) and BWL (24) LW, CR and U4 rankings

12 Academic Rankings and Incomplete Data 2-3 Data HB RP GS PI PI PI PI Table 2: Ranking scores of selected project's PI of CRC 649 within HB (VWL LW), RP (total score) and GS (citations)

13 Academic Rankings and Incomplete Data 2-4 Data: Merging No HB Yes RP Yes No Figure 3: Mosaic plot of number of researchers, when merging of HB, RP and GS rankings takes place (Yes/No). The number of GS proles is here gurative YesNo Yes No GS

14 BWL VWL Ranking Systems Dependence 3- HB 2 23 LW CR U4 Figure 4: Parallel coordinates plots for scores of researchers within HB VWL from 2 to LW CR U4

15 BWL VWL Ranking Systems Dependence 3-2 HB 2 23 LW CR U LW CR U4 Figure 5: Parallel coordinates plots for scores of researchers within HB BWL from 29 to 24.

16 Q-Q plot Scatter plot Ranking Systems Dependence 3-3 Regression VWL CR -5 5 VWL U4-5 5 BWL CR -5 5 BWL U4-5 5 BWL LW Figure 6: Scatter plot and QQ plot of the relationship of HB VWL LW ranking with other ve rankings for top - researchers

17 Q-Q plot Scatter plot Ranking Systems Dependence 3-4 Regression VWL CR -2 2 VWL U4-2 2 BWL CR -2 2 BWL U4-2 2 BWL LW Figure 7: Scatter plot and QQ plot of the relationship of HB VWL LW ranking with other ve rankings for top - researchers

18 Q-Q plot Scatter plot Ranking Systems Dependence 3-5 Regression VWL CR -5 5 VWL U4-5 5 BWL CR -5 5 BWL U4-5 5 BWL LW Figure 8: Scatter plot and QQ plot of the relationship of HB VWL LW ranking with other ve rankings for top - researchers

19 Ranking Systems Dependence 3-6 Regression Top Top - Top - β β β β β β VWL CR VWL U BWL CR BWL U BWL LW Table 3: Estimates of the regression model parameters for rankings between HB VWL LW (dependent variable) and other ve rankings (explanatory variables) for researchers in

20 GS Ranking Systems Dependence 3-7 RP and GS # RP # 4 Figure 9: Scatterplot of RP and GS citations for 24 researchers in 25; correlation coecient is.7

21 Citations Ranking Systems Dependence 3-8 GS Figure : Parallel coordinates plot for GS citations from 27 to 24 for researchers. Max = 2953 citations

22 Ranking scores Ranking Systems Dependence 3-9 HB, RP and GS HB RP GS Figure : Parallel coordinates plot for three variables (HB common score, RP total score and GS citations) on 7 researchers

23 GS RP RP Ranking Systems Dependence 3- HB, RP and GS # HB 3 6 GS # HB Figure 2: Scatterplots of HB common score, RP total score and GS citations for 7 researchers

24 HB RP GS Empirical Results 4- Age # Age Figure 3: Scatterplots for age and ranking scores of HB, RP and GS for Top-7 scientists within each ranking system

25 Empirical Results 4-2 Age groups Age Ranking RP HB GS < > Figure 4: Mosaic plot of HB, RP and GS for Top-7 scientists within each ranking system. The width of the columns represents the number of persons within each age group

26 Conclusions 5- Conclusions (i) Treatment of incomplete ranking data Statistics and Scientometrics Handelsblatt (HB), RePEc (RP) and Google Scholar (GS) (ii) Ranking systems dependence Perspectives: ranks, dynamics Case study: age groups across ranks

27 A Comparison of Academic Ranking Scales Alona Zharova Andrija Mihoci Wolfgang Karl Härdle Ladislaus von Bortkiewicz Chair of Statistics C.A.S.E. Center for Applied Statistics and Economics Collaborative Research Center 649: Economic Risk Humboldt Universität zu Berlin lvb.wiwi.hu-berlin.de case.hu-berlin.de sfb649.wiwi.hu-berlin.de

28 References 6- References Wohlrabe, K. Einige Anmerkungen zum Handelsblatt-Ranking 23 ifo Schnelldienst, 79-83, 23. Iselin, D., Schläpfer, J. and Vogel S. Die publikationsstärksten Volkswirte: das Handelsblatt-VWL- Ranking 23 (accessed 3 Nov 24), 23. Zimmermann, C. Academic Rankings with RePEc Econometrics, MDPI, Open Access Journal, (3): , 23.