World University Rankings Downloaded from www.worldscientific.com World University Rankings Statistical Issues and Possible Remedies
World University Rankings Downloaded from www.worldscientific.com This page intentionally left blank
World University Rankings Downloaded from www.worldscientific.com World University Rankings Statistical Issues and Possible Remedies Kaycheng SOH World Scientific NEW JERSEY LONDON SINGAPORE BEIJING SHANGHAI HONG KONG TAIPEI CHENNAI TOKYO
Published by World Scientific Publishing Co. Pte. Ltd. 5 Toh Tuck Link, Singapore 596224 USA office: 27 Warren Street, Suite 401-402, Hackensack, NJ 07601 UK office: 57 Shelton Street, Covent Garden, London WC2H 9HE Library of Congress Control Number: 2017001766 World University Rankings Downloaded from www.worldscientific.com British Library Cataloguing-in-Publication Data A catalogue record for this book is available from the British Library. WORLD UNIVERSITY RANKINGS Statistical Issues and Possible Remedies Copyright 2017 by World Scientific Publishing Co. Pte. Ltd. All rights reserved. This book, or parts thereof, may not be reproduced in any form or by any means, electronic or mechanical, including photocopying, recording or any information storage and retrieval system now known or to be invented, without written permission from the publisher. For photocopying of material in this volume, please pay a copying fee through the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, USA. In this case permission to photocopy is not required from the publisher. ISBN 978-981-3200-79-1 Desk Editor: Shreya Gopi Typeset by Stallion Press Email: enquiries@stallionpress.com Printed in Singapore
Contents World University Rankings Downloaded from www.worldscientific.com About the Author Chapter 1 Chapter 2 Chapter 3 Chapter 4 Chapter 5 Don t Read University Rankings Like Reading Football League Tables: Taking a Close Look at the Indicators 1 Mirror, Mirror on the Wall: A Closer Look at the Top 10 in University Rankings 19 Profiling Universities, Not Only Ranking Them: Maximizing the Information of Predictors 29 World University Rankings: Take with a Large Pinch of Salt 49 The U21 Rankings of National Higher Education Systems: Re-analyze to Optimize 67 ix Chapter 6 Book Review: U21 Ranking of National Higher Education Systems 87 Chapter 7 World University Rankings: What Is in for Top 10 East Asian Universities? 93 v
vi World University Rankings Statistical Issues and Possible Remedies Chapter 8 Chapter 9 Simpson s Paradox and Confounding Factors in University Rankings: A Demonstration Using QS 2011 2012 Data 117 Nominal Versus Attained Weights in Universitas 21 Ranking 137 Chapter 10 Times Higher Education 100 Under 50 Ranking: Old Wine in A New Bottle? 145 World University Rankings Downloaded from www.worldscientific.com Chapter 11 Misleading University Rankings: Cause and Cure for Discrepancies between Nominal and Attained Weights 159 Chapter 12 Rectifying an Honest Error in World University Rankings: A Solution to the Problem of Indicator Weight Discrepancies 173 Chapter 13 Multicollinearity and Indicator Redundancy Problem in World University Rankings: An Example Using Times Higher Education World University Ranking 2013 2014 Data 191 Chapter 14 Problems of Indicator Weights and Multicollinearity in World University Rankings: Comparisons of Three Systems 211 Chapter 15 What the Overall doesn t Tell About World University Rankings: Examples from ARWU, QSWUR, and THEWUR in 2013 229 Chapter 16 Nearing World-class: Singapore s Two Universities in QSWUR 2015/16 249
World University Rankings Downloaded from www.worldscientific.com Contents vii Chapter 17 Improved Academic Excellence Leading to Lower Rankings and Vice Versa: Examples of Inconsistency from Three World University Ranking Systems 267
World University Rankings Downloaded from www.worldscientific.com This page intentionally left blank
About the Author World University Rankings Downloaded from www.worldscientific.com Dr. Soh Kay Cheng (1934-) is currently Research Consultant at the Singapore Centre for Chinese Language, Nanyang Technological University. He holds the degrees of Doctor of Philosophy (Education) of the National University of Singapore and Master of Education (Educational Psychology) of the University of Manchester. Positions held include Senior Fellow and Head of Centre for Applied Research in Education, National Institute of Education and Head of Testing and Guidance, Institute of Education. He also served consecutively as Deputy Director (Languages), Assistant Director (Curriculum), Ministry of Education and Principal, Nanyang Academy of Fine Arts. Besides being a teacher educator and education researcher, Dr. Soh consulted with several Singapore ministries on social survey projects. He also worked for four years as a consultant to the Education Bureau of Hong Kong SAR to modify its school appraisal system. Dr. Soh published very widely on topics such as world university rankings, international achievement rankings, creativity, and language learning in learned journals such as Compare (UK). Crossroads (USA), Journal of Educational Research (Hong Kong). Journal of Multilingual and Multicultural Development (UK), Language and Education (UK), Psychologia (Japan), Science Education (USA), Studies in Educational Evaluation (UK), The Journal of Creative Behavior (USA), The Journal of Social Psychology (USA), Journal of Higher Education Policy and Management (UK), and European Journal of Higher Education (UK). His books include Social and Educational Rankings: Problems and Prospects (2013), Analyzing Data & Interpreting Outcomes: Statistical Toolbox ix
x World University Rankings Statistical Issues and Possible Remedies World University Rankings Downloaded from www.worldscientific.com for Teacher-Researchers (2009). Workshop on Action Research: Transforming Teachers to Action Researchers (2008), Chinese Language without Tears: Tips for English-Speaking Parents (1999) and On Assessment: Ten Talks for Educational Practitioners (1978). Besides this monograph on test score interpretation, his most recent monograph is Teaching Chinese Language in Singapore: Retrospect and Challenges (Springer, 2016) of which he is the editor and author. Another of his recent publications is Understanding Test and Exam Results Statistically: An Essential Guide for Teachers and School Leaders (Springer, 2013).