Drilling down - Subjects, Regions and Cities An overview of methodologies and highlights of recent rankings compiled by the QS Intelligence Unit Ben Sowter, Head of Division, QS Intelligence Unit, London, United Kingdom ABSTRACT In 2008, the QS Intelligence Unit was established to protect and improve upon the QS World University Rankings and in doing so have since established regional rankings for Asia and Latin America, subject rankings in 26 disciplines, a ranking of the world s Best Student Cities, and a multidimensional rating system QS Stars as well as a range of data and reporting innovations around the evaluations. This paper takes a common thread across all QS evaluations employability as a focused lens to examine how the various exercises work together. QS is unique amongst global rankings agencies in naming prospective international students as the principal audience for its work and the employability component is an essential signal of this. It has been five years since the last QS presentation at an IREG conference. At that time, the first major review of the methodology had just been completed and the opportunity was taken to explain the resulting refinements and their influence on the results. Since that time the QS Intelligence Unit (QSIU) has been established to ring-fence all QS research and to drive the volume and quality of the work. In 2008, a new layer of institutional reporting was added to give institutions fair warning of their results, for better or worse; in 2009, the QS Asian University Rankings were launched using a modified methodology to delve deeper into the performance characteristics of more institutions in the region; in 2010, the QS Stars university ratings system was devised using a much broader based methodology to highlight more diverse strata of excellence amongst institutions choosing to participate; in 2011, the QS World University Rankings were extended to embrace 26 distinct disciplines and regional efforts were extended to Latin America; and 2012 has seen the inaugural release of the QS Best Student Cities ranking and three additional subjects will be added to the mix when the QS World University Rankings by Subject are released in June. Over that period the QS global academic and employer surveys have grown to each be the world's largest of their kind with over 16,000 employers (QSIU, 2011) and 33,000 academics (QSIU, 2011) contributing to results published last year. Furthermore, QS has just been accepted as one of the first international rankings agencies to undertake the IREG audit process.
The last five years have been a journey rich with challenge and innovation and an entire conference could be dedicated to exploring the intricacies of the methodologies and the depths of the results. Perhaps the most important aspect to re-emphasise, however, so often lost in the bedlam that ensues following the launch of ranking results, is the point of it all. QS is all about the students. Students, specifically those with aspirations to undertake an international study experience, sit at the heart of QS' mission, "To enable motivated people worldwide to achieve their potential through international mobility, educational achievement and career development". This is reflected in the essential inclusion of employability as a key focus of every QS university evaluation initiative. Whilst all substantial proportion of internationally mobile students are at a graduate level, validating the inclusion of research metrics, a still larger proportion do not envisage a career in academia. Students, and their parents, faced with a global trend of escalating higher education costs, are more focused than ever in ensuring their choices lead to a successful career. The numbers facing a graduating Ivy Leaguer are eye-watering, recent developments in the UK have led to projections that the next generation of students are likely to be drawing their student loans before their parents have paid off their own. So the quality of the prospects is an increasingly important aspect. In the 2011 QS Global Employer Survey a new question was posed, "Do you actively seek or attribute value to an international study experience when recruiting?". The question attracted over 10,000 responses, and the answer is yes by a weighted majority of 60% (Molony, Sowter, & Potts, 2012). So an international study experience is a good thing but should all such experiences be considered equal? Employers have a whole new minefield to navigate and one for which, prior to the development of international university rankings and evaluations, there would have been no road map. Internationally, despite the contrary trend of increased cost, higher education participation rates have dramatically increased over the recent years (Altbach, Reisberg, & Rumbley, 2009). In fact, increased participation rates are largely to blame for making earlier student funding models unsustainable. As a result the employment market is flooded with graduates of indeterminate quality and a degree is no longer a qualification of quality in its own right. Employers are complaining that universities are not lending enough focus to skills development and work readiness training and, in some geographies, graduates are emerging to find themselves in white collar jobs that are paid less well than the blue collar jobs they went to university to avoid.
Employers are another key stakeholder group that need to understand the relative quality of international universities and specifically value the opinion of other employers in attempting to do so. And so a virtuous circle begins with both employers and career-focused students using the same set of results to refine their choices. A straightforward measure of employer reputation has been part of the QS World University Rankings since responses were deemed sufficient in 2005 but since then the pool of respondents has grown from around 400 to over 16,000 and the potential insights are far more revealing. This unique aspect of QSIU's data warehouse has been central to the speed of development of more detailed evaluations since, for all the arguments expressed here, employability is seen as a studentcentric pre-requisite to any university evaluation from QS. In chronological order of first release these are the initiatives with a brief outline of how employability is taken into account. QS Asian University Rankings - May 2009 Increased survey response levels enabled QSIU to delve deeper into the performance characteristics of institutions in the region. Considering over 400 institutions against a modified methodology, and this regional ranking maintains a 10% focus on Employer Reputation despite a reduced emphasis on Academic Reputation. QS Stars - Fall 2010 After five years of presenting the QS World University Rankings, the list of suggested alternative or additional measures had grown very long indeed. For a variety of reasons most suggestions are not practical in a global rankings context perhaps due to data availability and comparability reasons or because they are not sufficiently discerning. Graduate employment rate is a very common suggestion, but because the world rankings deal with the world s top 0.5% of universities, employment rates are almost universally high which means the indicator offers little discernment and is very sensitive to minor fluctuations. However, in a rating where institutions opt-in to the process a richer collection of indicators can and should be utilized. Employability accounts for 15% of the QS Stars (QSIU, 2012) weighting and takes into consideration three indicators: Employer Reputation, Graduate Employment Rate and Careers Service Support (the number of staff on-hand to help students find jobs). QS World University Rankings by Subject - Summer 2011 The majority of prospective international students know what they want to study before deciding where to study. This simple truth makes aggregate rankings as much a tool to help validate choices rather than an actual aid to making them. Drilling down to a discipline level is an inevitable and essential next step for a ranking that describes its central focus to be students. Disciplines differ from
one another in a number of ways, however. Publications and citations may be a valid measure for medicine, but far less so for economics and not at all for English. Based on relevance, survey response levels and publication volume, the weightings for our discipline rankings vary from one discipline to the next as follows: Indicator weightings for QS World University Rankings by Subject 2012 English Sociology Philosophy Linguistics History Modern Languages Statistics Geography Law Education Politics Communication Acc & Finance Psychology Engineering - Civil Economics Environmental Sciences Engineering - Mechanical Engineering - Electrical Engineering - Chemical Computer Science Physics Mathematics Chemistry Pharmacy Materials Science Medicine Earth Sciences Biological Sciences Academic Employer Citations 0% 20% 40% 60% 80% 100% QS University Rankings: Latin America - October 2011 The second regional ranking in the QS stable was launched in 2011. As part of the preparation for that exercise, QSIU initiated a survey asking academics and administrators in the region to give us their opinion on the priorities for the region and what indicators were most important indicators of institutional quality. Employer reputation came out very strongly in that context and as a result carries a 20% weighting in the resulting ranking.
QS Best Student Cities - February 2012 Research conducted into the motivations amongst inbound international students to Australia (Lawrence, 2009) revealed the top five motivations to be: 1. International ranking 2. Reputation in field of study 3. The city where the institution is located 4. Reputation of institution 5. Attractive course / program structure With the first two categories covered, the natural next step is to look at the third and analyze destinations. The QS Best Student Cities was released in February 2012. The methodology (Sowter, 2012) draws on twelve indicators in five equally weighted categories, one of which is Employer Activity. The two indicators in the category look at the number of employers expressing a preference for at least one institution in the given city an emphasis is placed on international responses. There are more students than ever studying outside their own country (OECD, 2011) and more universities than ever interested in recruiting them. With higher education costs increasing globally, and competition for the best graduate jobs tougher than ever, there is more riding on study selection choices than ever before and an orgy of marketing information, flooding every channel, designed to influence them. QSIU aims to map the landscape and provide a toolkit to assist students in navigation. Look for more developments in the next five years including a greater sense of how all the different tools fit together. Bibliography Altbach, P. G., Reisberg, L., & Rumbley, L. E. (2009). Trends in Global Higher Education: Tracking an Academic Revolution. Paris: UNESCO. Lawrence, R. (2009). The Buyer Behaviour of International Students: How They Decide. AIEC. Sydney: Prospect Research and Marketing Pty Ltd. Molony, J., Sowter, B., & Potts, D. (2012). QS Global Employer Survey Report 2011 - How Employers Value an International Study Experience. London: QS Intelligence Unit.
OECD. (2011). Education at a Glance. Paris: OECD. QSIU. (2011, September 15). 2011 Academic Survey Responses. Retrieved from QS Intelligence Unit: http://www.iu.qs.com/university-rankings/world-university-rankings/2011-academicsurvey-responses/ QSIU. (2011, September 15). 2011 Employer Survey Responses. Retrieved from QS Intelligence Unit: http://www.iu.qs.com/university-rankings/world-university-rankings/2011-employersurvey-responses/ QSIU. (2012, March). QS Stars Methodology. Retrieved from QS Intelligence Unit: http://www.iu.qs.com/projects-and-services/qs-stars/qs-stars-methodology/ Sowter, B. (2012, February). QS Best Student Cities. Retrieved from QS Intelligence Unit: http://www.iu.qs.com/university-rankings/qs-best-student-cities/