Potential Teaching Excellence Framework Measures.

Strategy, Planning & Governance. Potential Teaching Excellence Framework Measures. Professor Wyn Morgan Dr Tony Strike February 2016

2

Potential Teaching Excellence Framework Measures Contents Introduction 3 1 Average Tariff 4 2 UG degree outcomes 6 3 Student/Staff ratio (SSRs) 8 4 Destination of leavers: Positive destinations 10 5 Salaries for new graduates 12 6 NSS: Overall Satisfaction 14 7 NSS: Teaching on my course 16 8 NSS: Learning resources 18 9 International Staff 20 10 Students no longer in HE after 1 year 22 11 Students from LPN (low participation neighbourhoods) 24 12 Applications per registration 26 13 Proportion from state sector 28 14 About the data 30 1

The University of Sheffield 2

Potential Teaching Excellence Framework Measures Introduction This is a consultation document for internal use only. The next step in the building of national proposals for a Teaching Excellence Framework (TEF) is to be a technical consultation. In anticipation of universities being invited to submit comments later in 2016, the university can develop and strengthen its own view. In the first year it is likely the TEF will be based on results from Quality Assurance. Subsequently, it is proposed the TEF will include metrics used as proxies to indicate teaching excellence. This suggestion has already produced a series of responses to the government s Green Paper as to why this approach could be viewed as problematic. The University of Sheffield Students Union has been clear that much of what students come to recognise as educational excellence in higher education is in the ethos and values of the learning community, which is tangible but resistant to objectification and measurement. In showing some possible proxy measures, we are preparing for the TEF technical consultation without wanting to fall into the epistemological trap of legitimising the object of our concerns. This pamphlet shows potential measures that may be incorporated into the TEF, but does not propose that such measures are used. The commentary provided with each measure offers a critical appraisal. By providing a set of available measures with data, Strategy, Planning & Governance aims to stimulate an internal debate on which, if any, of these measures the University might seek to support over others if metrics are still to be a part of the TEF. By showing the data for the University of Sheffield, by comparing it to the rest of the Russell Group and to the sector, it may be possible to identify defining strengths and necessary areas for improvement. Working closely as we are with colleagues in the Department of Business, Innovation and Skills, who are based in Sheffield, on the development of the TEF, we recognise we do not have a determining influence on the national policy. We can provide greater clarity on what the University of Sheffield means by high quality teaching and educational excellence and how we intend to evidence that to others. Results are shown here at an institutional level of analysis; aware that students often make decisions at the subject level and that there are subject level variances in the underlying data. Dr Tony Strike 3

Potential Teaching Excellence Framework Measures 1. Average Tariff on Entry This is an input measure and might say something about the seen reputation of the University and its ability to attract academically able students. Being part of a student group which is academically able may influence the quality of the educational experience for those students through the level and pace of the teaching and through peer learning. However, if tariff is simply measuring reputation and the University is adding no value then teaching excellence is not indicated by this measure. Universities which have a specific policy of accepting students with lower grades as part of an access policy will tend to have their average score depressed. The unintended consequence of a measure of entry tariff might be to discourage access in favour of elitism. If linked to learning gain; measured by comparing degree classification to average entry tariff, the incentive could be for institutions to accept lower tariff students and award more good honours degrees. The Russell Group performs well on this measure. The University of Sheffield does well on this measure in the sector and sits in the middle of the Russell Group. 4

Potential Teaching Excellence Framework Measures Average Tariff Top 5 1 The University of Cambridge 601.0 2 The University of Oxford 570.8 3 Imperial College of Science, Technology and Medicine 566.8 4 London School of Economics and Political Science 531.4 5 University of Durham 522.4 Sheffield +/- 3 21 The University of Manchester 430.6 22 The University of Leeds 429.8 23 Glasgow School of Art 426.6 24 The University of Sheffield 425.6 25 University of Nottingham 425.4 26 The University of Birmingham 425.2 27 University of Newcastle-upon-Tyne 423.6 Bottom 5 149 Birkbeck College 221.2 150 Harper Adams University 220.0 151 University College Birmingham 218.7 152 London Metropolitan University 217.7 153 Institute of Education 203.6 Russell Group 1 The University of Cambridge 601.0 2 The University of Oxford 570.8 3 Imperial College of Science, Technology and Medicine 566.8 4 London School of Economics and Political Science 531.4 5 University of Durham 522.4 7 University College London 505.2 8 The University of Bristol 484.6 9 The University of Edinburgh 482.2 10 The University of Warwick 479.7 12 The University of Glasgow 464.8 13 The University of Exeter 460.2 15 King s College London 452.3 19 The University of York 433.9 21 The University of Manchester 430.6 22 The University of Leeds 429.8 24 The University of Sheffield 425.6 25 University of Nottingham 425.4 26 The University of Birmingham 425.2 27 University of Newcastle-upon-Tyne 423.6 29 Cardiff University 421.2 32 Queen Mary University of London 408.8 33 The University of Southampton 407.5 35 The University of Liverpool 402.8 45 The Queen s University of Belfast 378.8 Definition: Average tariff of all subjects taken; home students only Source: HESA sector data, 2013/14 5

The University of Sheffield 2. UG degree outcomes This is an output measure and might indicate the effectiveness of teaching at an institution in enabling students to achieve good degree results. The QAA ensures high academic standards across all higher education institutions, and so measuring degree classifications would demonstrate the number of students that achieved a certain standard of education. However, degree outcomes are influenced by the quality of students recruited by an institution, and does not indicate the quality of the education that they receive or the value added to a student s education. While the QAA ensures standards, institutions still set and mark their own exams, so data on undergraduate degree outcomes is not comparable between institutions. Including undergraduate degree outcomes as a measure in the TEF could promote grade inflation, as institutions would be able to succeed in the TEF by lowering the standards required for higher level degree classifications. It could also disadvantage those institutions that are successful in widening participation, as there is evidence that some WP students are less likely to achieve higher degree classifications. The Russell Group performs well on this measure. The University of Sheffield does well on this measure in the sector and sits in the middle of the Russell Group. 6

Potential Teaching Excellence Framework Measures Good Honours Top 5 % 1 Courtauld Institute of Art 90.2 2 Central School of Speech and Drama 87.9 3 Royal Academy of Music 87.8 4 The University of Oxford 87.7 5 University of Durham 87.4 Sheffield +/- 3 32 Royal Holloway and Bedford New College 75.9 33 Leeds College of Art 75.5 34 The University of East Anglia 75.4 35 The University of Sheffield 75.4 36 Bournemouth University 75.2 37 The University of Winchester 74.9 38 Bath Spa University 74.5 Bottom 5 148 Royal Conservatoire of Scotland 59.4 149 The Robert Gordon University 39.4 150 SRUC (Scotland s Rural College) 30.4 151 University of the Highlands and Islands 19.8 152 The University of the West of Scotland 19.3 Russell Group % 4 The University of Oxford 87.7 5 University of Durham 87.4 14 London School of Economics and Political Science 82.1 16 The University of Exeter 81.3 17 University College London 80.3 19 The University of York 79.8 22 The University of Warwick 78.7 25 The University of Leeds 77.8 27 The University of Birmingham 76.6 28 The University of Bristol 76.4 31 The University of Southampton 76.0 35 The University of Sheffield 75.4 42 Imperial College of Science, Technology and Medicine 73.7 43 University of Nottingham 73.6 47 Cardiff University 71.9 50 The University of Cambridge 71.1 51 University of Newcastle-upon-Tyne 71.0 52 King s College London 70.6 55 The University of Manchester 70.1 58 The Queen s University of Belfast 69.7 68 The University of Edinburgh 67.8 76 The University of Liverpool 66.9 84 Queen Mary University of London 65.5 100 The University of Glasgow 62.9 Definition: Good honours defined as 1st and 2.1 Source: HESA sector data, 2013/14 7

The University of Sheffield 3. Student/Staff ratio (SSRs) There is evidence that students particularly value time spent learning in small groups. A greater proportion of staff to students might allow students to benefit from more direct contact with their teachers, and allow staff a greater capacity to focus on the needs of individual students. However, data on SSRs between forms of provision is not comparable. Full-time, part-time, distance learning and study in the workplace may all require different forms of provision and different SSRs would be appropriate in each case. There are also issues with the data as collected by HESA, as it does not account for the wide range in the proportion of time spent on teaching by academics. Measuring SSRs could also reward inefficiency, as institutions could perform better in the TEF by assigning more staff to teach than is optimal for student learning. The measure might also simply reflect the wealth of institutions, as those who can afford to employ more teaching staff will achieve better SSRs. The Russell Group has a lower staff to student ratio in comparison with the sector as a whole. The University of Sheffield sits within the bottom third of the Russell Group. 8

Potential Teaching Excellence Framework Measures SSRs Top 5 Ratio 1 Institute of Education 1.71 2 Courtauld Institute of Art 3.57 3 Royal Academy of Music 4.30 4 Guildhall School of Music and Drama 4.72 5 London School of Economics and Political Science 4.82 Sheffield +/- 3 38 The University of Glasgow 10.84 39 Cardiff University 10.86 40 The University of Birmingham 10.92 41 The University of Sheffield 10.93 42 University of Nottingham 10.93 43 The University of Dundee 11.02 44 The University of East Anglia 11.08 Bottom 5 150 Bishop Grosseteste University 22.81 151 University Campus Suffolk 24.01 152 The Open University 29.76 153 Ravensbourne 31.84 154 University of the Highlands and Islands 360.27 Definition: Student staff ratio; includes all UG students (first degree & other UG) and staff on both T&R and T only contracts Source: HEIDI 2013/14 Russell Group Ratio 5 London School of Economics and Political Science 4.82 7 University College London 5.86 9 The University of Oxford 6.39 10 Imperial College of Science, Technology and Medicine 6.51 14 The University of Cambridge 7.22 15 King s College London 7.26 19 The University of Warwick 8.81 22 The University of Edinburgh 8.88 23 The University of Southampton 8.93 28 Queen Mary University of London 9.29 30 The University of Manchester 9.89 32 The University of Bristol 10.63 35 The University of Liverpool 10.73 38 The University of Glasgow 10.84 39 Cardiff University 10.86 40 The University of Birmingham 10.92 41 The University of Sheffield 10.93 42 University of Nottingham 10.93 45 The University of Leeds 11.13 51 University of Durham 11.61 52 The University of York 11.61 58 University of Newcastle-upon-Tyne 12.30 64 The Queen s University of Belfast 12.89 65 The University of Exeter 13.00 9

The University of Sheffield 4. Destination of leavers: Positive destinations This is an output measure which could show those institutions which best prepare their students for graduate employment through teaching, course design and links with employers. Students increasingly cite the importance of job prospects as a measure of their time at university. However, research suggests that information about graduate employment rates tells higher education applicants little about the quality of education they can expect to receive, as the data is influenced by factors such as institutional reputation, league table position, and students entry grades and socioeconomic background. The data only indicates if the students destinations were positive as currently defined; it does not reveal the extent to which the teaching they received and the qualifications they gained helped them find employment. Furthermore, the data will vary widely between subjects, and knowing an institutional average will not help a student pick a particular course. The measure could damage widening participation, as universities could recruit students whose existing education and socio-economic background increase the chances of them finding graduate employment. It could also incentivise universities to focus on the provision of professional courses which may have better employment outcomes. The Russell Group performs well on this measure. The University of Sheffield is positioned at the lower end of the Russell Group rank. 10

Potential Teaching Excellence Framework Measures Positive Destinations Top 5 % 1 Royal College of Music 100.0 2 Royal Academy of Music 95.6 3 St George s Hospital Medical School 93.1 4 Imperial College of Science, Technology and Medicine 91.4 5 The University of Cambridge 89.8 Sheffield +/- 3 55 Glyndwr University 76.8 56 The University of Liverpool 76.6 57 The University of York 76.5 58 The University of Sheffield 76.2 59 The University of Aberdeen 75.9 60 The University of Huddersfield 75.6 61 The University of Stirling 75.3 Bottom 5 150 Leeds College of Art 52.6 151 London Metropolitan University 51.9 152 University of the Highlands and Islands 50.2 153 The University of East London 49.0 154 SRUC (Scotland s Rural College) 30.9 Definition: Positive destination is defined using the HESA positive destination marker and excludes N/A, refused and unknown Source: HESA sector data from DLHE 2013/14 Russell Group % 4 Imperial College of Science, Technology and Medicine 91.4 5 The University of Cambridge 89.8 8 The University of Oxford 87.3 10 King s College London 86.9 11 The University of Birmingham 86.7 16 University College London 84.4 17 University of Durham 84.4 25 University of Nottingham 82.5 26 The University of Warwick 82.1 31 Cardiff University 80.4 32 The University of Exeter 80.3 33 The University of Glasgow 80.1 34 London School of Economics and Political Science 79.9 35 University of Newcastle-upon-Tyne 79.7 36 The University of Southampton 79.7 37 The University of Bristol 79.6 41 The University of Leeds 78.8 43 The University of Manchester 78.8 44 The University of Edinburgh 78.7 45 The Queen s University of Belfast 78.7 56 The University of Liverpool 76.6 57 The University of York 76.5 58 The University of Sheffield 76.2 64 Queen Mary University of London 73.9 11

The University of Sheffield 5. Salaries for new graduates This is an output measure which could show whether institutions prepare their students for the highest earning graduate jobs through teaching, course design and links with employers. Students with income-dependent educational loans to pay back may be more concerned about likely salaries on graduation. The measure could damage widening participation, as institutions recruit students whose existing education and socio-economic background increase the chances of them finding higher paid employment. It could incentivise institutions to focus on the provision of professional courses targeted at higher paying sectors. It may provide a disincentive for universities to produce civically minded graduates who might take lower paid work in the public or third sector, or graduates who go on to study for a doctorate. It could also penalise institutions that are based in regions with lower average salaries, but who have enabled their students to find employment within their region. The Russell Group has a range of outcomes on this measure. The University of Sheffield does not perform particularly well, positioned at the lower end of the Russell Group rank. 12

Potential Teaching Excellence Framework Measures Average Salary Top 5 1 Birkbeck College 35,861 2 Imperial College of Science, Technology and Medicin 29,648 3 The Open University 28,678 4 Institute of Education 28,214 5 London School of Economics and Political Science 28,144 Sheffield +/- 3 60 The University of Exeter 21,677 61 Coventry University 21,659 62 University of Northumbria at Newcastle 21,656 63 The University of Sheffield 21,609 64 The University of Lancaster 21,601 65 The University of York 21,428 66 Staffordshire University 21,424 Bottom 5 150 Glasgow School of Art 14,800 151 Royal Northern College of Music 13,714 152 Conservatoire for Dance and Drama 13,182 153 Royal College of Music 12,077 154 Trinity Laban Conservatoire of Music and Dance 12,053 Definition: Average salary from all employed DLHE respondents; graduate and non-graduate employment Source: HESA sector data from DLHE 2013/14 Russell Group 2 Imperial College of Science, Technology and Medicine 29,648 5 London School of Economics and Political Science 28,144 7 The University of Warwick 26,677 8 The University of Oxford 26,087 9 King's College London 25,951 10 The University of Cambridge 25,718 16 University College London 24,423 18 Queen Mary University of London 24,110 21 The University of Bristol 23,927 28 The University of Edinburgh 23,355 30 University of Durham 23,170 32 The University of Glasgow 22,746 34 University of Nottingham 22,676 36 The University of Southampton 22,568 40 University of Newcastle-upon-Tyne 22,416 42 The University of Birmingham 22,253 51 Cardiff University 21,957 56 The University of Manchester 21,769 59 The University of Liverpool 21,729 60 The University of Exeter 21,677 63 The University of Sheffield 21,609 65 The University of York 21,428 67 The Queen's University of Belfast 21,392 74 The University of Leeds 20,890 13

The University of Sheffield 6. NSS: Overall Satisfaction This measure could indicate students' overall satisfaction with their university experience, a large part of which is the teaching that they receive. Final year undergraduates give feedback with high rates of participation in the NSS, so universities should respect and take seriously the results given. However, student satisfaction is a difficult measure to interpret and there is no evidence that satisfaction is linked to teaching quality or learning gains. There is not much differentiation between institutions in terms of NSS scores, and scores vary between subjects within institutions. The level of granularity needed to provide useful information can be foiled by small sample sizes and missing data. If FE colleges are included along with HE institutions, a different rank is obtained than when only HE institutions are included. Both ranks have been shown the first including FE colleges and the second excluding them to allow for variation in the scope of the TEF. This has also been applied to the following two NSS measures. The University performs well on this measure, both within the sector and within the Russell Group. 14

Potential Teaching Excellence Framework Measures NSS: Overall Satisfaction Top 5 =1 Italia Conti Academy of Theatre Arts 100 =1 South Thames College 100 =1 Hartlepool College of Further Education 100 =1 Kendal College 100 =1 Spirit SSR Limited 100 Sheffield +/- 3 =58 Oxford Brookes University 90 =58 The College of Animal Welfare Limited 90 =58 The University of Bath 90 =58 The University of Sheffield 90 =58 University of Durham 90 =58 University of Exeter 90 =58 Cardiff University 90 Bottom 5 363 Shockout Arts 47 364 Shrewsbury College of Arts & Technology 45 365 Halesowen College 33 366 Carshalton College 26 367 City of Glasgow College 18 Top 5 1 St Mary's University College 96 =2 The University of Keele 95 =2 University of Buckingham 95 4 Courtauld Institute of Art 94 =5 Harper Adams University 93 Sheffield +/- 3 =20 The University of Leeds 90 =20 Cardiff University 90 =20 Aston University 90 =20 The University of Sheffield 90 =20 University of Exeter 90 =20 University of Durham 90 =20 University of Glasgow 90 Bottom 5 Note: Excluding all FE colleges Definition: NSS institutional level responses to Overall Satisfaction; all full-time undergraduates Source: NSS 2015 =146 Bournemouth University 78 =146 University of Wales Trinity Saint David 78 150 The Royal Central School of Speech & Drama 77 =151 Writtle College 75 =151 University of the Arts, London 75 Russel Group =43 University of Oxford 91 =43 University of Newcastle-upon-Tyne 91 =58 The University of Leeds 90 =58 University of Cambridge 90 =58 University of Glasgow 90 =58 The University of Sheffield 90 =58 University of Durham 90 =58 University of Exeter 90 =58 Cardiff University 90 =79 The University of Birmingham 89 =105 The University of York 88 =105 Queen's University Belfast 88 =105 Imperial College London 88 =105 Queen Mary University of London 88 =105 The University of Southampton 88 =125 The University of Warwick 87 =142 The University of Nottingham 86 =142 The University of Manchester 86 =162 The University of Liverpool 85 =183 University of Bristol 84 =183 University of Edinburgh 84 =197 University College London 83 =233 King's College London 81 =233 London School of Economics & Political Science 81 15

The University of Sheffield 7. NSS: Teaching on my course This metric could be seen as a direct measure of teaching quality from a student perspective. An institution could be gaining positive feedback from students on its teaching because the teaching is high quality. However, there is evidence that students assess lecturers on how entertained they felt in classes rather than on the substance of what they were taught, which could negatively impact on how staff teach. The NSS does not sufficiently account for student engagement with learning, which is not fully captured by this measure. This measure could incentivise institutions to promote a style of teaching which satisfies and entertains students, rather than engaging and challenging them. The University performs relatively well on this measure and is positioned towards the top of the Russell Group. 16

Potential Teaching Excellence Framework Measures NSS: Teaching on my course Top 5 =1 Italia Conti Academy of Theatre Arts 100 =1 South Thames College 100 =3 Hartlepool College of Further Education 99 =3 Kendal College 99 =3 Spirit SSR Limited 99 Sheffield +/- 3 =83 Oxford Brookes University 90 =83 The College of Animal Welfare Limited 90 =83 The University of Bath 90 =83 The University of Sheffield 90 =83 University of Durham 90 =83 University of Exeter 90 =83 Northampton College 90 Bottom 5 364 Shrewsbury College of Arts & Technology 59 365 Newham Training and Education Centre 56 366 City of Glasgow College 50 367 Carshalton College 46 368 Shockout Arts 45 Top 5 =1 St Mary's University College 95 =1 The Royal Veterinary College 95 =3 University of Buckingham 94 =3 Courtauld Institute of Art 94 =5 University of Oxford 93 Sheffield +/- 3 =15 Stranmillis University College 91 =27 Queen's University Belfast 90 =27 Loughborough University 90 =27 The University of Sheffield 90 =27 University of Exeter 90 =27 University of Durham 90 =27 University of Derby 90 Bottom 5 Note: Excluding all FE colleges Definition: NSS institutional level responses to Teaching on my Course; all full-time undergraduates Source: NSS 2015 =146 London School of Economics & Political Science 81 =149 University of the Arts, London 80 =149 Writtle College 80 =151 SRUC (Scotland's Rural College) 79 =151 The University of Westminster 79 Russel Group =33 University of Oxford 93 =42 The University of Leeds 92 =42 University of Cambridge 91 =42 The University of York 91 =42 University of Glasgow 91 =42 The University of Birmingham 91 =83 Queen's University Belfast 90 =83 The University of Sheffield 90 =83 University of Exeter 90 =83 University of Durham 90 =105 Imperial College London 89 =105 University of Newcastle-upon-Tyne 89 =105 Cardiff University 89 =105 University of Bristol 89 =130 The University of Southampton 88 =130 The University of Warwick 88 =130 The University of Nottingham 88 =130 Queen Mary University of London 88 =172 The University of Manchester 87 =172 University of Edinburgh 87 =203 King's College London 86 =203 The University of Liverpool 86 =230 University College London 85 =305 London School of Economics & Political Science 81 17

The University of Sheffield 8. NSS: Learning Resources This measure could be used as a proxy for the investment that an institution has made into teaching, and the importance that it places on teaching its students. It could also indicate how well an institution is placed to enable engagement with their course and in their learning. However, learning resources are not necessary, or in some cases even an important factor in the provision of excellent teaching. A good score on this measure may also simply reflect the wealth of an institution and how much it can afford to invest in its facilities. This measure could incentivise institutions to shift funding towards developing its learning resources at the expense of hiring and training teaching staff. It could also penalise less wealthy institutions. The University does relatively well on this measure, and is positioned in the middle of the Russell Group. 18

Potential Teaching Excellence Framework Measures NSS: Learning Resources Top 5 =1 Leeds College of Building 100 =1 Edinburgh's Telford College 100 3 Brighton and Sussex Medical School 99 4 The College of Animal Welfare Limited 98 5 St Mary's University College 97 Sheffield +/- 3 =48 Coventry University 89 =48 Bangor University 89 =48 University of Glasgow 89 =48 The University of Sheffield 89 =48 University of Durham 89 =48 University of Exeter 89 =48 Liverpool Hope University 89 Bottom 5 364 Halesowen College 51 365 Bournville College 49 366 Italia Conti Academy of Theatre Arts 47 367 Somerset SCITT Consortium 41 368 Newham Training and Education Centre 36 Top 5 1 St Mary's University College 97 =2 Royal Northern College of Music 95 =2 University of Cambridge 95 =2 University of Oxford 95 =2 Imperial College London 95 Sheffield +/- 3 =32 Bangor University 89 =32 Liverpool Hope University 89 =32 University of Ulster 89 =32 The University of Sheffield 89 =32 University of Exeter 89 =32 Liverpool John Moores University 89 =32 University for the Creative Arts 89 Bottom 5 Note: Excluding all FE colleges Definition: NSS institutional level responses to Learning Resources; all full-time undergraduates Source: NSS 2015 =148 Glasgow School of Art 74 =148 Goldsmiths' College 74 =148 University of the Highlands & Islands 74 151 Glyndwr University 73 152 University of Wales Trinity Saint David 70 Russel Group =6 University of Cambridge 95 =6 University of Oxford 95 =6 Imperial College London 95 =13 Queen's University Belfast 92 =19 The University of Leeds 92 =19 The University of Southampton 92 =25 University of Newcastle upon Tyne 91 =25 Cardiff University 91 =33 The University of York 90 =33 The University of Manchester 90 =48 The University of Sheffield 89 =48 University of Exeter 89 =48 University of Durham 89 =48 University of Glasgow 89 =48 The University of Warwick 89 =48 The University of Nottingham 89 =48 University of Edinburgh 89 =71 The University of Liverpool 88 =109 King's College London 86 =153 The University of Birmingham 84 =153 University of Bristol 84 =153 University College London 84 =202 London School of Economics & Political Science 81 =219 Queen Mary University of London 80 19

The University of Sheffield 9. Percentage of International Staff Students who are being taught by the best academics from around the world will arguably experience teaching excellence. However, research intensive institutions tend to have higher numbers of international staff, brought to the UK for their research contribution and not because they are necessarily the best teachers. There may be a bias by institutional type. The measure could also favour institutions based in urban centres who attract more international staff. The unintended consequence could be that institutions begin to look not for the best academic they can find but to prefer non-uk hires. The Russell Group performs well on this measure. The University of Sheffield performs well in the sector but not so well compared to the Russell Group. 20

Potential Teaching Excellence Framework Measures International Staff Top 5 % 1 London Business School 81.8 2 St Mary's University College 75.0 3 London School of Economics and Political Science 63.5 4 The School of Oriental and African Studies 53.5 5 Imperial College of Science, Technology and Medicine 50.1 Sheffield +/- 3 47 Heythrop College 29.4 48 The University of Reading 29.3 49 The University of Leicester 29.1 50 The University of Sheffield 28.6 51 The University of East London 28.5 52 University of Newcastle-upon-Tyne 27.7 53 Liverpool Hope University 27.6 Bottom 5 158 Leeds Trinity University 3.7 159 Newman University 3.3 160 Leeds College of Art 3.1 161 Bishop Grosseteste University 0.0 162 University of the Highlands and Islands 0.0 Russell Group % 3 London School of Economics and Political Science 63.5 5 Imperial College of Science, Technology and Medicine 50.1 7 The University of Oxford 42.0 8 The Queen's University of Belfast 41.4 10 University College London 40.4 13 The University of Cambridge 39.5 14 King's College London 39.4 17 The University of Warwick 38.6 18 Queen Mary University of London 38.3 25 The University of Edinburgh 35.7 28 University of Durham 35.2 34 The University of Southampton 32.1 35 University of Nottingham 31.8 38 The University of Birmingham 30.4 42 The University of York 30.0 43 The University of Glasgow 29.8 44 The University of Manchester 29.8 45 The University of Exeter 29.7 46 The University of Bristol 29.7 50 The University of Sheffield 28.6 52 University of Newcastle-upon-Tyne 27.7 58 The University of Liverpool 26.3 59 The University of Leeds 26.1 65 Cardiff University 22.6 Definition: Percentage of International Staff, including EU & OS (non-uk) Source: HEIDI 2013/14 21

The University of Sheffield 10. Percentage of full-time First Degree entrants no longer in Higher Education after a year A university or a subject that has a high student dropout (or non-retention) rate could be the sign of a difficult, challenging programme and tough assessment criteria or of poor quality admissions criteria. It could also be a sign of poor student support or poor teaching. As different student groups from particular backgrounds have different rates of retention, substantially influenced by the quality of those students, it is possible this measure is a proxy for student type rather than measuring any characteristic of the programme they are on. It is a measure open to unintended consequences: incentivising providers to keep students on programmes regardless of progress or suitability. If students seek to leave a programme for the right reasons; whether moving home, changing subject, moving institution or accepting a job offer, we wouldn t want to incentivise them staying on their course if it s no longer right for them. A very high or low retention rate might be a signal to investigate further. The measure used is a HESA Performance Indicator, which only measures retention after one year and not after subsequent years of study. The Russell Group performs well on this measure. The University of Sheffield performs well on this measure in the sector and in the Russell Group. 22

Potential Teaching Excellence Framework Measures Students no longer in HE after one year Top 5 % 1 The University of Buckingham 0.0 2 Royal Academy of Music 0.0 3 The University of Cambridge 1.4 4 The University of Oxford 1.4 5 University of Durham 1.6 Sheffield +/- 3 12 The University of Exeter 1.9 13 St Mary's University College 2.3 14 Courtauld Institute of Art 2.4 14 The University of Sheffield 2.4 14 University of Newcastle-upon-Tyne 2.4 17 The University of Southampton 2.5 18 The University of Birmingham 2.5 Bottom 5 148 London South Bank University 13.7 149 London Metropolitan University 14.0 150 The University of the West of Scotland 14.4 151 The University of Bolton 14.5 152 Heythrop College 15.2 Definition: Percentage of full-time first degree entrants who are no longer in HE after one year (young entrants only) Source: HEIDI Performance Indicators 2013/14 Russell Group % 3 The University of Cambridge 1.4 4 The University of Oxford 1.4 5 University of Durham 1.6 10 The University of Bristol 1.8 12 The University of Exeter 1.9 14 The University of Sheffield 2.4 14 University of Newcastle-upon-Tyne 2.4 17 The University of Southampton 2.5 18 The University of Birmingham 2.5 19 Cardiff University 2.6 20 The University of York 2.7 22 University of Nottingham 2.8 24 University College London 2.9 27 The University of Leeds 3.0 36 Imperial College of Science, Technology and Medicine 3.5 37 The University of Manchester 3.5 39 The University of Warwick 3.7 40 The University of Edinburgh 3.8 45 King's College London 4.0 47 The Queen's University of Belfast 4.1 48 The University of Liverpool 4.2 64 The University of Glasgow 5.0 68 London School of Economics and Political Science 5.2 76 Queen Mary University of London 5.7 23

The University of Sheffield 11. Students from LPN (low participation neighbourhoods) This measure, when combined with other measures, could indicate which institutions are best at teaching students from a diversity of backgrounds. Achieving excellence is about providing opportunity to a wide variety of people, not simply educating an elite group of students. However, the measure could be seen as making universities responsible for failures in earlier stages of the education system. It could incentivise the recruitment of students from LPN at the expense of students who might have more ability but who are not from a target area. It also risks penalising more selective institutions who find it harder to recruit students from LPNs by virtue of their higher entry requirements. On this measure the University does not do particularly well within the higher education sector, but is positioned near the top of the Russell Group. 24

Potential Teaching Excellence Framework Measures LPN Top 5 % 1 Teesside University 30.57 2 The University of Sunderland 29.26 3 University Campus Suffolk 27.93 4 The University of Bolton 25.68 5 Glyndwr University 25.28 Sheffield +/- 3 68 The Arts University Bournemouth 9.72 69 Buckinghamshire New University 9.30 70 The University of Leeds 8.97 71 The University of Sheffield 8.91 72 The University of Surrey 8.87 73 University of Abertay Dundee 8.84 74 Cardiff University 8.70 Bottom 5 149 Glasgow School of Art 2.48 150 The University of Oxford 2.25 151 Royal College of Music 1.64 152 Institute of Education 1.11 153 Harper Adams University 0.00 Definition: Percentage of first year undergraduate students from low participation neighbourhoods, defined by postcode; excluding unknowns Source: HESA sector data 2013/14 Russell Group % 63 The University of Liverpool 10.40 70 The University of Leeds 8.97 71 The University of Sheffield 8.91 74 Cardiff University 8.70 81 The University of Manchester 8.35 82 University of Newcastle-upon-Tyne 8.34 85 The University of Southampton 8.16 90 University of Nottingham 7.69 95 The University of York 7.16 99 The Queen's University of Belfast 7.06 102 The University of Birmingham 6.79 120 The University of Warwick 5.32 123 University of Durham 5.20 124 The University of Exeter 5.17 125 The University of Glasgow 4.93 127 Queen Mary University of London 4.41 128 University College London 4.39 131 London School of Economics and Political Science 4.12 132 King's College London 3.97 133 The University of Bristol 3.86 135 The University of Cambridge 3.82 136 The University of Edinburgh 3.76 141 Imperial College of Science, Technology and Medicine 3.52 150 The University of Oxford 2.25 25

The University of Sheffield 12. Applications per registration This measure could indicate how popular and desirable an institution is to applicants. A measure of desirability with potential students could signal the quality of its teaching provision. However, the number of applications could be affected by the entry requirements of an institution. Higher entry standards could deter potential students from applying as many are unlikely to be accepted onto a course. This measure could penalise more selective universities and incentivise institutions to lower their stated entry requirements in order to attract more applicants, which would mislead applicants and lead to wasted applications. The Russell Group performs well on this measure. The University of Sheffield performs well within the higher education sector, but is positioned in the lower third of the Russell Group. 26

Potential Teaching Excellence Framework Measures Applications per registration Top 5 1 The Royal Central School of Speech and Drama 26.43 2 The Liverpool Institute for Performing Arts 15.92 3 Rose Bruford College 14.22 4 London School of Economics and Political Science 12.36 5 The University of Aberdeen 11.71 Sheffield +/- 3 36 Leeds College of Art 6.32 37 The University of York 6.29 38 University of Durham 6.28 39 The University of Sheffield 6.23 40 The University of Leicester 6.17 41 The University of Brighton 6.17 42 Royal Holloway and Bedford New College 6.13 Bottom 5 136 The University of Sunderland 2.41 137 University of South Wales 2.38 138 University Campus Suffolk 2.34 139 University College Birmingham 1.92 140 University of the Highlands and Islands 1.11 Notes: There is more competition to gain a place at institutions higher up in the ranking Definition: Number of UCAS applications in 2013/14 to number of FT first year undergraduate students in 2013/14 Source: HEIDI 2013/14 Russell Group 4 London School of Economics and Political Science 12.36 7 The University of Edinburgh 9.58 9 The University of Bristol 8.58 10 King's College London 7.95 14 University College London 7.50 16 Queen Mary University of London 7.34 19 University of Nottingham 7.09 21 The University of Southampton 6.99 22 The University of Leeds 6.97 24 The University of Glasgow 6.93 26 The University of Manchester 6.83 29 The University of Warwick 6.61 30 The University of Birmingham 6.60 34 Imperial College of Science, Technology and Medicine 6.41 37 The University of York 6.29 38 University of Durham 6.28 39 The University of Sheffield 6.23 43 The University of Liverpool 6.13 48 The Queen's University of Belfast 5.78 51 The University of Exeter 5.69 52 Cardiff University 5.64 59 The University of Oxford 5.41 61 University of Newcastle-upon-Tyne 5.37 86 The University of Cambridge 4.75 27

The University of Sheffield 13. Proportion from state schools This measure, when combined with other measures, could indicate which institutions are best at teaching students from a diversity of backgrounds. Achieving excellence is about providing opportunity to a wide variety of people, not simply educating an elite group of students. However, the measure could be seen as making universities responsible for inequality in the opportunities provided to students in earlier stages of their education. It could incentivise the recruitment of students from state schools and the expense of students who might have ability but who are privately educated. It also risks penalising institutions who recruit more students from private schools by virtue of their higher entry requirements or because of their location in the country. The University of Sheffield performs less well within the higher education sector, but is positioned towards the top of the Russell Group. 28

Potential Teaching Excellence Framework Measures % State School Top 5 % 1 St Mary's University College 100.0 2 Stranmillis University College 100.0 3 University of Ulster 100.0 4 Institute of Education 99.5 5 Teesside University 99.5 Sheffield +/- 3 111 Heythrop College 87.3 112 The University of Reading 86.9 113 Queen Mary University of London 86.8 114 The University of Sheffield 86.5 115 Royal Conservatoire of Scotland 86.0 116 The University of Southampton 85.8 117 St George's Hospital Medical School 85.8 Bottom 5 149 Courtauld Institute of Art 58.5 150 The University of Oxford 57.4 151 Royal Agricultural University 55.4 152 Royal College of Music 51.3 153 Royal Academy of Music 32.9 Definition: Percentage of known UG home student population from state schools Source: HESA sector data 2013/14 Russell Group % 21 The Queen's University of Belfast 98.6 108 The University of Glasgow 88.2 110 The University of Liverpool 87.6 113 Queen Mary University of London 86.8 114 The University of Sheffield 86.5 116 The University of Southampton 85.8 118 Cardiff University 85.5 123 The University of Manchester 83.1 128 The University of York 81.4 129 The University of Birmingham 81.4 132 The University of Leeds 80.1 133 University of Nottingham 78.9 134 King's College London 78.2 136 The University of Warwick 76.6 138 University of Newcastle-upon-Tyne 75.4 139 London School of Economics and Political Science 72.0 140 University College London 71.0 141 The University of Edinburgh 71.0 142 The University of Exeter 70.1 144 Imperial College of Science, Technology and Medicine 65.9 145 University of Durham 62.0 146 The University of Cambridge 61.5 147 The University of Bristol 60.9 150 The University of Oxford 57.4 29

The University of Sheffield About the Data This data has been collected from the following sources: HESA sector data 2013/14 HEIDI 2013/14 NSS 2015 HEIDI Performance Indicators 2013/14 1. Average Tariff 2. UG Degree outcomes 4. Destination of leavers: Positive destinations 5. Salaries for new graduates 11. Students from LPN 13. Proportion from state sector 3. Student/Staff ratio 9. International staff 12. Applications per registration 6. NSS: Overall satisfaction 7. NSS: Teaching on my course 8. NSS: Learning resources 10. Students no longer in HE after 1 year For any queries about data definitions, please contact Catherine Spencer: c.m.spencer@sheffield.ac.uk 30

Potential Teaching Excellence Framework Measures 33