Benchmarking Tertiary Education Systems Jamil Salmi WB Tertiary Education Coordinator Bali, 6 June 2011
Romania: Coalition for Clean Universities iti 2007 survey - 77% of students and 35% of lecturers consider high level of corruption 14 coalition members from civil society, including students and professors associations
scoring sheet administrative probity 30 points academic probity 20 points democratic governance 35 points sound financial practice 15 points
integrity ranking universities divided from 5 stars to 0 no university received 5 stars 3 universities iti received 4 stars 18 universities received 3 stars 10 universities received 2 stars 5 universities received 1 star 6 universities received 0 stars ( you don t want to study there! )
meet my son Yacine 5
outline of the presentation what rankings fail to measure benchmarking of tertiary education systems 6
what do rankings measure? research? quantity or relevance? quality of teaching and learning? equality of opportunities? Oxbridge? engagement with the region? technology transfer? culture? values?
Victoria University multi sector University university (bachelors through PhD) vocational education (technical and trades education) further education(language, literacy numeracy; career support and planning; learning skills development) 20.7% of enrolments from low SES 1 in 2 students from non English speaking background majority are first in family to attend tertiary study
Victoria University and its region VU has a commitment to serve Western Melbourne enshrined in its Foundation Act to engage in research and education that makes a difference to peoples lives to work-based and community-based learning for 25% of every students course
absolute achievement and value added achie evemen nt A B C D E F highest h ranked greatest value added entry graduation 10
multi-dimension comparisons enriches the diagnosis i
what do rankings tell us about a country s performance? 12
top 50 universities (2010) CANADA, 2 JAPAN, 2 UK,,5 ARWU:2010 THES: 2010 WESTERN EUROPE, 5 JAPAN, 3 CANADA, 3 USA, 20 WESTERN EUROPE, 6 AUSTRALIA, 5 USA, 35 OTHER ASIA, 6 UK, 8
well-performing economies without world-class universities WEF WB K4D SJTU USA Denmark US (1) Switzerland Sweden UK (4) Denmark Finland Japan (19) Sweden Netherlands Switzerland (24) Singapore Norway Canada (24) Finland Canada France (42) Germany Switzerland Denmark (45) Netherlands UK Netherlands (47) Japan USA Sweden (51) Canada Australia Germany (55) 14
size effect
ARWU ranking related to population Country No. Top 500s population (000s) people required to produce each top 500 U Ireland 3 4,451 1,484 Denmark 4 5,565 1,391 Australia 17 22,327 1,313 Norway 4 4,883 1,221 Austria 7 8,382 1,197 Switzerland 7 7,790 1,113 Israel 7 7,577 1,082 Finland 6 5,363 894 New Zealand 5 4,371 874 Sweden 11 9,394 854 16
social mobility and inequality High Social Mo obility 100 90 80 70 60 50 40 30 Finland Norway Sweden Denmark Germany Canada Low 20 10 0 UK USA 0 10 20 30 40 50 60 70 80 90 100 Low High Income Inequality
what the rankings lens does not allow us to see overall performance of tertiary education systems access and equity quality and relevance contribution to local economic and social development
The United States doesn't have a world-class higher education system because it has many world-class universities; instead it has world-class universities because it has a worldclass higher education system. (Birnbaum, 2007)
outline of the presentation what rankings fail to measure benchmarking of tertiary education systems 20
what is benchmarking? comparing the performance of one s tertiary education system to that of other systems competitors good practices
purpose improving performance diagnosis (identification of areas for improvement) definition iti of specific corrective interventions ti 22
purpose (II) to understand the determinants of performance in complex systems and situations no consensus on what countries should do to improve their performance wide variations in system performance with similar funding levels and common country characteristics 23
comparing Brazil and Chile 40% 35% Chile 38% 30% Enrollm ment rate 25% 20% 15% 10% Brazil 24% 5% 0% 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 public spending as % of GDP 24
elaborating the theoretical framework distinction between performance (results) and health of system (determinants) how good are the system s actual outcomes? does it operate under conditions known to lead to high performance? informed by empirical evidence 25
what works in tertiary education: conceptual framework World Bank: Constructing Knowledge Societies (2002) OECD Synthesis of Tertiary Education Reviews (2007) Salmi: Challenge of Establishing World-Class Universities (2009) Aghion et al: Governance and Performance of Research Universities (2009)
political & economic stability, rule of law, basic freedoms telecommunications & digital infrastructure re results vision, leadership & reform capacity location attainment learning equity research technology transfer values governance & regulatory framework diversification, articulation & information mechanisms quality assurance & enhancement resources & incentives
levers of performance are like the steering mechanisms of an aircraft
it s all about alignment
2010 results drivers of performance 2000
three types of indicators quantitative qualitative - observed qualitative - interpreted objective measure objective description value judgement
examples of indicators (results) attainment achievement gap proportion of the workingage population (25-64) with a tertiary degree proportion from highest quintile over proportion from lowest quintile number of ranked quality universities per 100,000000 inhabitants
examples of indicators (results) research output technology transfer values number of citations per 100,000 inhabitants number of patents per 100,000 inhabitants proportion of voting age people who actually vote
examples of indicators (system health) regulatory framework institutional autonomy quality assurance legislation and QA requirements favorable to private institutions (Y/N) Board selects university leader (Y/N) proportion of accredited programs
examples of indicators (system health) financing allocation mechanisms resource utilization investment in tertiary education as proportion of GDP proportion of public funds allocated with performance criteria i average cost of a graduate
tertiary attainment rate in adult population (2010) Singapore Republic of Korea Japan Hong Kong SAR, China Philippines Thailand Mongolia Malaysia China Vietnam Lao People's Democratic Republic Indonesia Cambodia 7.0 6.6 5.4 3.2 3.1 1.5 0.7 13.7 25.5 25.2 37.7 47.3 66.6 0.00 10.00 20.00 30.00 40.00 50.00 60.00 70.00
growth of attainment rate between 1960 and 2010
tertiary attainment growth rates between 2000 and 2010 top 25% Korea; Thailand; Vietnam middle 50% Cambodia; China; Hong Kong, SAR, China; Japan; Malaysia; Philippines; Singapore bottom 25% Lao; Mongolia; Indonesia
what drives attainment?
expansion module No. Driver 1 secondary education completion rate 2 public and private spending on tertiary education as a percentage of GDP 3 share of private spending as a proportion of total spending on tertiary education 4 proportion of public spending, tertiary on total student aid (loans plus grants) 5 private enrolment share, tertiary (%) 6 proportion of students studying at non-university institutions (open university, polytechnics. etc) (%)
gross enrolment rate (2010) Korea, Rep. Japan Hong Kong SAR, China Mongolia Thailand Malaysia Philippines China Indonesia Lao PDR Cambodia 32.11 28.69 22.69 21.26 13.37 7.02 58.03 56.63 52.74 44.60 98.09 000 0.00 20.0000 40.0000 60.0000 80.0000 100.00 120.0000 42
secondary school completion rate vs. enrolment 100.00 90.0000 Korea 74.9 80.00 enrolment 70.00 60.00 50.00 40.00 Thailand 19.1 Mongolia Philippines 52.1 Japan 66.3 30.00 49.4 20.00 10.00 0.00 0 10 20 30 40 50 60 70 80 secondary school completion rate 43
total expenditure vs. enrolment 100.00 90.00 80.00 2.5 Korea en nrolment 70.00 60.00 50.00 40.00 Philippinesi Thailand 0.8 Japan 1.5 Mongolia 2.9 30.00 0.3 20.00 10.00 0.00 0.00 0.50 1.00 1.50 2.00 2.50 3.00 3.50 public and private spending on tertiary as % of GDP 44
public spending on student aid vs. enrolment 100.00 3.1 Korea Mongolia Japan enrolment 0.6 Philippinesi 21.5 7.2 0.00 0.10 1.00 10.00 100.00 public spending, tertiary on student aid, (%), log scale 45
private enrolment share vs. total enrolment 100.00 Korea 80.2 90.00 80.00 enrolment 70.00 60.00 50.00 40.0000 Thailand 17.5 Mongolia 34.3 Philippines Japan 79.2 30.00 65.9 20.00 10.00 0.00 0.00 10.00 20.00 30.00 40.00 50.00 60.00 70.00 80.00 90.00 private enrolment share (%) 46
differentiation vs. enrolment 100.00 90.00 24.1 Korea 80.00 enrolment 70.00 60.00 50.00 40.0000 Mongolia 2.6 Philippinesi Japan 21.9 Thailand 43.5 30.00 9.6 20.00 10.00 0.00 0.00 5.00 10.00 15.00 20.00 25.00 30.00 35.00 40.00 45.00 50.00 enrolment in non-university it institutions (%of total t enrolment) 47
key messages combination of factors drive enrolment each country has own strategy for increasing enrolment high levels of expenditure does not necessarily correspond to higher enrolment figures countries with high enrolment rates (Japan, Korea, Philippines) have larger degree of institution diversity (non university institutions and private enrolment share) 48
benchmarking different type of analysis system-wide rather than institutional multi-dimensional i l alignment of key dimensions (of performance and system health) actionable policy levers informed decisions avoiding complacency (like rankings)