What Makes Schools and School Systems Successful OECD EMPLOYER BRAND Lessons for the GCC States Playbook from PISA 2012 Tue Halgreen 4 March 2015 1
2 PISA in brief Over half a million students representing 28 million 15-year-olds in 65 countries/economies took an internationally agreed 2-hour test Goes beyond testing whether students can reproduce what they were taught to assess students capacity to extrapolate from what they know and creatively apply their knowledge in novel situations Mathematics, reading, science, problem-solving, financial literacy Total of 390 minutes of assessment material and responded to questions on their personal background, their schools and their engagement with learning and school Parents, principals and system leaders provided data on school policies, practices, resources and institutional factors that help explain performance differences.
3 The structure of the PISA assessment 2000 2003 2006 2009 2012 Reading Reading Reading Reading Reading Mathematics Mathematics Mathematics Mathematics Mathematics Science Science Science Science Science Problem Solving Digital Reading Problem Solving, Financial literacy, Digital Math, Digital reading
4
3 Mean performance in mathematics PISA 2012 Fig I.2.13 Mean score 580 570 560 550 540 530 520 510 500 490 480 470 460 450 440 430 420 410 400 390 380 370 360 350 Brazil High mathematics performance Shanghai-China is above this level (613) Chinese Taipei Poland Belgium Germany Austria Slovenia New Zealand France Portugal Spain United States Hungary Israel Greece Romania Chile Tunisia Indonesia Singapore Hong Kong-China Korea Japan Switzerland Netherlands Estonia Finland Canada Viet Nam Australia Ireland United Kingdom Norway Italy Russian Fed. Lithuania Sweden Croatia Serbia Turkey Bulgaria U.A.E. Kazakhstan Thailand Malaysia Mexico Jordan Colombia Qatar Peru Argentina Low mathematics performance
6 Qatar and UAE Trends in PISA Mathematics 2006 2009 2012 Qatar 318 368 376 United Arab Emirates 411 423 Science 2006 2009 2012 Qatar 349 379 384 United Arab Emirates 429 439 Reading 2006 2009 2012 Qatar 312 372 388 United Arab Emirates 423 432 Figures in bold indicate a statistically significant annualised change
Variation in student performance as % of OECD average variation Albania Finland Iceland Sweden Norway Denmark Estonia Ireland Spain Canada Poland Latvia Kazakhstan United States Mexico Colombia Costa Rica Russian Fed. Malaysia Jordan New Zealand Lithuania Greece Montenegro United Kingdom Argentina Australia Brazil Portugal Indonesia Chile Thailand Romania Tunisia Switzerland Peru Uruguay Croatia U.A.E. Macao-China Serbia Viet Nam Korea Hong Kong-China Singapore Austria Italy Luxembourg Czech Republic Japan Bulgaria Israel Qatar Shanghai-China Germany Slovenia Slovak Republic Turkey Belgium Hungary Liechtenstein Netherlands Chinese Taipei 7 Variability in student mathematics performance between and within schools Fig II.2.7 100 80 60 40 20 Performance differences between schools OECD average 0 20 40 Performance variation of students within schools 60 80 100 OECD average
Student performance 8 School performance and socio-economic background: Finland Student performance and students socio-economic background School performance and schools socio-economic background Student performance and students socio-economic background within schools 700 494 200-3 -2-1 0 1 2 3 Disadvantage PISA Index of socio-economic background Advantage
Student performance 9 School performance and socio-economic background: Qatar Student performance and students socio-economic background School performance and schools socio-economic background Student performance and students socio-economic background within schools 700 494 200-3 -2-1 0 1 2 3 Disadvantage PISA Index of socio-economic background Advantage
Student performance 10 School performance and socio-economic background: United Arab Emirates Student performance and students socio-economic background School performance and schools socio-economic background Student performance and students socio-economic background within schools 700 494 200-3 -2-1 0 1 2 3 Disadvantage PISA Index of socio-economic background Advantage
Student performance 11 School performance and socio-economic background: United Arab Emirates Student performance and students socio-economic background School performance and schools socio-economic background Student performance and students socio-economic background within schools 700 494 200-3 -2-1 0 1 2 3 Disadvantage PISA Index of socio-economic background Advantage
Qatar U.A.E. Australia Thailand Macao-China Montenegro Israel Jordan United States Serbia Hungary Hong Kong-China Singapore Slovak Republic Canada New Zealand Chinese Taipei Lithuania Kazakhstan Malaysia Ireland Turkey United Kingdom Croatia Luxembourg Latvia Chile Costa Rica Argentina Russian Fed. Czech Republic OECD average Germany Slovenia Greece Norway Estonia Iceland Italy Austria Netherlands Spain France Portugal Sweden Liechtenstein Denmark Switzerland Belgium Brazil Finland Mexico Colombia Peru Shanghai-China Score point difference 12 Differences in mathematics performance between students without and with an immigrant background Fig II.3.4 before accounting for students' socio-economic status after accounting for students' socio-economic status 150 100 Students without an immigrant background perform better 50 0-50 Students with an immigrant background perform better -100
13 math teaching math teaching PISA = reason mathematically and understand, formulate, employ and interpret mathematical concepts, facts and procedures
Viet Nam Macao-China Shanghai-China Turkey Uruguay Greece Hong Kong-China Chinese Taipei Portugal Brazil Serbia Bulgaria Singapore Netherlands Japan Argentina Costa Rica Lithuania Tunisia New Zealand Czech Republic Israel Korea Latvia Qatar Italy United States Estonia Ireland Australia Mexico United Arab Emirates Norway Malaysia Kazakhstan United Kingdom Romania OECD average Albania Colombia Indonesia Sweden Belgium Peru Thailand Denmark Russian Federation Canada Slovak Republic Hungary Germany Croatia Luxembourg Montenegro Chile Poland Finland Austria Slovenia France Switzerland Jordan Liechtenstein Spain Iceland Index of exposure to word problems 14 Students' exposure to word problems Fig I.3.1a 2.50 2.00 1.50 1.00 Formal math situated in a word problem, where it is obvious to students what mathematical knowledge and skills are needed 0.50 0.00
Sweden Iceland Tunisia Argentina Switzerland Brazil Luxembourg Ireland Netherlands New Zealand Costa Rica Austria Liechtenstein Malaysia Indonesia Denmark United Kingdom Uruguay Lithuania Germany Australia Chile OECD average Slovak Republic Thailand Qatar Finland Portugal Colombia Mexico Peru Czech Republic Israel Italy Belgium Hong Kong-China Poland France Spain Montenegro Greece Turkey Slovenia Viet Nam Hungary Bulgaria Kazakhstan Chinese Taipei Canada United States Estonia Romania Latvia Serbia Japan Korea Croatia Albania Russian Federation United Arab Emirates Jordan Macao-China Singapore Shanghai-China Iceland Index of exposure to formal mathematics 15 Students' exposure to conceptual understanding Fig I.3.1b 2.50 2.00 1.50 1.00 0.50 0.00
Czech Republic Macao-China Shanghai-China Viet Nam Uruguay Finland Costa Rica Sweden Japan Chinese Taipei Italy Israel Norway Estonia Hong Kong-China Austria Serbia Korea Croatia Latvia Slovak Republic Greece United Kingdom Ireland Luxembourg Belgium Montenegro Argentina Slovenia Bulgaria OECD average Lithuania Hungary Switzerland New Zealand Germany Turkey Denmark Russian Federation Singapore Iceland United States Spain Qatar Liechtenstein Poland Australia France Brazil Malaysia Peru Canada Chile United Arab Emirates Romania Tunisia Netherlands Portugal Colombia Albania Kazakhstan Jordan Mexico Indonesia Thailand Index of exposure to applied mathematics 16 Students' exposure to applied mathematics Fig I.3.1c 2.50 2.00 1.50 1.00 0.50 0.00
Mean score in mathematics 17 Relationship between mathematics performance and students' exposure to applied mathematics Fig I.3.2 510 490 470 450 430 0.0 0.5 1.0 1.5 2.0 2.5 3.0 never rarely sometimes frequently Index of exposure to applied mathematics
Mathematics performance (score points) Countries that grant schools autonomy over curricula and assessments tend to perform better in mathematics Fig IV.1.15 650 Shanghai-China 600 550 500 450 400 Chinese Taipei Viet Nam Korea Singapore Estonia Hong Kong-China Japan Latvia Poland Slovenia Czech Rep. Switzerland Belgium Canada Portugal Germany Finland New Zealand Lithuania Croatia Austria Hungary Netherlands Serbia Spain France Australia Italy UK Turkey Norway Macao-China Greece Bulgaria Denmark Iceland Thailand Kazakhstan Romania Slovak Rep. R² = 0.13 Israel Malaysia Uruguay USA Sweden Chile Jordan Costa Rica Brazil Indonesia Luxembourg Tunisia Albania Colombia UAE Argentina Peru 350 Qatar 300-1.5-1 -0.5 0 0.5 1 1.5 Index of school responsibility for curriculum and assessment (index points)
Schools with more autonomy perform better than schools with less autonomy in systems with more accountability arrangements Fig IV.1.16 School autonomy for curriculum and assessment x system's level of posting achievement data publicly Score points 478 476 474 472 470 468 466 464 School data public Less school autonomy School data not public More school autonomy
Mathematics performance (score points) Money makes a difference but only up to a point Fig IV.1.8 650 Shanghai-China Cumulative expenditure per student less than USD 50 000 Cumulative expenditure per student USD 50 000 or more 600 550 500 450 400 350 Viet Nam Turkey Tunisia Brazil Jordan Colombia Peru Japan Switzerland Poland Canada Finland Netherlands Germany Belgium Australia Austria New Zealand Latvia Slovenia Ireland Denmark France Portugal Iceland UK Norway Slovak Republic Israel Italy Sweden United States Hungary Spain Estonia Czech Republic Lithuania Croatia Thailand Bulgaria Malaysia Chile Mexico Montenegro Uruguay R² = 0.37 Korea Singapore R² = 0.01 Luxembourg 300 0 20 000 40 000 60 000 80 000 100 000 120 000 140 000 160 000 180 000 200 000 Average spending per student from the age of 6 to 15 (USD, PPPs)
Mathematics performance (score points) Countries with better performance in mathematics tend to allocate educational resources more equitably Fig IV.1.11 700 Adjusted by per capita GDP 650 600 550 500 450 400 30% of the variation in math performance across OECD countries is explained by the degree of similarity of educational resources between advantaged and disadvantaged schools Mexico Costa Rica Peru Shanghai-China Chinese Taipei Viet Nam Korea Singapore R² = 0.19 Hong Kong-China Estonia Japan Poland Switzerland Slovenia Canada Latvia Finland Belgium Macao-China Germany New Zealand Slovak UK Rep. Ireland Iceland France Austria Australia Denmark Spain Romania Croatia Sweden Israel Hungary USA Turkey Greece Bulgaria Portugal Italy Norway Thailand Serbia Chile Malaysia Uruguay Kazakhstan Brazil Jordan Indonesia UAE Montenegro Colombia Argentina Tunisia Luxembourg 350 Qatar 300 1.5 1 0.5 0-0.5 Less equity Equity in resource allocation (index points) Greater equity
PISA-Based Test for Schools Overview Based on international PISA test of 15-year olds. All results are comparable to international PISA scales Can be used by schools, networks of schools and districts Goes beyond testing whether students can reproduce what they were taught to assess students capacity to apply their knowledge in novel situations Provides information on students engagement and the learning environment at the school
PISA-Based Test for Schools What does the assessment look like? Experience for students similar to that of the main PISA tests: 2h test +30min questionnaire Three assessment domains: reading, maths, science Student sample size per school: 85 A comprehensive (150 pages) school report for each participating school
Find out more about PISA at www.pisa.oecd.org All national and international publications The complete micro-level database Thank you! Email: tue.halgreen@oecd.org