1 (PISA) How the world s school systems perform - and why this matters OECD s 2006 PISA survey of the knowledge and skills of 15-year-olds Paris, 20 February 2008 Barbara Ischinger OECD Directorate for Education
2 This session 1. Key priorities in education What the OECD seeks to achieve What PISA measures and why 2. Where we are and where we can be Where countries stand in terms of quality, equity and efficiency in education What the best performing countries show can be achieved 3. How we can get there Some policy levers that emerge from international comparisons Why we need to care.
Briefing of Council 14 November 2007 3 Key priorities in education What the OECD seeks to achieve What PISA measures and why
4 What the OECD seeks to achieve Developing effective policies that help Raising teaching and learning effectiveness Teacher development, quality of educational workforce Innovation in curriculum and pedagogy Efficiency and effectiveness of education systems Closing the skills gap Meeting changes in the demand and supply of skills (both with regard to quantity and nature of skills), etc. Raising quality and relevance of higher education Internationalisation, changing roles of institutions, governance issues, etc Promoting social mobility and inclusive education Distribution of resources and outcomes, integration of immigrants, special needs etc.
Mean task input as percentiles of the 1960 task distribution 5 The world keeps changing do our school systems? Economy-wide measures of routine and non-routine task input (US) 65 Routine manual 60 55 50 45 Nonroutine manual Routine cognitive Nonroutine analytic 40 1960 1970 1980 1990 2002 Nonroutine interactive (Levy and Murnane)
6 Assessing educational performance looking back at what students were expected to have learned or looking ahead to how well they can extrapolate from what they have learned and apply their knowledge and skills in novel settings. For PISA, the OECD countries chose the latter.
7 PISA The OECD Programme for A three-yearly global assessment that examines the performance of 15-year-olds in key subject areas as well as a wider range of educational outcomes Including students attitudes to learning and their learning behaviour collects contextual data from students, parents, schools and systems in order to identify policy levers Coverage Representative samples of between 3,500 and 50,000 15-year-old students drawn in each country Most federal countries also draw regional samples PISA covers roughly 90% of the world economy.
8 PISA countries in 2001 2003 2006 2009 1998 2000 Coverage of world economy 77% 81% 83% 85% 86% 87%
Briefing of Council 14 November 2007 9 Key findings from PISA 2006 Where we are how students perform across countries Where we can be the top performers
United States Czech Republic PISA Estonia Germany Switzerland Denmark Canada Norway Sweden Russian Federation4 Austria3 Slovenia Israel Slovak Republic New Zealand Hungary Finland United Kingdom3 Netherlands Luxembourg EU19 average OECD average France Australia Iceland Belgium Poland Ireland Korea Chile2 Greece Italy Spain Turkey Portugal Mexico Brazil2 10 School completion A world of change in the global skill supply Approximated by percentage of persons with high school or equivalent qualfications in the age groups 55-64, 45-55, 45-44 und 25-34 years 1990s 1980s 1970s 1960s % 100 90 80 70 13 1 60 50 40 30 20 10 0 1 27 1. Excluding ISCED 3C short programmes 2. Year of reference 2004 3. Including some ISCED 3C short programmes 3. Year of reference 2003.
11 Moving targets Future supply of high school graduates 14,000,000 12,000,000 10,000,000 8,000,000 6,000,000 4,000,000 2003 2010 2015 2,000,000 0 China EU India US
12 High science performance 565 Chinese Taipei Estonia Liechtenstein United Kingdom Czech Republic Macao-China Ireland France Iceland United States Norway Portugal Finland 545 Hong Kong-China Russian Federation Italy Greece Average performance of 15-year-olds in science extrapolate and apply Canada Japan New Zealand Australia 525 Netherlands Korea Slovenia Germany Switzerland Austria Belgium 505 Hungary Sweden Denmark Poland Croatia Latvia Slovak Republic, Spain, Lithuania Luxembourg 485 465 Israel 445 16 18 countries perform below this line Low science performance 6
13 High average performance Large socio-economic disparities Strong socioeconomic impact on student performance High science performance 565 Chinese Taipei Estonia Liechtenstein United Kingdom Czech Republic Macao-China Ireland France Iceland United States Norway Portugal Finland 545 Hong Kong-China Russian Federation Italy Greece Average performance of 15-year-olds in science extrapolate High social equity and apply High average performance Canada Japan New Zealand Australia 525 Netherlands Korea Slovenia Germany Switzerland Austria Belgium 505 Hungary Sweden Denmark Poland Croatia Latvia Slovak Republic, Spain, Lithuania Luxembourg 485 Socially equitable distribution of learning opportunities 465 Low average performance Large socio-economic disparities 16 Israel 445 Low science performance Low average performance High social equity 6
14 High average performance Large socio-economic disparities Strong socioeconomic impact on student performance High science performance Durchschnittliche High average performance Schülerleistungen im High social equity Bereich Mathematik Chinese Taipei Canada New Zealand Estonai Japan Australia Netherlands Liechtenstein Korea Slovenia 520 Germany United Kingdom Czech Republic Switzerland Belgium Austria Ireland Hungary Sweden 500 Poland France Denmark Croatia United States Latvia Iceland Slovak Republic Lithuania Spain Luxembourg Norway 560 540 Hong Kong-China Finland Macao-China Socially equitable distribution of learning opportunities Portugal 480 Russian Federation Italy Greece Low average performance Large socio-economic disparities 460 Low Israel average performance High social equity 22 440 Low science performance 12 2
15 600 500 400 Immigrants and student performance The challenges international migration poses to education systems can be successfully addressed Native students First-generation students Second-generation students Education in the host country can make a difference OECD average = 500 The challenges won t go away Native by students themselves 550 Second-generation students In some countries second-generation students perform equally poor than their first generation counterparts Schools should do better in building on the emotive capital of immigrant students as a driving source for 450 enhancing their learning Much of this motivational potential dissipates as students stay First-generation students longer in the country 350 Institutional barriers (selection/tracking) tend to reinforce socio-economic disparities 300 Early diagnosis and intervention pay Language is a key foundation Immersion combined with support can work. PISA 2006: Science Competencies for Tomorrow s World, Figure 4.2a.
New Zealand Finland United Kingdom Australia Japan Canada OECD average Portugal Italy Turkey Mexico United States Korea 16 % 100 80 60 Top and bottom performers in science These students can consistently identify, Level 1explain Below and apply Level scientific 1 Level knowledge, 2 Level link 3 Level 4 Level 5 Level 6 different information sources and explanations and use evidence from these to justify decisions, demonstrate advanced scientific thinking in unfamiliar situations 40 20 0 These students often confuse key features of a scientific investigation, apply incorrect information, mix personal beliefs with facts in support of a position 530 563 515 527 531 534 500 474 475 424 410 489 522 20 40 60 Large proportion of top performers Large prop. of poor perf.
17 Top performers matter Excellence in education and countries research intensity Number of researchers per thousand employed, full-time equivalent 18 17 16 15 14 13 Finland y = 1.695x 0.604 R² = 0.70 12 11 10 9 8 7 6 5 4 3 2 1 0 Sweden New Zealand Japan Norway United States Denmark France Belgium Australia Germany Canada Luxembourg Korea Switzerland Ireland Spain Austria United Kingdom GreeceSlovak Poland Netherlands Portugal RepublicHungary Czech Republic Italy Turkey Mexico 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 Percentage of students at Levels 5 or 6 in the PISA assessment
19 Investments and outcomes Since 2000, expenditure per primary and secondary student increased across OECD countries by 39% (in real terms) while PISA outcomes generally remained flat but there are notable exceptions
OECD 2006 Poland 2000 Poland 2003 Poland 2006 PISA 20 Poland raised its reading performance by 28 PISA points, equivalent to ¾ of a school year - What happened? % 90 Level 4 Level 3 Level 2 Below Level 1 Level 1 70 Between 2000 and 2003, performance variation among schools fell from 51% to 16% of the variation of student performance 50 30 Between 2000 and 2003 showed the secondlargest increase in reading (17 points) and a further 11 point increase since 2003 But did this lead to genuine improvements of school performance? 10 10 30 Most of that increase resulted from smaller proportions at the bottom level (23% in 2000, and three-quarters in vocational tracks, 17%in 2003) Did this harm the better performers? OECD (2007), Learning for tomorrow s world: First results from PISA 2006, Table 6.1a
21 Student attitudes 15-year-olds generally value science Recognition of economic and social benefits but somewhat less so when it concerns themselves only a minority expresses interest in a scientific career, on average across the OECD 37% would like to work in a career involving science 31% would like to study science after school 27% would like to work in a science project later Close relationships with performance and socioeconomic background.
Denmark Switzerland France Finland Sweden Canada United States Belgium Austria Germany Portugal United Kingdom Netherlands Hungary PISA Spain Slovak Republic Ireland Czech Republic Luxembourg Poland Greece Italy 24 A second chance? Expected hours in non-formal job-related training (2003) This chart shows the expected number of hours in non-formal job-related education 1400 1200 % and training, over a forty year period, for 25-to-64 year olds. All levels of education Lower secondary education Upper secondary and post-secondary non-tertiary education Tertiary education 1000 800 600 400 200 0
Briefing of Council 14 November 2007 25 Some levers for policy that emerge from OECD s comparisons
26 High ambitions and universal standards Access to best practice and quality professional development
27 Challenge and support Strong support Poor performance Improvements idiosyncratic Strong performance Systemic improvement Low challenge High challenge Poor performance Stagnation Conflict Demoralisation Weak support
28 High ambitions Devolved responsibility, the school as the centre of action Accountability and intervention in inverse proportion to success Access to best practice and quality professional development
29 School autonomy, standards-based examinations and science performance School autonomy in selecting teachers for hire 70 PISA score in science 60 50 40 41 30 20 10 Yes 0 No School autonomy in selecting teachers for hire Yes No Standards based external examinations
30 Strong ambitions Integrated educational opportunities Accountability Devolved responsibility, the school as the centre of action From prescribed forms of teaching and assessment towards personalised learning Access to best practice and quality professional development
31 High ambitions Integrated educational opportunities Accountability and intervention in inverse proportion to success Access to best practice and quality professional development Devolved responsibility, the school as the centre of action Personalized learning
32 Progress Why care? Concerns about skill barriers to economic growth, productivity growth and rates of technological innovation One additional year of education equals to between 3 and 6% of GDP Rising college-level qualifications seem generally not to have led to an inflation of the labour-market value of qualifications (in all but three of the 20 countries with available data, the earnings benefit increased between 1997 and 2003, in Germany, Italy and Hungary by between 20% and 40%) Fairness Concerns about the role of skills in creating social inequity in economic outcomes Both average and distribution of skill matter to long-term growth Value for money Concerns about the demand for, and efficiency and effectiveness of, investments in public goods
33 www.oecd.org; www.pisa.oecd.org All national and international publications The complete micro-level database and remember: Thank you! Without data, you are just another person with an opinion
Briefing of Council 14 November 2007 PISA 34 [Backup slides]
Greece Germany Finland1 Japan Korea Norway Ireland Czech Republic4 Switzerland Poland Denmark United Kingdom EU19 average Hungary Slovak Republic OECD average Italy Iceland Sweden Luxembourg United States Spain New Zealand Portugal Turkey Mexico 35 High school completion rates Percentage of graduates to the population at the typical age of graduation % 100 90 2005 1995 80 70 60 50 40 30 20 10 0 1. Year of reference 2004.
Australia1 Iceland1 New Zealand1 Finland1,2 Denmark1 Poland Netherlands1 Italy Norway1 United Kingdom Ireland Sweden1 OECD average Hungary Japan EU19 average Israel1 United States Spain1 Portugal1 Slovak Republic1 Switzerland Greece Czech Republic4 Austria1 Germany1 Slovenia1 Turkey PISA 36 College-level graduation rates Percentage of tertiary type A graduates to the population at the typical age of graduation % 70 60 2005 1995 50 40 15 2 Decline of the relative position of the US from 1995 to 2005 30 20 10 0 A3.1 1. Net graduation rate is calculated by summing the graduation rates by single year of age in 2005. 2. Year of reference 2004.
-35-30 -25-20 -15-10 -5 0 5 10 15 20 25 30 35 PISA 37 Gender differences in science performance United States OECD average Overall Girls do better Boys do better Identifying scientific issues Explaining phenomena scientifically Using scientific evidence scale Knowledge about science scale Earth and space systems scale Living systems scale Physical systems scale PISA score points OECD (2007), PISA 2006 Science Competencies for Tomorrow s World, Tables 2.1c, 2.2c, 2.3c, 2.4c, 2.7, 2.8, 2.9, 2.10
Student performance 38 School performance and socio-economic background Germany 700 Student performance and students socio-economic background within schools School performance and schools socio-economic background Student performance and students socio-economic background Schools proportional to size 500 300-3 -2-1 0 1 2 3 Disadvantage PISA Index of socio-economic background Advantage
Student performance 39 School performance and socio-economic background United States Student performance and students socio-economic background within schools School performance and schools socio-economic background Student performance and students socio-economic background Schools proportional to size Disadvantage PISA Index of socio-economic background Advantage
Student performance 40 School performance and socio-economic background Finland Student performance and students socio-economic background within schools School performance and schools socio-economic background Student performance and students socio-economic background Schools proportional to size Disadvantage PISA Index of socio-economic background Advantage
41 % Luxembourg Japan Italy Switzerland Finland Denmark Czech Republic Sweden Hungary Austria Portugal United States Netherlands Slovak Republic Korea Ireland Spain Canada Mexico New Zealand Germany OECD United Kingdom Government schools Government dependent private Public and private schools Government independent private Observed performance difference Difference after accounting for socio-economic background of students and schools 0 20 40 60 80-150 100-100 -50 0 50 100 Score point difference Private schools perform better Public schools perform better
42 Score point difference in science Pooled international dataset, effects of selected school/system factors on science performance after accounting for all other factors in the model 100 64% of US 90 students in schools that 80 compete with more than 2 schools in same area, 11% with 70 one school, 26% with no school 60 50 40 30 91% of US students in Approx. one schools posting 20 school achievement year data publicly (OECD 38%) 10 26% of US 0students in schools with no vacant science teaching 10 positions (OECD 38%), 20 71% where all vacant positions had been filled (OECD 30 59%), BUT 20% where principals report that instruction is hindered by a lack of qualified science teachers Measured effect Gross School principal s positive evaluation of quality of Schools with more educational materials competing schools (gross only) (gross only) Schools with greater autonomy (resources) (gross and net) School activities to promote One additional science hour learning of self-study (gross or and homework net) One additional hour of (gross and net) science learning at school School (gross results and net) posted publicly Academically (gross selective and net) schools (gross and net) but no system-wide effect Schools practicing ability One grouping additional (gross hour and of net) outof-school additional lessons 10% of Each School (gross public principal s funding and net) perception (gross that only) lack of Effect after accounting qualified teachers hinders instruction for the socio-economic (gross only) background Net of students, schools and countries OECD (2007), PISA 2006 Science Competencies from Tomorrow s World, Table 6.1a
43 High average performance Large socio-economic disparities New Zealand Netherlands High science performance Germany Czech Republic United Kingdom Belgium Austria Switzerland Strong socioeconomic impact on Poland Sweden 500 Ireland Hungary student performancefrance Denmark Slovak Republic United States Spain Luxembourg Portugal 560 540 520 480 Greece Durchschnittliche Finland High average performance Schülerleistungen im High social equity Bereich Mathematik Canada Australia Japan Korea Socially equitable distribution of learning opportunities Iceland Norway Italy Early selection and institutional differentiation Low average performance High degree of stratification Large Low degree socio-economic of stratification disparities 460 440 Low average performance High social equity Low science performance
44 Increased likelihood of postsec. particip. at age 19 associated with reading proficiency at age 15 (Canada) after accounting for school engagement, gender, mother tongue, place of residence, parental, education and family income (reference group Level 1) 20 18 16 14 12 10 8 6 4 2 0 Level 2 Level 3 Level 4 Level 5