1 () Science competencies for tomorrow s world Seeing education in Norway through the prism of Oslo, 10 December 2007 Andreas Schleicher Head, Indicators and Analysis Division OECD Directorate for Education
2 Today 1. OECD s Programme for () What measures and why 2. Where we are and where we can be Where countries stand in terms of quality, equity and efficiency in education What the best performing countries show can be achieved 3. How we can get there Some policy levers that emerge from international comparisons
3 OECD s Programme for International Student Assessment () What measures and why How works Briefing of Council 14 November 2007
4 A three-yearly global assessment that examines the performance of 15-year-olds in key subject areas as well as a wider range of educational outcomes Including students attitudes to learning and their learning behaviour collects contextual data from students, parents, schools and systems in order to identify policy levers Coverage Representative samples of between 3,500 and 50,000 15-year-old students drawn in each country Most federal countries also draw regional samples covers roughly 90% of the world economy.
5 countries in 2001 2003 2006 2009 1998 2000 Coverage of world economy 77% 81% 83% 85% 86% 87%
6 How works r A strong international network of expertise among the participating countries From establishing the assessment frameworks The assessments include tasks from more than 40 countries developing the instruments Cross-national and cross-cultural validity to analysing and interpreting the results National, regional and international analyses and reports In-depths country peer reviews supported by a consortium of leading research institutions ACER, CITO, ETS, NIER, WESTAT co-ordinated through the OECD in collaboration with other international organisations.
7 Key findings from 2006 Where we are how students perform across countries Where we can be the top performers How we can get there some school and system factors Briefing of Council 14 November 2007
8 2006 r The latest assessment emphasizes science competencies, defined in terms of an individual s: Scientific knowledge and use of that knowledge to identify scientific issues, explain scientific phenomena, and draw evidence-based conclusions about science-related issues Understanding of the characteristic features of science as a form of human knowledge and enquiry Awareness of how science and technology shape our material, intellectual and cultural environments Willingness to engage with science-related issues r A large proportion of complex open-ended tasks.
9 Deciding what to assess... looking back at what students were expected to have learned or looking ahead to how well they can extrapolate from what they have learned and apply their knowledge and skills in novel settings. For, the OECD countries chose the latter.
10 How the demand for skills has changed Economy-wide measures of routine and non-routine task input (US) (Levy and Murnane) Mean task input as percentiles of the 1960 task distribution
11 Quality in learning outcomes Science performance Briefing of Council 14 November 2007
12 High science performance 565 Chinese Taipei Estonia Liechtenstein United Kingdom Czech Republic Macao-China Ireland France Iceland United States Norway Portugal Finland 545 Hong Kong-China Russian Federation Italy Greece Israel Average performance of 15-year-olds in science extrapolate and apply Canada Japan New Zealand Australia 525 Netherlands Korea Slovenia Germany Switzerland Austria Belgium 505 Hungary Sweden Denmark Poland Croatia Latvia Slovak Republic,Spain, Lithuania Luxembourg 485 465 445 16 18 countries perform below this line Low science performance 6
13 Strengths and weaknesses of countries in science relative to their overall performance France Science competencies Overall science score France=495 Identifying scientific issues Explaining phenomena scientifically Using scientific evidence Science knowledge Knowledge about science Earth and space Living systems Physical systems -35-25 -15-5 5 15 25 35 OECD (2007), 2006, Figure 2.13
14 Strengths and weaknesses of countries in science relative to their overall performance Czech Republic Scientific competencies Overall science score France=495 Czech Republic=512 Identifying scientific issues Explaining phenomena scientifically Using scientific evidence Scientific knowledge Knowledge about science Earth and space Living systems Physical systems -35-25 -15-5 5 15 25 35 OECD (2007), 2006, Figure 2.13
15 Strengths and weaknesses of countries in science relative to their overall performance Norway Science competencies Overall science score Norway=487 Identifying scientific issues Explaining phenomena scientifically Using scientific evidence Science knowledge Knowledge about science Earth and space Living systems Physical systems -35-25 -15-5 5 15 25 35 OECD (2007), 2006, Figure 2.13
16 Gender differences in science performance Overall Identifying scientific issues Explaining phenomena scientifically Norway Girls do better OECD average Boys do better Using scientific evidence scale Knowledge about science scale Earth and space systems scale Living systems scale Physical systems scale -35-30 -25-20 -15-10 -5 0 5 10 score points 15 20 25 30 35 OECD (2007), 2006, Tables 2.1c, 2.2c, 2.3c, 2.4c, 2.7, 2.8, 2.9, 2.10
17 Distribution of student performance Briefing of Council 14 November 2007
18 Top and bottom performers in science These students can consistently identify, explain and apply scientific knowledge, link different information sources and explanations and use evidence from these to justify decisions, demonstrate advanced scientific thinking in unfamiliar situations These students often confuse key features of a scientific investigation, apply incorrect information, mix personal beliefs with facts in support of a position Large proportion of top performers Large prop. of poor perf.
19 Top performers matter Excellence in education and countries research intensity
20 Money matters - but other things do too
21 Investments and outcomes r Since 2000, expenditure per primary and secondary student increased across OECD countries by 39% (in real terms) while outcomes generally remained flat but there are notable exceptions
22 Poland raised its Between reading 2000 performance and by 28 points, 2003 Poland delayed the equivalent to ¾ of a school year - What happened? separation of students into different school types beyond the age of 15 years In 2003, performance variation among schools Between 2000 and 2003 had fallen from 51% to showed the secondlargest increase in 16% of the variation of student performance reading (17 points) and a further 11 point increase But did this lead to since 2003 genuine improvements of school performance? Most of that increase resulted from smaller proportions at the bottom level (23% in 2000, and three-quarters in vocational tracks, 17%in 2003) Did this harm the better performers? OECD (2007), Learning for tomorrow s world: First results from 2006, Table 6.1a
23 Students attitudes to science and their awareness of the life opportunities science may open Briefing of Council 14 November 2007
24 Student attitudes r 15-year-olds generally value science Recognition of economic and social benefits but somewhat less so when it concerns themselves only a minority expresses interest in a scientific career Close relationships with performance and socioeconomic background
25 Students generally value science but report stronger belief in the technological potential of science than in its capacity to make social improvements Norway Science is important for helping us to understand the natural world Advances in science and technology usually improve people's living conditions Science is valuable to society OECD average Advances in science and technology usually help to improve the economy Advances in science and technology usually bring social benefits Change in science performance per unit of the index 0 25 50 75 100 % Score points 0 20 40 OECD (2007), 2006, Figure 3.2
26 but somewhat less so when it concerns themselves Norway I find that science helps me to understand things around me I will use science in many ways when I am an adult Some concepts in science help me see how I relate to other people OECD average When I leave school there will be many opportunities for me to use science Science is very relevant to me Change in science performance per unit of the index 0 25 50 75 100 % Score points 0 20 40 OECD (2007), 2006, Figure 3.4
27 and only a minority report interest in a scientific career Norway I would like to work in a career involving science I would like to study science after secondary school I would like to work on science projects as an adult I would like to spend my life doing advanced science Change in science performance per unit of the index OECD average 0 25 50 75 100 % Score points -20-10 0 10 20 30 40 OECD (2007), 2006, Figure 3.13
28 Science and the environment r r r 58% of Norwegian 15-year-olds report familiarity and knowledge of the increase of greenhouse gases in the atmosphere, 77% with consequences of clearing forests for other land use, 51% with issues around pollution and acid rain (large variation across countries) Awareness of environmental issues is closely linked with students science performance and with their social background Norwegian students express a below-average concern for environmental issues and a below-average sense of personal responsibility for sustainable development Like in other countries, only a minority are optimistic that the issues will be successfully addressed and the more students know and the better they perform in science, the less optimistic they are
29 Some degree of pessimism among students about the future of the natural environment problems associated with the areas below will improve over the next 20 years Norway Energy shortages Water shortages Air pollution OECD average Nuclear waste Extinction of plants and animals Clearing of forests for other land use Change in science performance per unit of the index 0 25 50 75 100 % Score points -40-30 -20-10 0 OECD (2007), 2006, Figure 3.20
30 Equity in educational opportunities Briefing of Council 14 November 2007
31 High average performance Large socio-economic disparities Strong socioeconomic impact on student performance Low average performance Large socio-economic disparities High science performance 565 Chinese Taipei Estonia Liechtenstein United Kingdom Czech Republic Macao-China Ireland France Iceland United States Norway Portugal 16 Finland 545 Hong Kong-China Israel Low science performance Average performance of 15-year-olds in science extrapolate High social equity and apply High average performance Canada Japan New Zealand Australia 525 Netherlands Korea Slovenia Germany Switzerland Austria Belgium 505 Hungary Sweden Denmark Poland Croatia Latvia Slovak Republic,Spain, Lithuania Luxembourg 485 Russian Federation Italy Greece 465 445 Socially equitable distribution of learning opportunities Low average performance High social equity 6
32 High average performance Large socio-economic disparities Strong socioeconomic impact on student performance Low average performance Large socio-economic disparities High science performance Portugal Durchschnittliche High average performance Schülerleistungen im High social equity Bereich Mathematik Hong Kong-China Chinese Taipei Canada New Zealand Estonai Japan Australia Netherlands Liechtenstein Korea Slovenia 520 Germany United Kingdom Czech Republic Switzerland Belgium Austria Ireland Hungary Sweden 500 Poland France Denmark Croatia United States Latvia Iceland Slovak Republic Lithuania Spain Luxembourg Norway 560 540 480 Russian Federation Italy Greece 460 Finland Socially equitable distribution of learning opportunities Low Israel average performance High social equity Macao-China 22 440 Low science performance 12 2
33 School performance and socio-economic background Germany Student performance Student performance and students socio-economic background within schools School performance and schools socio-economic background Student performance and students socio-economic background Schools proportional to size Disadvantage Index of socio-economic background Advantage
34 School performance and socio-economic background Norway Student performance Student performance and students socio-economic background within schools School performance and schools socio-economic background Student performance and students socio-economic background Schools proportional to size Disadvantage Index of socio-economic background Advantage
35 School performance and socio-economic background Finland Student performance Student performance and students socio-economic background within schools School performance and schools socio-economic background Student performance and students socio-economic background Schools proportional to size Disadvantage Index of socio-economic background Advantage
36 Coherence of educational standards across schools Briefing of Council 14 November 2007
37 140 120 100 80 60 40 20 0 Variation in student performance Germany Czech Republic Austria Hungary Netherlands Belgium Japan Italy Greece Slovak Republic Turkey Switzerland Korea Luxembourg United States Portugal Mexico United Kingdom New Zealand Australia Canada Ireland Denmark Spain Poland Sweden Norway Iceland Finland OECD (2007), Learning for tomorrow s world: First results from 2006, Table 4.1a
38 100 80 60 40 20 0-20 -40-60 -80 Variation in student performance Variation of performance within schools Variation of performance between schools Germany Czech Republic Austria Hungary Netherlands Belgium Japan Italy Greece Slovak Republic Turkey Switzerland Korea Luxembourg United States Portugal Mexico United Kingdom New Zealand Australia Canada Ireland Denmark Spain Poland Sweden Norway Iceland Finland OECD (2004), Learning for tomorrow s world: First results from 2003, Table 4.1a
39 Some levers for policy that emerge from OECD s comparisons Briefing of Council 14 November 2007
40 Some myths r No relationship between size of countries and average performance r No relationship between proportion of immigrants and average performance r Few difference in students reported test motivation r Limited impact of national item preferences.
41 High ambitions and universal standards Access to best practice and quality professional development
42 Challenge and support Poor performance Improvements idiosyncratic Low challenge Poor performance Stagnation Strong support Strong performance Systemic improvement Conflict High challenge Demoralisation Weak support
43 High ambitions Accountability and intervention in inverse proportion to success Access to best practice and quality professional development Devolved responsibility, the school as the centre of action
44 Luxembourg Japan Italy Switzerland Finland Denmark Czech Republic Sweden Hungary Austria Portugal United States Netherlands Slovak Republic Korea Ireland Spain Canada Mexico New Zealand Germany OECD United Kingdom % Government schools Public and private schools Government dependent private Government independent private Observed performance difference Difference after accounting for socio-economic background of students and schools 0 20 40 60 80-150 100-100 -50 0 50 100 Score point difference Private schools perform better Public schools perform better
45 Pooled international dataset, effects of selected school/system factors on science performance after accounting for all other factors in the model 64% of US students in schools that compete with more than 2 schools in same area, 11% with one school, 26% with no school 91% of US students in schools posting achievement data publicly (OECD 38%) 26% of US students in schools with no vacant science teaching positions (OECD 38%), 71% where all vacant positions had been filled (OECD 59%), BUT 20% where principals report that instruction is hindered by a lack of qualified science teachers Measured effect School principal s positive evaluation of quality of Schools with more educational materials competing schools (gross only) (gross only) Schools with greater autonomy (resources) (gross and net) School activities to promote One additional science hour learning of self-study (gross or and homework net) One additional hour of (gross and net) science learning at school School (gross results and net) posted publicly Academically (gross selective and net) schools (gross and net) but no system-wide effect Schools practicing ability grouping One additional (gross and hour net) of Each out-of-school additional lessons 10% of School (gross public principal s funding and net) perception (gross that only) lack of qualified teachers hinders instruction (gross only) Effect after accounting for the socio-economic background of students, schools and countries OECD (2007), 2006 from Tomorrow s World, Table 6.1a
46 score in science School autonomy, standards-based examinations and science performance School autonomy in selecting teachers for hire
47 School autonomy, standards-based examinations and science performance School autonomy in deciding on budget allocation within the school score in science
48 Integrated educational opportunities Accountability Strong ambitions Devolved responsibility, the school as the centre of action From prescribed forms of teaching and assessment towards personalised learning Access to best practice and quality professional development
49 High average performance Large socio-economic disparities New Zealand Netherlands Germany Czech Republic United Kingdom Belgium Austria Switzerland Strong socioeconomic impact on Poland Sweden 500 Ireland Hungary student performancefrance Denmark Slovak Republic United States Spain Luxembourg Portugal Early selection and institutional differentiation Low average performance High degree of stratification Low degree of stratification Large socio-economic disparities High science performance 560 540 520 480 Greece 460 440 Durchschnittliche Finland High average performance Schülerleistungen im High social equity Bereich Mathematik Canada Australia Japan Korea Socially equitable distribution of learning opportunities Iceland Norway Italy Low average performance High social equity Low science performance
50 Integrated educational opportunities Accountability and intervention in inverse proportion to success High ambitions Access to best practice and quality professional development Devolved responsibility, the school as the centre of action Personalized learning
51 A second chance? Expected hours in non-formal job-related training (2003) This chart shows the expected number of hours in non-formal job-related education and training, over a forty year period, for 25-to-64 year olds. 1400 1200 % All levels of education Lower secondary education Upper secondary and post-secondary non-tertiary education Tertiary education 1000 800 600 400 200 0 Denmark Switzerland France Finland Sweden Canada United States Belgium Austria Germany Portugal United Kingdom Netherlands Hungary Spain Slovak Republic Ireland Czech Republic Luxembourg Poland Greece Italy C5.1a
52 r r r Progress Why care? Concerns about skill barriers to economic growth, productivity growth and rates of technological innovation One additional year of education equals to between 3 and 6% of GDP Rising college-level qualifications seem generally not to have led to an inflation of the labour-market value of qualifications (in all but three of the 20 countries with available data, the earnings benefit increased between 1997 and 2003, in Germany, Italy and Hungary by between 20% and 40%) Fairness Concerns about the role of skills in creating social inequity in economic outcomes Both average and distribution of skill matter to long-term growth Value for money Concerns about the demand for, and efficiency and effectiveness of, investments in public goods
53 www.oecd.org; www.pisa.oecd.org All national and international publications The complete micro-level database email: pisa@oecd.org Thank you! Andreas.Schleicher@OECD.org and remember: Without data, you are just another person with an opinion
54 Backup slides Briefing of Council 14 November 2007
55 Effort expended by students in 2003 (Butler and Adams, 2007)
56 Effort expended by students in 2003, relative to an important school test (Butler and Adams, 2007)
57 Ranks comparisons: Overall vs favourites Rank on own most appropriate items 0 5 10 15 20 25 30 30 Rank on favourites higher than overall rank Norway 13 th overall 10 th on favourites 25 GRC ESP ITA CZE HUN 20 DNK NOR FRA ISL 15 Rank on all items AUS NZL AUT SWE JPN KOR BEL Korea 3 rd overall 9 th on favourites For all other countries, the ranks were not significantly different. CANFIN Rank on favourites lower than overall rank 10 5 0
58 How the demand for skills has changed Economy-wide measures of routine and non-routine task input (US) (Levy and Murnane) Mean task input as percentiles of the 1960 task distribution
59 Increased likelihood of postsec. particip. at age 19 associated with reading proficiency at age 15 (Canada) after accounting for school engagement, gender, mother tongue, place of residence, parental, education and family income (reference group Level 1) 20 18 16 14 12 10 8 6 4 2 0 Level 2 Level 3 Level 4 Level 5