PISA 2012 Results: Briefing materials for Macao teachers Prof. Cheung Kwok Cheung Educational Testing and Assessment Research Centre, University of Macau This Powerpoint is excerpted and adapted from Andreas Schleicher s presentation on PISA 2012: Evaluating school systems to improve education, which can be download from www.pisa.oecd.org
2 PISA in brief Over half a million students representing 28 million 15-year-olds in 65 countries/economies took an internationally agreed 2-hour test Goes beyond testing whether students can reproduce what they were taught to assess students capacity to extrapolate from what they know and creatively apply their knowledge in novel situations Mathematics, reading, science, problem-solving, financial literacy Total of 390 minutes of assessment material and responded to questions on their personal background, their schools and their engagement with learning and school Parents, principals and system leaders provided data on school policies, practices, resources and institutional factors that help explain performance differences.
3 PISA in brief Key principles Crowd sourcing and collaboration PISA draws together leading expertise and institutions from participating countries to develop instruments and methodologies guided by governments on the basis of shared policy interests Cross-national relevance and transferability of policy experiences Emphasis on validity across cultures, languages and systems Frameworks built on well-structured conceptual understanding of academic disciplines and contextual factors Triangulation across different stakeholder perspectives Systematic integration of insights from students, parents, school principals and system-leaders Advanced methods with different grain sizes A range of methods to adequately measure constructs with different grain sizes to serve different decision-making needs Productive feedback, at appropriate levels of detail, to fuel improvement at every level of the system.
4 What do 15-year-olds know and what can they do with what they know? OECD, 2013
Shanghai-China Singapore Korea Estonia Macao-China Japan Finland Chinese Taipei Canada Liechtenstein Vietnam Poland Netherlands Denmark Ireland Austria Belgium Australia Latvia Slovenia Czech Republic Iceland United Kingdom Norway France New Zealand OECD average Spain Russian Fed. Luxembourg Italy Portugal United States Lithuania Sweden Slovak Republic Hungary Croatia Israel Greece Serbia Romania Turkey Bulgaria Kazakhstan U.A.E. Thailand Chile Malaysia Uruguay Montenegro Costa Rica Albania Argentina Brazil Tunisia Jordan Qatar Colombia Peru Indonesia 5 How proficient are students in mathematics? % 100 80 60 40 20 0 20 40 60 80 100 Level 6 Level 5 Level 4 Level 3 Level 2 Below Level 1 Level 1 At Level 6, students can conceptualise, generalise and utilise information based on their investigations and At modelling Level 5 students of complex can problem develop situations, and work and with can models use for their complex knowledge situations, in relatively identifying non-standard constraints contexts. and specifying They can link different information sources and representations and At assumptions. Level 4 students They can select, work effectively compare, with and explicit evaluate models flexibly translate among them. Students at this level are for appropriate complex problem-solving concrete situations strategies that may for involve dealing constraints with capable of advanced mathematical thinking and reasoning. or complex At call These Level for students 3 problems making students assumptions. related can can apply execute to these They this insight clearly models. can and described select Students and integrate this understanding, different level procedures, can representations, work along with a including strategically mastery those including using of symbolic that require broad, symbolic, and formal sequential well-developed linking thinking and reasoning skills, appropriate linked mathematical decisions. them At directly Their Level representations, operations interpretations to 2 students aspects of can and relationships, are real-world interpret sufficiently situations. and symbolic and to formal develop sound recognise Students characterisations, new to be situations approaches a base this for in contexts level building can that and insight and strategies a utilise simple require their pertaining for model no limited more to attacking for range than these situations. novel selecting direct of skills and can reason situations. and inference. applying They They begin Students simple can to reflect at this extract At with problem-solving Level some relevant 1 students insight, information straightforward contexts. They can on their level work can reflect strategies. can answer from can on formulate their actions, Students questions a single and communicate and at can this source involving formulate level and can familiar make their and interpret use contexts construct and of and communicate explanations and arguments interpretations precisely use a single where representations representational all relevant communicate and reasoning. their based information mode. actions different Students is present and reflections information and this the level can questions based sources employ their interpretations, arguments, and actions.or form regarding and are basic their reason clearly algorithms, defined. findings, directly interpretations, from formulae, They them. are able procedures, They to arguments, typically identify or and show the conventions information may some be unfamiliar. appropriateness ability and to to solve handle to carry problems of these percentages, out routine involving procedures to the original fractions whole situation. and numbers. according decimal They to are direct numbers, capable instructions and of making to work explicit literal with proportional situations. interpretations They relationships. of can the perform results. Their actions solutions that reflect are almost that they always have obvious engaged and in follow basic immediately from interpretation the given and stimuli. reasoning.
Mean score 580 570 560 550 High mathematics performance Shanghai-China performs above this line (613) Chinese Taipei Singapore Korea Average performance of 15-year-olds in Mathematics Fig I.2.13 540 530 520 510 500 490 480 470 460 450 440 430 420 410 Poland Belgium Austria Slovenia New Zealand Denmark Czech Republic France Luxembourg Latvia Portugal Spain Slovak Republic United States Hungary Israel Greece Romania Chile Macao-China Japan Liechtenstein Netherlands Estonia Finland Canada Viet Nam Australia Ireland United Kingdom Iceland Norway Italy Russian Fed. Lithuania Sweden Croatia Serbia Turkey Bulgaria U.A.E. Kazakhstan Thailand Malaysia Low mathematics performance Macao ranked sixth to eighth in the league table of PISA 2012 Mathematics Study. 12 countries perform below this line
High mathematics performance Chinese Taipei Singapore Korea Average performance of 15-year-olds in mathematics Strong socio-economic impact on student performance Poland Belgium Austria Slovenia New Zealand Denmark Czech Republic France Luxembourg Latvia Portugal Spain Slovak Republic United States Hungary Israel Macao-China Japan Liechtenstein Netherlands Estonia Finland Canada Viet Nam Australia Ireland United Kingdom Iceland Norway Italy Russian Fed. Lithuania Sweden Croatia Socially equitable distribution of learning opportunities Greece Romania Chile Serbia Turkey Bulgaria U.A.E. Kazakhstan Thailand Malaysia Low mathematics performance
Shanghai-China 2012 Singapore Chinese Taipei Korea Strong socio-economic impact on student performance 26 Slovak Rep. 24 Across 65 economies, Macao s basic educational system replicates the findings of previous cycles of assessment Japan Liechtenstein in succeeding to provide equitable schooling Estonia opportunities Netherlands for student body it served. Poland Belgium Canada Finland Viet Nam Denmark Austria New Zealand Australia Slovenia Ireland Czech Rep. Iceland 22France 20 18 16 14 12 10 8 6 4 UK Luxembourg Latvia Norway Portugal Italy US Russian Fed. Spain Lithuania Sweden Hungary Croatia Israel Macao-China Socially equitable distribution of learning opportunities 2 0 Chile Bulgaria Romania Greece Turkey Serbia United Arab Emirates Malaysia Kazakhstan Thailand
OECD average 2003 Norway Luxembourg Austria Spain Jordan United States Liechtenstein Japan Latvia Croatia Indonesia Estonia Thailand Macao-China Peru Korea Greece Colombia Russian Fed. Argentina Chinese Taipei Montenegro Chile Serbia Poland Italy Portugal Tunisia Turkey Dubai (UAE) Singapore Brazil Bulgaria Shanghai-China Israel Romania Albania U.A.E. * Malaysia Kazakhstan Qatar Sweden Finland New Zealand Czech Republic Iceland Australia Denmark Belgium Netherlands France Slovak Republic Uruguay Lithuania Canada Hungary Costa Rica Slovenia Ireland United Kingdom Average annual mathematics score change 9 Change in mathematics performance throughout participation in PISA: Annualised change Fig I.2.15 12 10 8 Countries/economies whose mathematics performance improved 6 4 Macao s mathematics performance average increases 1 point per year 2 0-2 -4-6 Countries/economies whose mathematics performance declined -8 * Excludes Dubai
Malaysia Sweden Slovenia Uruguay Finland Argentina Australia Iceland New Zealand Costa Rica Canada Ireland Czech Republic United States Jordan Spain Austria Netherlands Slovak Republic France Belgium Norway Denmark OECD average 2000 Bulgaria Italy Greece Luxembourg United Kingdom Macao-China Kazakhstan Korea Hungary Romania Russian Fed. Thailand Lithuania Brazil Croatia Liechtenstein Japan Portugal Latvia Indonesia Estonia Poland Dubai (UAE) Colombia Chile Israel Tunisia Albania Turkey Chinese Taipei Shanghai-China U.A.E.* Montenegro Peru Singapore Serbia Qatar Average annual reading score change 10 Change in reading performance throughout participation in PISA: Annualised change Fig I.4.3 12 10 8 6 4 2 Countries/economies whose reading performance improved Macao s reading performance average increases 1 point per year 0-2 -4-6 Countries/economies whose reading performance declined -8 * Excludes Dubai
Sweden Finland Slovak Republic New Zealand Uruguay Jordan Iceland Indonesia Hungary Canada Chinese Taipei Malaysia Greece Czech Republic Australia Slovenia Belgium Austria Costa Rica Netherlands Montenegro Croatia United Kingdom Denmark Liechtenstein OECD average 2006 France Luxembourg Russian Fed. Chile Norway Spain Lithuania Peru United States Serbia Estonia Macao-China Shanghai-China Colombia Bulgaria Latvia Albania Tunisia Brazil Ireland Argentina Dubai (UAE) Portugal Korea Japan Israel Italy Singapore Romania Thailand Poland U.A.E.* Qatar Turkey Kazakhstan Average annual science score change 11 Change in science performance throughout participation in PISA: Annualised change Fig I.5.3 12 10 8 Countries/economies whose science performance improved 6 4 Macao s science performance average increases nearly 2 points per year 2 0-2 -4-6 Countries/economies whose science performance declined -8 * Excludes Dubai
Average annual mathematics score change 12 Change in performance between PISA 2003 and 2012 Fig I.2.18 5 PISA 2003 performance below the OECD average PISA 2003 performance above the OECD average 4 3 2 1 0-1 -2-3 Tunisia Brazil Turkey growth in mathematics literacy Russian Fed. Thailand Greece performance from 2003 to 2012. Macao-China Korea Latvia Indonesia United States Japan Spain Austria Liechtenstein Luxembourg Norway OECD average Ireland Uruguay Portugal Italy Macao s students have made considerable Poland Slovak Republic Hungary France Denmark Iceland Belgium Czech Republic Sweden Canada Netherlands Australia New Zealand Finland Improving performance Deteriorating performance -4 350 400 450 500 550 600 Average mathematics performance in PISA 2003
13 Of the 65 countries 45 improved at least in one subject
14 Improvement in mathematics, reading or science Mathematics, reading and science Mathematics and reading Mathematics and science Israel, Poland, Portugal, Turkey, Brazil, Dubai (UAE),, Macao-China, Qatar, Singapore, Tunisia Chile,,, Albania, Montenegro, Serbia, Shanghai-China Italy, Kazakhstan, Romania Reading and science Japan, Korea, Latvia, Thailand Mathematics only Reading only Science only Greece, Bulgaria, Malaysia, United Arab Emirates (ex. Dubai) Estonia, Hungary, Luxembourg,, Colombia, Indonesia, Liechtenstein, Peru, Russian Federation, Chinese Taipei Ireland
Shanghai-China Singapore Chinese Taipei Korea Liechtenstein Macao-China Japan Belgium Netherlands Poland Canada Finland New Zealand Australia Estonia Austria Slovenia Viet Nam France Czech Republic OECD average United Kingdom Luxembourg Iceland Slovak Republic Ireland Portugal Denmark Italy Norway Israel Hungary United States Lithuania Sweden Spain Latvia Russian Federation Croatia Turkey Serbia Bulgaria Greece United Arab Emirates Romania Thailand Qatar Chile Uruguay Malaysia Montenegro Kazakhstan Albania Tunisia Brazil Peru Costa Rica Jordan Colombia Indonesia Argentina 15 % 100 90 Percentage of top performers in mathematics Tab I.2.1a 80 70 60 50 40 30 In Macao, there are about 25% of the students who are high performers in mathematical literacy. 20 10 0
Shanghai-China Singapore Korea Estonia Macao-China Japan Finland Chinese Taipei Canada Liechtenstein Viet Nam Poland Netherlands Denmark Ireland Austria Belgium Australia Latvia Slovenia Czech Republic Iceland United Kingdom Norway France New Zealand OECD average Spain Russian Federation Luxembourg Italy Portugal United States Lithuania Sweden Slovak Republic Hungary Croatia Israel Greece Serbia Romania Turkey Bulgaria Kazakhstan United Arab Emirates Thailand Chile Malaysia Uruguay Montenegro Costa Rica Albania Argentina Brazil Tunisia Jordan Qatar Colombia Peru Indonesia 16 % 100 90 Percentage of low-performing students in mathematics Tab I.2.1a 80 70 60 50 About 11% of the students performed at low level (i.e. below level 2) 40 30 20 10 0
Shanghai-China Singapore Japan Finland Australia New Zealand Estonia Netherlands Korea Canada United Kingdom Poland Ireland Liechtenstein Slovenia Belgium OECD average Chinese Taipei Luxembourg Viet Nam France Austria Czech Republic Norway United States Denmark Macao-China Sweden Italy Hungary Israel Iceland Lithuania Slovak Republic Spain Croatia Portugal Latvia Russian Federation Bulgaria United Arab Emirates Greece Turkey Serbia Qatar Uruguay Chile Thailand Romania Albania Montenegro Malaysia Brazil Jordan Argentina Costa Rica Kazakhstan Colombia Tunisia Peru Indonesia 17 % 100 90 Percentage of top performers in science Tab I.5.1a 80 70 60 50 There are 6.6% of the students who are high performers in scientific literacy. 40 30 20 10 0
Shanghai-China Estonia Korea Viet Nam Finland Japan Macao-China Poland Singapore Chinese Taipei Liechtenstein Canada Ireland Latvia Slovenia Netherlands Australia Czech Republic United Kingdom Spain Austria Lithuania New Zealand Denmark Croatia Belgium OECD average Hungary United States Italy France Russian Federation Portugal Norway Luxembourg Sweden Iceland Greece Turkey Slovak Republic Israel Thailand Chile Serbia United Arab Emirates Bulgaria Romania Costa Rica Kazakhstan Malaysia Uruguay Jordan Montenegro Argentina Albania Brazil Tunisia Colombia Qatar Indonesia Peru 18 % 100 90 Percentage of low-performing students in science Tab I.5.1a 80 70 60 50 About 9% of the students performed at low level (i.e. below level 2) 40 30 20 10 0
Shanghai-China Singapore Japan Korea New Zealand Finland France Canada Belgium Chinese Taipei Australia Ireland Liechtenstein Norway Poland Netherlands Israel Luxembourg United Kingdom OECD average Estonia United States Sweden Macao-China Italy Czech Republic Iceland Portugal Hungary Spain Austria Denmark Greece Slovenia Russian Federation Viet Nam Croatia Slovak Republic Turkey Bulgaria Latvia Lithuania Serbia United Arab Emirates Qatar Romania Albania Montenegro Uruguay Thailand Chile Costa Rica Argentina Brazil Peru Colombia Tunisia Jordan Malaysia Indonesia Kazakhstan 19 % 100 90 Percentage of top performers in reading Tab I.4.1a 80 70 60 50 40 There are 7% of the students who are high performers in reading literacy. 30 20 10 0
Shanghai-China Korea Estonia Viet Nam Ireland Japan Singapore Poland Canada Finland Macao-China Chinese Taipei Liechtenstein Netherlands Australia Denmark Belgium Norway New Zealand United States United Kingdom Czech Republic Latvia OECD average Spain Croatia Portugal France Austria Italy Hungary Iceland Slovenia Lithuania Turkey Luxembourg Russian Federation Greece Sweden Israel Slovak Republic Costa Rica Thailand Chile Serbia United Arab Emirates Romania Bulgaria Montenegro Uruguay Brazil Tunisia Jordan Colombia Albania Malaysia Argentina Indonesia Kazakhstan Qatar Peru 20 % 100 90 Percentage of low-performing students in reading Tab I.4.1a 80 70 60 50 About 11.4% of the students performed at low level (i.e. below level 2) 40 30 20 10 0
Korea + Liechtenstein Macao-China + Japan Belgium - Netherlands - Poland + Canada - Finland - New Zealand - Australia - Austria OECD average 2003 - France Czech Republic - Luxembourg Iceland - Slovak Republic Ireland Portugal + Denmark - Italy + Norway - Hungary United States Sweden - Spain Latvia Russian Federation Turkey Greece Thailand Uruguay - Tunisia Brazil Indonesia 21 Percentage of top performers in mathematics in 2003 and 2012 Fig I.2.23 % 2012 2003 40 30 20 Across OECD, 13% of students are top performers (Level 5 or 6). They can develop and work with models for complex situations, and work strategically with advanced thinking and reasoning skills. In Macao, close to a quarter of the adolescents are top performers. Between 2003 and 2012, Macao increased the share of top performers. 10 0
Korea Macao-China Japan Finland + Canada + Liechtenstein Poland - Netherlands + Denmark Ireland - Austria Belgium Australia + Latvia Czech Republic + Iceland + OECD average 2003 + Norway France + New Zealand + Spain Russian Federation - Luxembourg + Italy - Portugal - United States Sweden + Slovak Republic + Hungary + Greece Turkey - Thailand - Uruguay + Brazil - Tunisia - Indonesia 22 Percentage of low-performing students in mathematics in 2003 and 2012 Fig I.2.23 % 90 80 70 60 50 40 30 20 10 0 2012 2003 23% of students in OECD countries did not reach Level 2, i.e. have difficulties using basic algorithms, formulae, procedures or conventions to solve problems involving whole numbers. About 11% of Macao students are at this low level. Between 2003 and 2012, Macao only slightly reduced the share of low performers in mathematics.
23 Gender differences in the three literacies remain in a number of economies
Jordan Qatar Thailand Malaysia Iceland U.A.E. Latvia Singapore Finland Sweden Bulgaria Russian Fed. Albania Montenegro Lithuania Kazakhstan Norway Macao-China Slovenia Romania Poland Indonesia United States Estonia Chinese Taipei Shanghai-China Belgium Turkey Greece France Hungary Serbia Slovak Republic Vietnam Canada Netherlands OECD average Portugal Uruguay Croatia Israel Czech Republic Australia United Kingdom Argentina Denmark New Zealand Tunisia Ireland Spain Brazil Japan Korea Italy Peru Austria Liechtenstein Costa Rica Chile Luxembourg Colombia Score-point difference (boys-girls) 24 Gender differences in mathematics performance Fig I.2.25 30 20 Boys perform better than girls 10 0-10 -20 In PISA 2012, unlike previous cycles of PISA assessment, there is no longer any significant gender difference in mathematical literacy favouring males in the Macao s 15-year-old student population. -30 Girls perform better than boys -40-50
Jordan Qatar U.A.E. Bulgaria Thailand Montenegro Finland Latvia Lithuania Greece Malaysia Turkey Slovenia Kazakhstan Sweden Albania Argentina Russian Fed. Romania Serbia Norway Indonesia Iceland Poland France Estonia Croatia Portugal United States Macao-China Uruguay Israel Singapore Belgium Czech Republic Chinese Taipei Tunisia Viet Nam OECD average Brazil Italy Canada Hungary Netherlands Korea Ireland New Zealand Australia Shanghai-China Peru Chile Slovak Republic Spain Austria Denmark Japan Costa Rica United Kingdom Luxembourg Liechtenstein Colombia Score-point difference (boys-girls) 25 Gender differences in science performance Fig I.5.12 30 20 Boys perform better than girls 10 0-10 -20 In Macao PISA 2012, boys and girls perform similarly in science literacy and there is no gender gap favouring boys in science literacy performance. -30-40 Girls perform better than boys -50
Jordan Qatar Bulgaria Montenegro Finland Slovenia U.A.E. Lithuania Thailand Latvia Sweden Iceland Greece Croatia Norway Serbia Turkey Israel France Estonia Poland Romania Malaysia Russian Fed. Hungary Slovak Republic Portugal Italy Czech Republic Argentina OECD average Austria Kazakhstan Macao-China Uruguay Canada Australia New Zealand Chinese Taipei Singapore Belgium Viet Nam United States Denmark Tunisia Brazil Luxembourg Spain Ireland Indonesia Netherlands Costa Rica United Kingdom Liechtenstein Japan Shanghai-China Korea Chile Peru Colombia Albania Score-point difference (boys-girls) 26 Gender differences in reading performance Fig I.4.12 0-10 -20-30 -40-50 -60-70 In all countries and economies girls perform better than boys. In 2012, Macao has widen the gender gap in print reading performance from 34 to 36 in a 3-year-period. -80
27 Mathematical literacy for the 15- year-olds from PISA perspective: reason mathematically and understand, formulate, employ and interpret mathematical concepts, facts and procedures in a variety of problem solving contexts
Sweden Iceland Tunisia Argentina Brazil Luxembourg Ireland Netherlands New Zealand Costa Rica Austria Liechtenstein Malaysia Indonesia Denmark United Kingdom Uruguay Lithuania Australia Chile OECD average Slovak Republic Thailand Qatar Finland Portugal Colombia Peru Czech Republic Israel Italy Belgium Poland France Spain Montenegro Greece Turkey Slovenia Viet Nam Hungary Bulgaria Kazakhstan Chinese Taipei Canada United States Estonia Romania Latvia Serbia Japan Korea Croatia Albania Russian Federation United Arab Emirates Jordan Macao-China Singapore Shanghai-China Iceland Index of exposure to formal mathematics 28 Students' exposure to formal mathematics Fig I.3.1b 2.50 2.00 1.50 1.00 Like Shanghai, Macao students have very high exposure to formal mathematics. 0.50 0.00
Viet Nam Macao-China Shanghai-China Turkey Uruguay Greece Chinese Taipei Portugal Brazil Serbia Bulgaria Singapore Netherlands Japan Argentina Costa Rica Lithuania Tunisia New Zealand Czech Republic Israel Korea Latvia Qatar Italy United States Estonia Ireland Australia United Arab Emirates Norway Malaysia Kazakhstan United Kingdom Romania OECD average Albania Colombia Indonesia Sweden Belgium Peru Thailand Denmark Russian Federation Canada Slovak Republic Hungary Croatia Luxembourg Montenegro Chile Poland Finland Austria Slovenia France Jordan Liechtenstein Spain Iceland Index of exposure to word problems 29 Students' exposure to word problems Fig I.3.1a 2.50 2.00 1.50 1.00 Formal math situated in a word problem, where it is obvious to students what mathematical knowledge and skills are needed 0.50 Like Shanghai, Macao students have very low exposure to word problems. 0.00
Czech Republic Macao-China Shanghai-China Viet Nam Uruguay Finland Costa Rica Sweden Japan Chinese Taipei Italy Israel Norway Estonia Austria Serbia Korea Croatia Latvia Slovak Republic Greece United Kingdom Ireland Luxembourg Belgium Montenegro Argentina Slovenia Bulgaria OECD average Lithuania Hungary New Zealand Turkey Denmark Russian Federation Singapore Iceland United States Spain Qatar Liechtenstein Poland Australia France Brazil Malaysia Peru Canada Chile United Arab Emirates Romania Tunisia Netherlands Portugal Colombia Albania Kazakhstan Jordan Indonesia Thailand Index of exposure to applied mathematics 30 Students' exposure to applied mathematics Fig I.3.1c 2.50 2.00 1.50 1.00 0.50 Like Shanghai, Macao students have very low exposure to applied mathematics. 0.00
Mean score in mathematics 31 Relationship between mathematics performance and students' exposure to applied mathematics Fig I.3.2 510 490 470 450 OECD countries All participating countries and economies 430 0.0 0.5 1.0 1.5 2.0 2.5 3.0 never Macao students still have some room to increase exposure to applied mathematics from 1.6 to 1.8 for maximal mathematics performance rarely sometimes frequently Index of exposure to applied mathematics
Albania Tunisia Sweden Indonesia Slovenia Kazakhstan Iceland Argentina Romania Liechtenstein Costa Rica Turkey Jordan Brazil Belgium Montenegro Bulgaria Qatar Greece Thailand Italy Austria Hungary Peru Luxembourg Colombia Serbia Viet Nam Chile Croatia Uruguay France Lithuania United Arab Emirates Russian Federation Malaysia OECD average Czech Republic Netherlands Japan Poland Denmark Shanghai-China Slovak Republic Portugal Estonia Canada Latvia Israel Ireland Spain United States Macao-China Chinese Taipei Finland Australia New Zealand Korea United Kingdom Singapore Increase in PISA mathematics score associated with a one-unit increase in the index of exposure to formal mathematics 32 90 Relationship between the index of exposure to formal mathematics and students' mathematics performance Fig I.3.4c 80 70 60 50 Exposure to formal mathematics is positively associated with mathematics performance. This is especially so for Macao, Chinese Taipei, and Singapore. 40 30 20 10 0-10
Shanghai-China Macao-China Viet Nam Singapore Korea Chinese Taipei Japan Liechtenstein Estonia Netherlands Poland Canada Finland Belgium Portugal Turkey OECD average Italy Spain Latvia Ireland Australia Thailand Austria Luxembourg Czech Republic Slovenia United Kingdom Lithuania France Norway Iceland New Zealand Russian Fed. United States Croatia Denmark Sweden Hungary Slovak Republic Serbia Greece Israel Tunisia Romania Malaysia Indonesia Bulgaria Kazakhstan Uruguay Brazil Costa Rica Chile Colombia Montenegro U.A.E. Argentina Jordan Peru Qatar 33 Percentage of resilient students Fig II.2.4 20 18 16 A resilient student is situated in the bottom quarter of the PISA index of economic, social and cultural status (ESCS) in the country of assessment and performs in the top quarter of students among all countries, after accounting for socio-economic status. % 14 12 10 8 Socio-economically disadvantaged students not only score lower in mathematics, they also report lower levels of engagement, drive, motivation and self-beliefs. Resilient students break this link and share many characteristics of advantaged highachievers. 6 4 2 0 17% of students in Macao are resilient and they perform among the top 25% across all participating countries after taking ESCS into More than 10account. % resilient Between 5%-10% of resilient students Less than 5%
34 Disciplinary climate improved Teacher-student relations improved between 2003 and 2012 in all but one country (i.e. Tunisia); and disciplinary climate also improved during the period, on average across OECD countries and in 27 individual countries
Japan Luxembourg Norway Czech Republic Iceland Korea Indonesia Thailand Denmark Liechtenstein Italy Austria Macao-China Turkey Belgium Canada Portugal Poland Spain OECD average 2003 Brazil United States Greece Slovak Republic Netherlands Russian Federation Hungary Ireland New Zealand Australia Uruguay Sweden Latvia France Finland Tunisia Mean index change In most countries and economies, the disciplinary climate in schools improved between 2003 and 2012 Fig IV.5.13 Change between 2003 and 2012 in disciplinary climate in schools 0.5 0.4 0.3 0.2 0.1 Disciplinary climate is strongly associated with students engagement with and at school. Likewise, it is also strongly related to students intrinsic motivation to learn mathematics, and how anxious students reported themselves solving mathematics problems. Disciplinary climate improved 0-0.1-0.2 Disciplinary climate declined -0.3
36 Students' views of how conducive classrooms are to learning Fig IV.5.4 Percentage of students who reported that the following phenomena occur "never or hardly ever" or "in some lessons : Shanghai-China Macao-China Chinese Taipa OECD average Students don t listen to what the teacher says Shanghai has created an environment that is more conducive to effective learning and teaching There among is the noise four and disorder Chinese-speaking regions. The teacher has to wait a long time for students to quiet down. Students cannot work well Students don t start working for a long time after the lesson begins 0 20 40 60 80 100 %
37 Social and emotional dimensions matter too
Countries where students have stronger beliefs Fig III.4.5 38 in their abilities perform better in mathematics Mean mathematics performance OECD average Mean index of mathematics self-efficacy 650 Shanghai-China 600 550 500 450 400 350 Singapore Korea Chinese Taipei Japan Macao-China Netherlands Estonia Finland Canada Liechtenstein Belgium Poland Viet Nam Denmark Slovenia New Zealand Latvia Italy Portugal Austria Australia Russian Fed. Hungary Croatia Luxembourg Greece Slovak Republic Spain Turkey Israel Sweden Norway Serbia Lithuania Czech Republic U.A.E. United Kingdom Thailand Malaysia Romania Iceland Chile Bulgaria Kazakhstan Ireland United States Montenegro France Costa Rica Brazil Uruguay Albania Argentina Tunisia Colombia Qatar Jordan Indonesia Peru R² = 0.36 Macao students self-efficacy can be increased further closer to that of Hong Kong, Chinese Taipei or even Shanghai. Right now its mathematics performance is higher than what is predicated. 300-0.60-0.40-0.20 0.00 0.20 0.40 0.60 0.80 1.00 1.20
39 Students mathematics self-efficacy Fig III.4.2 Percentage of students who feel very confident or confident about having to do the foll owing tasks in mathematics: Shanghai-China Macao-China Chinese Taipei OECD average Calculating the petrol-consumption rate of a car Solving an equation like 2(x+3)=(x+3)(x-3) Students who believe they can solve mathem atics problems will become better at solving them. Macao students tend to believe that they can handle formal mathematical tasks better than applied mathematical tasks. Finding the actual distance between two places on a map with a 1:10 000 scale Solving an equation like 3x+5=17 Shanghai students have highest self efficacy among the four Chinese-speaking regions. Understanding graphs presented in newspapers Calculating how many square metres of tiles you need to cover a floor Calculating how much cheaper a TV would be after a 30% discount Using a <train timetable> to work out how long it would take to get from one place to another 40 50 60 70 80 90 100 %
Korea New Zealand Australia United Kingdom Finland Canada Czech Republic Sweden Lithuania Ireland Denmark Chinese Taipei Norway France Austria Spain Estonia Portugal OECD average United States Latvia Macao-China Liechtenstein Shanghai-China Iceland Greece Slovenia Hungary Japan Luxembourg Chile Poland Viet Nam Slovak Republic Singapore Russian Fed. Italy Belgium Netherlands Costa Rica Uruguay Croatia Turkey Israel Peru U.A.E. Serbia Tunisia Romania Jordan Argentina Bulgaria Malaysia Brazil Qatar Thailand Kazakhstan Indonesia Colombia Montenegro Albania Score-point difference 40 Students open to problem solving perform better Fig III.3.5 Score-point difference in mathematics associated with one unit of the index of students' openness to problem solving 60 50 Average student Change in performance per one unit of the index among lowest-achieving students Change in performance per one unit of the index among highest-achieving students Students who feel that they can handle a lot of information, seek explanations for things, can easily link facts together, and like to solve complex problems score 30 points higher in mathematics, on average 40 30 20 10 0-10
41 Openness to problem solving Fig III.3.4 Percentage of students who reported "agree" or "strongly agree" with the following statements: Shanghai-China Macao-China Chinese Taipei OECD average I like to solve complex problems I can easily link facts together Shanghai students are more willing to engage with problems and open to new I seek explanation for things challenges to solve complex problems among the four Chinese-speaking regions. Like Chinese Taipei there are around 25% of Macao students reported that they like to solve I am complex quick problems. to understand things I can handle a lot of information 0 10 20 30 40 50 60 70 %
42 Perceived self-responsibility for failure in mathematics Fig III.3.6 Percentage of students who reported "agree" or "strongly agree" with the following statements: Shanghai-China Macao-China Chinese Taipei OECD average Sometimes I am just unlucky Hong Kong students tend to attribute to The course material is too hard. Chinese Taipei s students tend The teacher did not get students interested in to attribute to I am not very good at solving mathematics problems. the material Interestingly, Macao students tend to attribute their self-responsibility for failure in mathematics to Teacher Sometimes did not get the students course interested material in is the too hard material. This week I made bad guesses on the quiz My teacher did not explain the concepts well this week I m not very good at solving mathematics problems 0 20 40 60 80 %
43 Students and perseverance Fig III.3.2 Percentage of students who reported that the following statements describe someone "very much like me" or "mostly like me" (*) or "not much like me" or "not at all like me" (**) Shanghai-China Macao-China Chinese Taipei OECD average Agree: When confronted with a problem, I do more than what is The index of perseverance expected masks of me differences across the four Chinese-speaking regions. Over 70% of Shanghai students indicated that Agree: they remain I continue interested working the tasks on tasks that they until start. everything is perfect Near 50% of Macao students indicated that when confronted with a problem, they do Agree: more than I remain what is expected interested of them. in the tasks Around 60% of Hong Kong students that I start indicated that they do not give up easily when confronted with a problem. Disagree: I put off difficult problems Disagree: When confronted with a problem, I give up easily 0 20 40 60 80
Korea Chinese Taipei Norway Finland Japan Denmark Sweden Iceland Greece Poland Australia Czech Republic United Kingdom Portugal Macao-China Estonia Canada Ireland France Shanghai-China Malaysia Viet Nam OECD average Spain Netherlands Liechtenstein Italy Latvia Slovenia Russian Fed. Austria Belgium Luxembourg New Zealand Hungary Lithuania United States Chile Croatia Jordan Turkey Qatar Tunisia Slovak Republic Singapore U.A.E. Serbia Thailand Montenegro Kazakhstan Costa Rica Uruguay Albania Israel Colombia Argentina Bulgaria Brazil Indonesia Romania Peru Score-point difference Fig III.3.13 44 Students who enjoy learning mathematics perform better 50 Score-point difference in mathematics associated with one unit of the index of intrinsic motivation to learn mathematics Average student Change in performance per one unit of the index among lowest-achieving students Change in performance per one unit of the index among highest-achieving students 40 30 20 10 0-10 -20-30
45 Students intrinsic motivation to learn mathematics Fig III.3.9 Percentage of students who reported "agree" or "strongly agree" with the following statements: Shanghai-China Macao-China Chinese Taipei OECD average I am interested in the things I learn in mathematics Intrinsic motivation refers to the drive to perform an activity purely for the joy gained from the activity I do itself. mathematics It affects because degree of student I enjoy it engagement in learning mathematics. Comparing the four Chinese-speaking regions, Shanghai students intrinsic motivation is the highest, followed by Hong Kong, Macao, and Chinese I look Taipei. forward to my mathematics lessons I enjoy reading about mathematics 0 10 20 30 40 50 60 70 %
Score-point difference Students who believe that learning mathematics Fig III.3.17 46 is useful perform better Korea Chinese Taipei Norway Finland Poland Japan Portugal Iceland Denmark Canada Sweden Australia New Zealand Spain Greece Qatar Malaysia Viet Nam Netherlands OECD average Estonia Belgium Lithuania United States France Luxembourg Jordan Thailand Tunisia Slovenia Hungary Shanghai-China Italy Latvia Ireland Czech Republic Macao-China Croatia United Kingdom U.A.E. Russian Fed. Turkey Chile Slovak Republic Israel Austria Bulgaria Serbia Montenegro Indonesia Kazakhstan Peru Argentina Costa Rica Brazil Uruguay Albania Singapore Colombia Liechtenstein Romania Score-point difference in mathematics associated with one unit of the index of instrumental motivation to learn mathematics 40 Average student Change in performance per one unit of the index among lowest-achieving students Change in performance per one unit of the index among highest-achieving students 30 20 10 0-10 -20
47 Students instrumental motivation to learn mathematics Fig III.3.14 Percentage of students who reported "agree" or "strongly agree" with the following statements: Shanghai-China Macao-China Chinese Taipei OECD average I will learn many things in mathematics that will help me get a job Instrumental motivation refers to the drive to learn mathematics because students perceive it as useful to them and to their future studies. Comparing the four Chinese-speaking regions, Shanghai students instrumental Mathematics is an important subject for me because I need it for what I want to study later on motivation is the highest, followed by Hong Kong, Macao, and Chinese Taipei. However, it is noteworthy that reduction in instrumental motivation to learn maths is significant in Macao, 0.18 points of this index when compared Learning mathematics is worthwhile for me because it will improve my career prospects and chances with that in 2003. Making an effort in mathematics is worth it because it will help me in the work that I want to do later on 0 20 40 60 80 100 %
Mean mathematics score Countries where students are more anxious about Fig III.4.14 48 mathematics perform less well 600 Shanghai-China 550 500 450 400 Singapore Chinese Taipei Korea Japan Macao-China Liechtenstein Canada Poland Estonia Netherlands Viet Nam Belgium Finland Slovenia Austria France Ireland Australia Czech Republic Denmark New Zealand Portugal UK Latvia Luxembourg Italy Spain Norway Iceland Russian Fed. Slovak Republic United States Sweden Croatia Hungary Lithuania Turkey Serbia Greece Israel Romania Bulgaria Thailand Chile Malaysia U.A.E. Kazakhstan Uruguay Costa Rica Montenegro Argentina Brazil Albania Tunisia Colombia Indonesia Jordan Qatar Peru R² = 0.31 350 0.80 More anxiety 0.60 0.40 0.20 0.00 Mean index of mathematics anxiety -0.20-0.40 Less anxiety -0.60
49 Students mathematics anxiety Fig III.4.10 Percentage of students who reported "agree" or "strongly agree" with the following statements: Shanghai-China Macao-China Chinese Taipei OECD average I worry that I will get poor <grades> in mathematics Higher level of anxiety is associated with lower students self efficacy, less openness to problem solving, less intrinsic and instrumental motivation to learn mathematics, lower mathematics self-concept and performance. Comparing the four Chinese-speaking regions, Chinese Taipei students anxiety is highest, followed by Macao, Hong Kong and Shanghai. Noteworthy for teachers is that Macao students anxiety has increased slightly since 2003 even this increase is considered as not significant. I feel helpless when doing a mathematics problem I get very nervous doing mathematics problems I get very tense when I have to do mathematics homework I often worry that it will be difficult for me in mathematics classes 0 20 40 60 80 100 %
50 Students' sense of belonging Fig III.2.12 Percentage of students who agree/disagree with the following statements: Shanghai-China Macao-China Chinese Taipei OECD average Agree: I am satisfied with my school Sense of belonging of students in Chinese speaking regions are lower than that of the average of OECD countries. Those students who attend schools with better teacher-student relations reported a stronger sense of belonging, and greater intrinsic motivation to learn mathematics. When students feel belonging to school, their engagement is always enhancing. Macao students sense of belonging has statistically significantly increased in magnitude when compared with 2003. Agree: Things are ideal in my school Agree: I feel happy at school Disagree: I feel lonely at school Agree: Other students seem to like me Disagree: I feel awkward and out of place in my school Agree: I feel like I belong at school Agree: I make friends easily at school Disagree: I feel like an outsider (or left out of things) at school 0 20 40 60 80 100 %
51 Students' mathematics self-concept Fig III.4.7 Percentage of students who agree*/disagree** with the following statements: Shanghai-China Macao-China Chinese Taipei OECD average Agree: In my mathematics class, I understand even the most difficult work Generally Agree: speaking, I have compared always believed with OECD that mathematics is one of my best subjects average, students mathematics concept regarding they get good grades in mathematics in the four Chinese-speaking economies are low. Agree: I learn mathematics quickly Admittedly, Macao students are already the highest amongst the four economies. Agree: I get good <grades> in mathematics Disagree: I am just not good at mathematics 0 20 40 60 80 %
52 Students' participation in mathematics-related activities Fig III.4.16 Percentage of students who reported "agree" or "strongly agree" with the following statements: Shanghai-China Macao-China Chinese Taipei OECD average I participate in a mathematics club I programme computers Amongst the four Chinese-speaking economies, Shanghai students generally are very active in a number of mathematics-related activities, I play such chess as talking about mathematics problems with friends, and do mathematics more than 2 hours I do mathematics more than 2 hours a day outside of school. outside of school I take part in mathematics competitions I do mathematics as an <extracurricular> activity I help my friends with mathematics I talk about mathematics problems with my friends 0 10 20 30 40 %
53 The parent factor Students whose parents have high educational expectations for them tend to report more perseverance, greater intrinsic motivation to learn mathematics, and more confidence in their own ability to solve mathematics problems than students of similar background and academic performance, whose parents hold less ambitious expectations for them.
Parents high expectations can nurture Fig III.6.11 54 students enjoyment in learning mathematics Mean index change Belgium (Flemish) Korea Italy Chile Portugal Hungary Croatia Macao-China Change in the index of intrinsic motivation to learn mathematics that is associated with parents expecting the child to complete a university degree 0.50 0.45 0.40 0.35 0.30 0.25 0.20 0.15 0.10 0.05 Parental expectation are strongly and positively associated with students intrinsic motivation to learn mathematics. Compare with the countries that distributed the parental questionnaire, Macao needs to enhance 15-year-olds parental expectation so as to nurture students enjoyment in learning mathematics. 0.00
Parents high expectations can foster Fig III.6.11 55 perseverance in their child Mean index change Portugal Italy Belgium (Flemish) Hungary Chile Croatia Korea Macao-China Change in the index of perseverance that is associated with parents expecting the child to complete a university degree 0.35 0.30 Parental expectation for their children s university study are strongly and positively associated with students perseverance. Macao needs to enhance 15-year-olds parental expectation further so as to foster their children s perseverance. 0.25 0.20 0.15 0.10 0.05 0.00
56 Schools make a difference
Japan Norway Iceland Russian Federation Thailand + Korea + Finland + Sweden Poland Greece - Denmark Czech Republic + New Zealand Australia - Slovak Republic + Canada - Latvia Ireland - Hungary Austria United States OECD average 2003 - Turkey - - Indonesia Italy - Liechtenstein Netherlands France - Spain + Portugal Luxembourg - Brazil Belgium + Uruguay Tunisia - Macao-China - Percentage of repeaters in 2003 and 2012 Tab IV.2.18 70 2012 2003 60 50 The grade repetition rate for Macao fell sharply from 2003 to 2012. But Macao s grade repetition rate is still highest amongst all PISA participating economies. 40 % 30 20 10 0
Slovak Republic Czech Republic France + Portugal - Greece Finland Poland Hungary + Belgium Sweden + Latvia Spain + Luxembourg + Uruguay - Macao-China + Russian Federation OECD average 2003 Tunisia Turkey Denmark Italy United States - Canada - Austria Netherlands Brazil - Liechtenstein New Zealand Australia - Ireland Indonesia - Thailand Score-point difference Change between 2003 and 2012 in the relationship between grade repetition and mathematics performance Fig IV.1.7 Score-point difference between students who had repeated a grade and those who hadn t 160 140 120 100 2012 2003 In PISA 2012, score-point difference in mathematics performance between Macao students who had repeated a grade and those who hadn t has widen when compared with PISA 2003. + 2012 higher than 2003-2012 lower than 2003 80 60 40 20 0
Netherlands Croatia Japan Thailand Serbia Viet Nam Hungary Singapore Bulgaria Liechtenstein Macao-China Luxembourg Austria U.A.E. Korea Indonesia Italy Albania Montenegro New Zealand Czech Republic Israel Malaysia Slovak Republic Shanghai-China Costa Rica Tunisia Qatar Chinese Taipei Kazakhstan Australia OECD average Turkey Colombia Canada Chile Estonia Portugal Jordan United States Romania France Peru Slovenia Latvia United Kingdom Uruguay Belgium Ireland Russian Fed. Iceland Brazil Lithuania Poland Argentina Denmark Sweden Greece Norway Spain Finland Most schools look at students past academic performance when considering admission Fig IV.1.6 100 90 80 70 Students in schools whose principals reported that "students' records of academic performance" or "recommendations of feeder schools" is always considered for admission In Macao, there are nearly 80% of students whose principals reported that students past academic performance is always considered for student admission. That of Hong Kong stands even higher at 95%. % 60 50 40 30 20 10 0
60 Money makes a difference but only up to a point
Chinese Taipei Greece Japan Korea Thailand Montenegro Turkey Shanghai-China Viet Nam Romania Macao-China Tunisia Croatia Hungary Malaysia New Zealand Ireland Liechtenstein Costa Rica Czech Republic Australia Bulgaria Netherlands Jordan Belgium Latvia Spain Argentina OECD average Indonesia Singapore Russian Fed. Austria Iceland France Brazil Uruguay Lithuania Israel Qatar Slovak Republic Canada Estonia U.A.E. Slovenia Serbia Italy Finland Colombia Chile United Kingdom Luxembourg United States Sweden Kazakhstan Portugal Peru Poland Denmark Norway In many countries, more advantaged than disadvantaged students attend after-school lessons Fig IV.3.11 Percentage of all students participating in after-school lessons Students in the bottom quarter of socio-economic status Students in the top quarter of socio-economic status 100 90 80 70 Comparing with western economies, advantaged students in East Asian economies tend to attend after-school lessons than their disadvantaged counterparts. Among the four Chinese-speaking regions, Macao has the lowest percentage of students attending after-school lessons 60 % 50 40 30 20 10 0
62 Adequacy of educational resources Fig IV.3.8 Percentage of students in schools whose principals reported that the following phenomena hindered student learning "not at all" or "very little : Shanghai-China Macao-China Chinese Taipei OECD average Shortage or inadequacy of science laboratory equipment Across the four Chinese-speaking regions, adequacy of educational resources is most favourable in Hong Kong, and least in Shanghai. That of Macao and Chinese Taipei is in-between. Shortage or inadequacy of instructional materials (e.g. textbooks) Shortage or inadequacy of computers for instruction Lack or inadequacy of Internet connectivity Shortage or inadequacy of computer software for instruction Shortage or inadequacy of library materials 50 60 70 80 90 100 %
Shanghai-China France Slovak Republic Macao-China Italy Qatar Czech Republic Israel Thailand Argentina Denmark Belgium Viet Nam U.A.E. United Kingdom Greece Indonesia Spain Chinese Taipei Singapore Japan Finland Uruguay Poland Sweden Australia New Zealand OECD average Netherlands Malaysia Austria Luxembourg Bulgaria Jordan Peru Iceland Portugal Brazil Turkey Romania Canada Norway Tunisia Lithuania Chile Serbia Korea United States Russian Fed. Costa Rica Kazakhstan Montenegro Colombia Croatia Slovenia Ireland Latvia Estonia Score point difference 63 Difference in mathematics performance, by attendance at pre-primary school Fig III.4.12 before accounting for students' socio-economic status after accounting for students' socio-economic status 140 120 100 80 Students who attended pre-primary school perform better. Between those who attended and who don t there is around 1.5 academic school year difference in mathematics performance in Macao. 60 40 20 0-20
64 Without data, you are just another person with an opinion. Andreas Schleicher
65 Find out more about PISA at www.pisa.oecd.org and www.umac.mo/fed/centre3(etarc).html All national and international publications The complete micro-level database Thank you!
66 References OECD (2013), PISA 2012 Results: What students know and can do Student performance in mathematics, reading and science (Volume I), PISA, OECD publishing. OECD (2013), PISA 2012 Results: Excellence through equity - Giving every student the chance to succeed (Volume II), PISA, OECD publishing. OECD (2013), PISA 2012 Results: Ready to learn - Students engagement, drive and self-beliefs (Volume III), PISA, OECD publishing. OECD (2013), PISA 2012 Results: What makes schools successful? Resources, policies and practices (Volume IV), PISA, OECD publishing. Schleicher, A. (2013). PISA 2012 Evaluating school systems to improve education. Available for download at: http://www.pisa.oecd.org