Determinants of Learning in PISA 2000 & 2003 Third Monterrey International Conference on Science in Basic Education March 18, 2005 Harry Anthony Patrinos The World Bank
Lifelong Learning in the Global Knowledge Economy Knowledge economy puts premium on learning Requires multi-sector strategy Focus on equity Expand access to learning Raise quality by changing content, pedagogy Variety of financing mechanisms needed Policy, institutional, legal framework
Learning in Knowledge Economy Then Information based Rote learning Teacher directed Just in case Formal education only Directive based Learn at a given age Terminal education Now Knowledge creation/application Analysis and synthesis Collaborative learning Just in time Variety of learning modes Initiative based Incentives, motivation to learn Lifelong learning
Learner-centered Learner: motivation, adaptability, analytical thinking, communication, problem solving Teacher: from director to facilitator
Learning by Doing Teacher: from director to facilitator Classroom: learn by doing, team work, individual learning plans Institution: professional community centered on achievement
Traditional Learning Differs From Lifelong Learning Traditional learning Teacher is source of knowledge Learners receive knowledge Learners work by themselves Tests given to prevent progress All learners do same thing Teachers receive initial training Good learners identified Lifelong learning Educators are guides to knowledge People learn by doing People learn in groups Assessments guide learning Individual learning plans Educators are lifelong learners Access to lifetime learning
Alternative Delivery Mechanisms Increase access to learning opportunities Increase variety of ways learners can learn Give access to knowledge resources Enhance quality through technology Learning by doing Self-directed learning Continuously updated curriculum Networks of good practice
Financing Lifelong Learning Expenditures increase, public resources limited Priority for public: basic education Balance between subsidies and market mechanisms given that Benefits both private and public Access to capital uneven
Variety of Finance Mechanisms Cost-recovery Traditional loan Human capital contracts Graduate tax Income contingent loans Subsidies Voucher Learning accounts Savings accounts Tax credits Entitlements: combination loan/voucher
Governance for Lifelong Learning Requires multi-sectoral Enabling environment for pluralistic approaches Focus on equity Demand-driven policy
Rate of Return to Schooling by Country Income Group Private Rate of Return (%) 40 35 30 20 26 20 26 23 15 18 23 21 26 12 12 10 0 363 1,698 4,852 22,530 Per Capita Income sacharopoulos and Patrinos 2004 Primary Secondary Higher
. Robertson (2002), Relative Prices and Wage Inequality: Evidence from Mexico (Macalester College)
Education Indicators GER/Illiteracy Rate (%) 140 120 100 80 60 40 20 0 119 103 97 106 70 69 62 46 36 40 12 8 16 Pre-Primary Primary Secondary Tertiary Adult Illiteracy Rate High Income Middle-income Low-income
Average Years of Schooling, Latin America (15 Years and Older) 9.0 8.0 7.0 6.0 5.0 4.0 3.0 2.0 1.0 0.0 ARG PER URU CHI MEX VEN ECU COR BOL COL ELS DOM ource: Barro and Lee 2000
Not Prepared for Knowledge Jobs Percent of 16-65 Year Olds Who Test at Low Information Processing Levels (1994-98) 90 80 70 60 50 40 30 20 10 0 Sweden OECD average Czech Republic Hungary Slovenia Poland Chile
Summary: Learning in the Knowledge Economy Premium on learning Expand access to learning through alternative mechanisms and financing Raise quality by changing content, pedagogy, incentives
PISA 2000 & 2003 Results
Key Findings: Positive Factors (1) Student and learning related factors associated with higher scores Time on homework Interest in subject Student perception of relationship with teacher Understanding that science and math are associated with better job opportunities and future financial security Mother s education Home educational resources
Key Findings: Positive Factors (2) School and teacher related factors associated with higher scores Private schools More girls in school Location of school (urban/rural) Good teacher-student relations High teacher morale Teacher behavior and school climate
Key Findings: Negative Factors Student and learning related factors associated with lower scores Memorization as a way of learning is not effective Mother s employment Number of siblings School and teacher related factors associated with lower scores High student-teacher ratio
Key Findings: Mixed Effects of Technology Total number of computers available to teachers does not have a significant positive impact Computer-student ratio does not have a clear impact Availability of science equipment and laboratories associated with higher science scores Students who used computers effectively at school achieved higher scores in all subjects
Mexico: Performance Highlights Low overall performance and low dispersion in scores School type, location, climate and material; student s interest, motivation and socioeconomic factors are significantly associated with achievement Although overall there is a need for improvement; some states benchmarked themselves in a good place internationally and nationally
Performance in Mathematics by country, PISA 2003 600 500 400 300 Average Score 200 100 0 Brazil Tunisia Indonesia Mexico Thailand Uruguay Turkey Serbia Greece Italy Portugal Russian Federation United States Latvia Spain Hungary Poland Luxembourg Norway Slovak Republic Germany Ireland Austria Sweden France Denmark Iceland Czech Republic New Zealand Australia Switzerland Macao-China Belgium Canada Japan Liechtenstein Netherlands Korea Finland Hong Kong-China
Performance in Reading by country, PISA 2003 600 0 500 400 300 Average Score 200 100 Tunisia Indonesia Mexico Brazil Serbia Thailand Uruguay Turkey Russian Slovak Greece Italy Portugal Luxembourg Spain Hungary Czech Latvia Germany Austria Iceland Denmark United France Poland Macao-China Japan Switzerland Norway Belgium Hong Netherlands Sweden Ireland New Liechtenstein Australia Canada Korea Finland
Performance in Science by country, PISA 2003 600 500 400 300 Average Score 200 100 0 Tunisia Brazil Indonesia Mexico Thailand Turkey Serbia Uruguay Portugal Denmark Greece Luxembourg Norway Italy Spain Latvia Russian Austria United States Iceland Slovak Poland Germany Hungary Ireland Sweden Belgium France Switzerland Canada New Zealand Czech Netherlands Australia Liechtenstein Macao-China Korea Hong Kong Finland Japan
Reading and Math Performance of Mexico by Level, PISA (Reading- 2000; Math- 2003) Reading Math Reading Literacy Level 5 Evaluating information and building hypotheses; drawing on specialized knowledge; accommodating concepts contrary to expectations 0.5% 4.3% 0.4% 2.7% Math Level 5 Students can develop and work with models for complex situations; can select, compare, and evaluate appropriate problem-solving strategies for dealing with complex problems related to these models; can work strategically using broad, well-developed thinking and reasoning skills, appropriately linked representations, symbolic and formal characterizations, and insight pertaining to these situations; can reflect on their actions and can formulate and communicate their interpretations and reasoning 15.6% 10.1% Reading Literacy Level 1 Recognize main theme in a familiar topic; make simple connections Below Level 1 These students may be able to read, but have not acquired the skills to use reading for learning 27.5% 27.1% 24.9% 20.8% 27.9% 38.1% Math Level 1 Students can answer questions involving familiar contexts where all relevant information is present and the questions are clearly defined; able to identify information and to carry out routine procedures according to direct instructions in explicit situations; can perform actions that are obvious and follow immediately from the given stimuli
Math Scores and Dispersion Across Countries, PISA 2003 600 550 500 Finland Ireland Canada Macao Latvia Iceland ESP NLD POL Korea DEN FRA Austria NOR AUS SWE HUN CZK US HK Swit NZD JPN Germany Belgium Score 450 Purtugal Serbia RUS Greece Italy 400 Thailand Uruguay TUR MEX 350 IDN Tunisia BZL 300 240 260 280 300 320 340 360 380 Dispersion
PISA 2003: Performance by subject and state 500 450 400 350 300 250 200 150 100 50 0 Oaxaca Tabasco Guerrero Tlaxcala Chiapas Veracruz Durango Sonora Campeche San Luis Potosí Puebla Baja California Sur Zacatecas Nayarit Baja California Edo. de México Guanajuato Coahuila Yucatán Quintana Roo Morelos Hidalgo Sinaloa Tamaulipas Nuevo León Querétaro Chihuahua Jalisco Aguascalientes Distrito Federal Colima MATH READING SCIENCE
Performance by state in science by GNP per capita, PISA 2003 460 Colima Distrito Fede 440 Aguascaliente Jalisco Querétaro SCIENCE 420 400 Tamaulipas Nuevo León Chihuahua Sinaloa Morelos Hidalgo Quintana Roo México Yucatán Guanajuato Nayarit Baja Coahuila Cal. Campeche Zacatecas San Puebla Luis Poto 380 Sonora DurangoBaja Cal. Sur Chiapas Guerrero Veracruz Tlaxcala Tabasco Oaxaca 360 0 10000 20000 30000 GNPPC
Benchmarking for states: performance in science and test score dispersion, PISA 2003 460 Distrito Fede Colima 440 Aguascaliente S c o r e 420 400 Jalisco Querétaro Tamaulipas Nuevo León Sinaloa Chihuahua Morelos Quintana Roo Guanajuato México Hidalgo All Yucatán Coahuila Baja Cal. Nayarit Campeche Zacatecas Puebla San Luis Poto 380 Baja Cal. Sur Sonora Chiapas Veracruz Durango Guerrero Tlaxcala Tabasco Oaxaca 360 200 220 240 260 280 300 320 Dispersion
Benchmarking by states with other countries in science (scores and dispersions), PISA 2003 S C I E N C E 600 500 400 Japan Finland Korea Hong Kong-China New Zealand Macao-China Netherlands Czech Republic Switzerland Canada Hungary Sweden France Ireland Spain United States Belgium Norway Germany Latvia Luxembourg Greece Italy Portugal Denmark Distrito FederalColima Aguascalientes Serbia Uruguay Thailand Jalisco Turkey Querétaro Tamaulipas Chihuahua Nuevo León Sinaloa Yucatán Guanajuato Hidalgo Mexico Morelos Quintana Roo Puebla Campeche Nayarit Coahuila Edo. de México Zacatecas Baja California Indonesia Sonora Tunisia Durango San Luis Potosí Guerrero Brazil Chiapas Baja California Sur Veracruz Tlaxcala Tabasco Oaxaca 300 200 220 240 260 280 300 320 340 360 380 Dispersion
Differences in results in science by school type (private and public), PISA 2003 600 500 Math Score 400 300 200 100 0 P10 P25 P50 P75 P90 Average Public Private Note: P10 is the lowest 10 th percentile in achievement P90 is the highest 10 th percentile in achievement
Key determinants of learning Gender Student factors School resources and materials
Gender inequality in science (effects of being female on science scores), PISA 2003 10th 25th 50th 75th 90th 10 0-10 -20-30 -40-50 -60 Colima Chiapas Campeche Baja California Baja California Aguascalientes
Effect of Memorization on Performance by 0.0 Achievement Level, PISA 2003-1.0 Science Math -2.0 Reading -3.0 10th 25th 50th 75th 90th
Effect of Motivating Students on Value of Subject in Labor Market by Achievement Level, PISA 2003 6 5 4 3 Science Math 2 1 0 10th 25th 50th 75th 90th
Effect of Teacher Morale on Performance by Achievement Level, PISA 2003 4 3 2 Science Math Reading 1 0 10th 25th 50th 75th 90th
Effect of Availability of Science Equipment and Labs on Science Scores, PISA 2003, across performance distribution 14 12 10 8 6 Score 4 2 0 10th 25th 50th 75th 90th
Effect of Student Use of Computers at School by Achievement Level, PISA 2003 12 10 8 6 4 2 Science Math Reading 0-2 -4 10th 25th 50th 75th 90th
Telesecundarias in Mexico Effective at expanding educational coverage to difficult-toreach populations Covers 1.2 million students Fastest growing type of secondary school; 1/5 of total secondary school enrollment However, telesecundarias score lower than most other types of secondary schools, controlling for other factors Recommend rigorous assessment of the impact of treatment by type of secondary school with emphasis on telesecundarias
Performance advantage of different school type over telesecundaria when controlling for other factors (significant effects only) 90 80 70 60 50 40 30 20 10 0 Upper Secondary General Upper Secondary Technical Technical Professional Lower Secondary General Lower Secondary Technical Training for workers Reading Math Science
The Three A s Autonomy More autonomous schools can implement appropriate education policies. Accountability A more accountable system will encourage more active participation by parents and others, which is key to improving learning outcomes. Assessment A system that is based on constant assessment and participation in international benchmarking exercises will improve costeffectiveness.
Increase School Autonomy at Public Schools To improve quality, efforts are needed to move decision-making to the school level, thus increasing school autonomy Increasing school autonomy can compensate disadvantaged schools Autonomy can help raise the schooling outcomes of indigenous peoples School autonomy reinforces the role of homework, learning styles and future value of education With more autonomy, schools could determine the appropriate mix of technology for their students
Improve Accountability Accountability mechanisms can improve school quality Accountability mechanisms that put people at the center of service provision can go a long way in making services work and improving outcomes Flexible and wide-ranging accountability mechanisms could encompass various types of services
Continue Learning from Assessments Assessment testing can be used to inform policy decisions. Analysis of assessments can foster public and civil society involvement in education reform. However, governments must be proactive in encouraging public debate using assessment results. Expand coverage of the national assessments. National and international assessments could be used to inform school reform process
Harry Anthony Patrinos hpatrinos@worldbank.org