Building Better Teachers in the Caribbean

Transcription

1 Building Better Teachers in the Caribbean Barbara Bruns, Lead ducation conomist World Bank Regional Learning vent Improving Teaching and Learning Outcomes in the nglish-speaking Caribbean Countries with ICT Bridgetown, BARBADOS April 2011

2 Overview 1. Context: Raising teacher quality is the central education challenge Student learning is key to economic gains from education Teacher quality is key to student learning Teacher effectiveness is highly variable 2. Key challenges for teacher policy 3. Understanding what makes teachers effective in the classroom New methods for looking inside the black box vidence from Brazil: Six conclusions Reactions from education policymakers 4. Learning from innovations in the Caribbean and Latin America

3 Student learning Is the prime determinant of individual returns and economy-wide gains from education (Hanushek and Woessman, 2007) 1. New evidence on the payoffs to teacher quality

4

5 Figure 3: Test Scores and Growth in Selected Developing Countries 4 Conditional Growth Peru Indonesia Morocco BRAZIL Tunisia Cyprus gypt Iran Turkey India Chile Colombia Romania Jordan Zimbabwe Ghana Philippines Singapore Taiwan Hong Korea, Kong Rep. of Thailand Portugal China Malaysia Conditional Test Score Source: Hanushek and Woessmann: ducation Quality and conomic Growth (2007)

6 Teacher quality is the prime determinant of student learning

7 Teacher effectiveness varies tremendously (measured as ability to generate year on year learning gains) Best teachers can raise student learning by 2-3 grade levels in a single year Worst teachers raise learning 0.5 grade (Hanushek and Rivkin, 2005) Cumulative effects of these variations are huge, esp. given evidence that lower quality teachers are clustered in low-income schools Highly variable teacher effectiveness found across all countries and contexts to date

8 and raising learning is a challenge for the Caribbean and LA PISA Math Upper middle income High income: NonOCD High income: OCD 500 Lower middle income Indonesia Tunisia Jordan Thailand Average: Lower middle income Panama Peru Albania Colombia Brazil Argentina Montenegro Kazakhstan (*) Mexico (*) Chile Romania Uruguay Bulgaria Azerbaijan Serbia (*) Turkey Russian Federation Lithuania Average: Upper middle income Qatar Trinidad and Tobago Israel Dubai (UA) Croatia Latvia stonia Macao-China Liechtenstein Hong Kong-China Singapore Average: High income: nonocd Greece Italy Spain Portugal Ireland United States Luxembourg Hungary United Kingdom Czech Republic Sweden Poland Austria Slovak Republic France Norway Slovenia Denmark Iceland Germany Australia Belgium New Zealand Netherlands Canada Japan Switzerland Finland Korea Average: High income: OCD

9 2. lements of teacher quality Recruiting, Grooming and Motivating Talent

10 Recruiting Failure to recruit talent Teachers recruited from bottom third of students, with low cognitive skills Shortages of math and science teachers: 85% of OCS science teachers lack science background Low entry requirements to teacher training programs: private schools w/o accreditation Few graduates from top universities pursue teaching as a career Recruiting talent Higher standards for content mastery. Brazil, Chile, Colombia, Jamaica, Guyana, Guatemala Special incentives for scarce skills Competitive entry into teacher training: only top 10-30% of students hired as teachers in Finland, Singapore and Korea Alternative certification and public service appeal (Teach For All model) Chile, Peru, Argentina, Brazil, Colombia, Mexico

11 Grooming Failure to groom Little supervision and support for new teachers despite documented low effectiveness in first year Grooming Talent Induction programs (like medical residency): ffective support, encouragement and performance evaluation in first years No de-selection of low-performing teachers once hired but also high attrition in some countries Theoretical and ineffective in-service training Teachers not convinced that every child can learn Reforms that allow de-selection. Chile 2011 reform: school directors can fire 5% of worst performing teachers each year Singapore bottom 10% of directors fired each year In-service training based on observed classroom practice Screen and train teachers for values: perseverance and belief in kids

12 Motivating Failure to motivate Flat salary scales Average salary after 15 years only 30% above salary at entry Salaries and promotions de-linked from performance No strategic staffing rewards Parents typically lack ability to pressure teachers to perform Alibis for failure: child poverty, lack of parent support, lack of school resources Motivating high performance More attractive career ladders Chile AVDI, Sao Paulo Prova de Promocão, Mexico CM, Peru CPM offer large pay increases with promotions based on competence Pay for performance (school bonus pay) Incentives to get best teachers to most challenging schools Parents empowered to demand quality services and informed about school performance (Brazil example) If the child is not learning, it s our fault Sao Paulo state Secretary of ducation

13 Motivating vidence on Pay for Performance Rigorous global evidence limited and conflicting Positive impact Rural India, Rural Kenya, Israeli high schools and Chile s SND (but effect sizes vary a lot) No impact US (Nashville), NYC (first year) merging evidence from Brazil Design features matter Size of bonus Criteria for bonus Coverage or predictability Possible perverse impacts over time (teaching to test, cheating) Ongoing evaluations in 3 different systems in Brazil, possible scope to launch more What changes in the classroom when incentives are introduced?

14 3. Understanding what makes teachers effective in the classroom

15 having materials in the classroom is not enough

16 having the teacher in the classroom is not enough

17 education is the transformation of inputs into learning outcomes INPUTS ACTIVITIS OUTPUTS OUTCOMS Funding salaries buildings books materials construction curriculum design training testing supervision Teacher practice in the classroom Student learning Completion rates Inputs and it happens in the classroom Activities Results

18 Tool for analyzing the black box of the Classroom Stallings method classroom observations vidence on teacher practice and classroom dynamics from LAC First large-scale, systematic classroom observations in LAC Ongoing in: Brazil (Pernambuco, MG, Rio municipality) Starting in: Colombia, Mexico, Jamaica, Guatemala Planned in: Guyana, l Salvador, Chile

19 Stallings classroom snapshot instrument: What does it measure? Teacher s use of instructional time Use of materials, including ICT Core pedagogical practices Ability to keep students engaged

20 Stallings classroom snapshot instrument: How does it work? 10 observations of each class taken at regular intervals and coded using a standardized grid

21 Stallings classroom snapshot instrument: What are some key advantages? High reliability - across observers/countries Benchmarking - Results can be compared to US/OCD good practice indicators Simple - Relatively short training needed to produce qualified observers Analytical power - Stallings variables are: statistically representative of the education system highly correlated with learning results

22 School No.: Classroom No. xact time of observation: OBSRVATION CODING GRID MATRIAL NO MATRIAL TXTBOOK NOTBOOK BLACK BOARD LARNING AIDS ICT COOPRATIV ACTIVITY 1. RADING ALOUD 2.DMONSTRATION/ LCTUR T S L I S L CHCK IF CHORAL RADING T S L I S L 3. DISCUSSION/QUSTI ONS AND ANSWRS 4. PRACTIC & DRILL 5. ASSIGNMNT/ CLASS WORK 6. COPYING T S L I S L T S L I S L T S L I S L T S L I S L 7. VRBAL INSTRUCTION 8. SOCIAL INTRACTION 9. STUDNT(S) UNINVOLVD 10. DISCIPLIN 11. CLASSROOM MANAGMNT T S L I S L T I I T T I 12. CLASSROOM MANAGMNT ALON T 13. TACHR SOCIAL INTRACTION OR TACHR UNINVOLVD T 14. TACHR OUT OF TH ROOM T

23 Coding grid FIGUR 1: XCRPT FROM CLASSROOM SNAPSHOT (APPNDIX 2) CLASSROOM OBSRVATION SNAPSHOT MATRIAL ACTIVITY 1. RADING ALOUD NO TXTBOOK NOTBOOK MATRIAL T BLACK BOARD LARNING ICT COOPRATIV AIDS S L I S L CHCK IF CHORAL RADING T line: Indicates activities that involve the teacher I line: Indicates activities that involve the student & not the teacher 1, S, L, : Indicate one individual, a small, large group and entire class respectively

24 School and Classroom Demographic information

25 OCD Good Practice Benchmarks

26 Results: vidence from Brazilian classrooms Six conclusions

27 #1 Brazil is far from OCD benchmarks Teachers Use of Instructional Time Rio de Janeiro City (100 schools) Pernambuco State (300 schools) Minas Gerais State Schools that improved Schools that declined OCD Good Practice Benchmarks Teaching activities 62% 61% 66% 63% 85% Classroom organization 31% 27% 27% 25% 15% Non-academic activities Teacher out of the classroom 7% 12% 8% 12% 0% 3% 8% 3% 6% -

28 #2 Improving and Declining Schools Look Different Teachers Use of Time All schools (average) Improving/ Declining Schools Teacher out of the classroom Teacher not engaged with students 4% 3% 6% 3% 2% 3% Teacher using Q and A 12% 15% 10% Small or large group of students not engaged 28% 19% 37%

29 #3 More time on learning is highly correlated with learning outcomes Teacher Use of Time 5 th grade 1. Learning Activities 2. Classroom management 3. Teacher off-task 3.1 Teacher out of room Total Sample Rio 58% 37% 6% 1% 10% highest IDB schools 70% 27% 3% 0% 10% lowest IDB schools 54% 39% 7% 3% Difference [0,09]* [0,09]* [0,02] [0,01]** - rros robustos em parênteses: *statisticamente significativo a 10%, **statisticamente significativo a 5%, ***statisticamente significativo a 1% - Dados no nível do ano/escola

30 #4 More time on learning predicted schools learning gains and attainment of bonus (P) Teacher use of Time Bônus Learning activities 2. Classroom management 3. Teacher off-task 3.1 Teacher out of the classroom Pernambuco State sample 61% 28% 11% 8% Achieved bonus % 27% 10% 8% Didn t achieve bonus % 30% 17% 12% Difference [0,04]** [0,03] [0,02]*** [0,02]*** Sample of 300 State schools, with 1,800 classrooms observed math and portuguese classes in 5th, 9th and 12th grades Standard errors in parenthesis. * Statistically significant at the 10% level, **Statistically significant at the 5% level, ***Statistically significant at the 1% level

31 # 5 Tool for identifying schools with highly positive (and negative) performance xtreme situations Teacher absent from the classroom more than 50% of total class time Teacher absent from classroom more than 25% of total class time Teacher spent over 50% of class time using no materials Teacher spent more than 50% of class time writing on black board Large group of students not engaged during more than 50% of class time Rio de Janeiro % of classrooms observed P Improving schools MG pilot Declining schools 0% 2% 0% 0% 1% 10% 3% 3% 4% 10% 17% 31% 24% 32% 5% 9% 9% 7% 9% 19%

32 #6 Teacher practice varies a lot with huge implications for learning TIM US IN TH CLASSROOM Rio de Janeiro schools, June % total schools Learning activities Classroom management Teacher off task On average, only 61% of class time is used for instruction But some schools average 81% Others average 37% 88 more days of instruction per school year

33 #6 Teacher practice varies a lot even within schools TIM US IN TH CLASSROOM Rio de Janeiro schools, June 2010 Across different classrooms in the same school Learning activities Classroom management Teacher off task % of time on learning in best class 52% in worst class 36 more days of instruction per year in the most efficient classrooms

34 Reactions in Brazil From ducation secretaries, district supervisors and school pedagogical coordinators

35 This type of classroom observation is directly relevant to my daily work...(p) We know this intuitively, but this is the first time we have hard data we can take back to our regional offices and schools to discuss and work on. Secretary of ducation (MG) I was surpised to see...very traditional teacher practice It allowed me to observe better the dynamics of class, because in my work environment, observations do happen, but not too closely or with such details

36 The textbooks that come via MC are rarely used in the classroom; the teachers prefer to write on the blackboard ven in different schools and municipalities the teacher practices are the same Teachers work in a very traditional way: exercises and corrections Despite the intense technological progress, we still give priority to old ways of teachinglearning

37 the opportunities for Caribbean countries and Latin America

38 Conclusions Virtually all Caribbean and LA countries trying to raise teacher quality Innovative approaches, universal issues Little hard evidence on impact or costeffectiveness in OCS, OCD or LAC Great opportunity to generate knowledge that will benefit OCS and others Combining test data with classroom observations creates a platform for solid evidence on how to Build Better Teachers

39 Thank you! Barbara Bruns