Synergies for Better Learning: An International Perspective on Evaluation and Assessment Key messages from the 3 year OECD review Claire Shewbridge Brussels, 2 May 2013
1. What is in the report?
Comprehensive analysis of trends, practices and policy developments across countries Evaluation and assessment is complex and potentially divisive. The report seeks a balanced approach. Analysis drawing on a broad evidence base 11 papers reviewing international literature International indicators Qualitative indicators collected from 29 systems 26 specific reports compiled by participating systems Description of policies in place; evidence of impact of policies; views of stakeholders OECD reports on specific systems (15 reviews) 2800+ interviews with different stakeholders Expertise contributed by 28 professionals external to OECD A range of policy options to resolve tensions and help shape a balanced approach No one-size-fits all solution Systems are at different stages of developing their E&A frameworks Policy options need to be tailored to context
How is the analysis organised? Should underpin all other choices! Chapter 3: Evaluation and assessment framework System School Chapter 4: Student Assessment Classroom Chapter 5: Teacher Appraisal Chapter 8: Education System Evaluation Chapter 7: School Leader Appraisal What do we want to achieve? (Purpose) Development Accountability Who is involved? (Capacity) Skills, time, resources How and what do we evaluate? (Procedures) Tools, reference standards What do we do with the results? (Use of results) Instructional adjustments, professional development, setting development goals Certification, promotion, public reporting, additional support
2. Why the need for an international review of evaluation and assessment policies?
Factors contributing to the increased role for evaluation and assessment in schooling Public demands for information on school quality Demands to use public resources efficiently Increased autonomy at local and school levels Increased quality assurance in public sector and demands for evidence-based policy making Rising importance of education Knowledge and the economy International competition
Expansion of educational evaluation in school systems Creating specific agencies To drive development of national student assessments (e.g. Italy, 2004; Austria, 2008; Mexico, 2002, 2012) To introduce reporting and monitoring requirements (e.g. Denmark, 2006, 2011; Chile, 2011) To strengthen external school evaluation (SWE, 2008) Stimulating school selfevaluation Almost universal (26/29 systems) Requirements vary significantly in nature, e.g. conduct self evaluation; produce specific report on school development; account for school quality Promoting school leader appraisal Central/state requirements for appraisal (17/29 systems) Employment-related appraisal (3/17 systems) Mandatory periodic appraisal (14/17 systems)
Educational measurement and indicators development are rising in importance By the late 1990s all OECD countries had participated in an international student assessment In 2012 national educational measurement is well established in the majority of systems Student assessments AUS AUT BFL BFR CAN CHL CZE DNK EST FIN FRA HUN ISL UK- IRL ISR ITA KOR LUX MEX NLD NZL NOR POL PRT SVN SVK ESP SWE NI Full cohort Sample based UK- Surveys AUS AUT BFL BFR CAN CHL CZE DNK EST FIN FRA HUN ISL IRL ISR ITA KOR LUX MEX NLD NZL NOR POL PRT SVN SVK ESP SWE NI Students Teachers Parents Longitudinal information UK- AUS AUT BFL BFR CAN CHL CZE DNK EST FIN FRA HUN ISL IRL ISR ITA KOR LUX MEX NLD NZL NOR POL PRT SVN SVK ESP SWE NI
Conflicting demands? Too much information? Does this contribute to improving student learning? 3. Aiming for coherence and balance
Creating synergies in a coherent evaluation and assessment framework Professionalism and trust Student assessment Attention to equity School evaluation Student learning Teacher and school leader appraisal Commitment to transparency Education system evaluation Integrating both development and accountability
Being strategic at the system level and engaging stakeholders in use of results High quality measures informing, but not driving policy Indicators of a strategic approach to information collection Mapping against system priorities and plan to prioritise new collection Mapping against system priorities Plan to prioritise collection of new information Neither Systems Australia; Czech Republic; Hungary; Israel; Netherlands; Slovak Republic France; Iceland; Ireland; Northern Ireland (UK) Belgium (French & Flemish Comm.); Chile; Finland; Slovenia; Spain Austria; Denmark; Italy; Korea; Luxembourg; Mexico; New Zealand; Norway; Poland; Sweden Summary overview report on priorities in school system (NOR) Engaging stakeholders to discuss key results (DNK; BFL)
Evaluation and assessment is everybody s business! Students Improving student learning School leaders Specific technical bodies Central & local authorities
Students (1) Students do not want to be passive objects of evaluation and assessment. They want to be actively involved, not just in their own assessment, but also in the evaluation of their teachers and schools. Representative of the European School Student Unions, OSLO conference 11 April 2013 Research underlines the important role that students can play in school improvement.
Students (2) Collecting feedback from students Developing surveys for feedback on quality of teaching and learning Norway annual survey for all students in Years 7 and 10 Flanders Secondary student organisation has developed possible tools for teachers to use for student feedback NYC-US annual survey results reported for accountability Strong student voice in school activities (SWE; BFR) Giving feedback to students Regular feedback on student progress (SWE; UK-NI) From external school evaluation (UK-ENG) Next day feedback from national assessments (DNK)
Parents Collecting feedback from parents Specific surveys in several systems Considering parental complaints in external school evaluation (NLD) Contributing to school annual reporting (SVN) Adequate reporting and communication on student assessment results Reporting on progress against central standards (LUX; DNK) Providing adequate information on school performance measures (AUS) Clear language in reports from external school evaluation (NZL)
Teachers (1) Creating specific roles in school self-evaluation E.g. data analysis teams (NLD, EST, UK-ENG) E.g. school improvement teams (HKG-CHN) Providing supporting tools for self-evaluation E.g. UK-SCO; SWE Feedback on teaching Regular observation of teaching and feedback (SVK, CZE) In external evaluation on standard observation tool (BFL) There is certainly room for improvement
Improving student learning: Focus on quality of teaching and learning? 100 Only half of 15 year olds surveyed in 90 PISA 2009 were in schools where 80 70 the principal reported he/she 60 observed instruction in classrooms 50 40 30 20 10 0 67% of lower secondary teachers surveyed in TALIS 2008 were in schools where the principal reported the direct appraisal of classroom teaching was considered with high or moderate importance in school selfevaluation or external evaluation. This was 70% in Flanders.
Teachers (2) Drawing on teacher professionalism in national assessments Teacher judgement against national standards (SWE; UK-NI) Engaging teachers in designing and marking (NZL) Feedback to teachers from national assessments Timeliness of feedback and analytical use (DNK) Keeping it as a purely pedagogical tool (UK-NI) Timing assessments to better inform instruction (NOR) Central training in better using results for school development and instructional improvement (LUX)
School leaders (1) Promoting their role in school self-evaluation Professional standards including school self-evaluation competencies (Ontario, CAN) Targeted training in school self-review (NZL) Support to develop quality assurance (SVN; AUT) Promoting distributed leadership in school leader appraisal (CHL) Building competencies in classroom observation Engaging school leaders in external school evaluation (UK- NI) Promoting peer reviews (BFL)
School leaders (2) Developing standards for school leadership Holding school leaders accountable and promoting their development Separate appraisal schemes (CHL; KOR) A cycle of formative appraisal ending with a summative decision (AUS; CAN; UK) Linking appraisal results to salary progression and professional development (NZL; NI-UK; MEX)
Creating synergies in a coherent evaluation and assessment framework Constructive feedback Consensus Student assessment Clarity of purpose Collegiality School evaluation Teacher and school leader appraisal Caution Use multiple measures Capacity Education system evaluation Central standards Local relevance Credibility
And there is so much more.! Synergies for Better Learning can be accessed for reading: oe.cd/betterlearning All other materials from the OECD Review are available for free: www.oecd.org/edu/evaluationpolicy