Quality of Education and Learning Outcomes

Similar documents
RECOGNITION OF THE PREVIOUS UNIVERSITY DEGREE

Berkeley International Office Survey

DEVELOPMENT AID AT A GLANCE

MEASURING GENDER EQUALITY IN EDUCATION: LESSONS FROM 43 COUNTRIES

ITEC / SCAAP PROGRAMMES ITEC/SCAAP Programmes Sponsored by : Ministry of External Affairs, Government of India

GHSA Global Activities Update. Presentation by Indonesia

Regional Capacity-Building on ICT for Development Item 7 Third Session of Committee on ICT 21 November, 2012 Bangkok

16-17 NOVEMBER 2017, MOSCOW, RUSSIAN FEDERATION OVERVIEW PRESENTATION

The Assistant Director-General for External Relations and Public lnfonnation

Overall student visa trends June 2017

REGIONAL CAPACITY BUILDING ON ICT FOR DEVELOPMENT

Introduction Research Teaching Cooperation Faculties. University of Oulu

In reviewing progress since 2000, this regional

Twenty years of TIMSS in England. NFER Education Briefings. What is TIMSS?

The Rise of Populism. December 8-10, 2017

SACMEQ's main mission was set down by the SACMEQ Assembly of Ministers as follows:

REPORT ON THE ACTIVITIES OF THE INSTITUTE IN 2011

Setting the Scene and Getting Inspired

OHRA Annual Report FY16

Organised by

Michuki Mwangi Regional Development Manager - Africa ISOC. AFTLD AGM 7 th March 2010 Nairobi, Kenya

JICA s Operation in Education Sector. - Present and Future -

11. Education: Gender Disparities [205]

Santa Barbara Peace Corps Association Members ALPHABETICAL ORDER by last name (as of 4/8/13)

Alternative education: Filling the gap in emergency and post-conflict situations

TESL/TESOL Certification

Dakar Framework for Action. Education for All: Meeting our Collective Commitments. World Education Forum Dakar, Senegal, April 2000

Master of Statistics - Master Thesis

The Conference Center. of the Americas. at the Biltmore Hotel. Miami, Florida

Department of Education and Skills. Memorandum

OHRA Annual Report FY15

OCW Global Conference 2009 MONTERREY, MEXICO BY GARY W. MATKIN DEAN, CONTINUING EDUCATION LARRY COOPERMAN DIRECTOR, UC IRVINE OCW

The Rise of Results-Based Financing in Education 2015

Improving education in the Gulf

APPLICATION GUIDE EURECOM IMT MASTER s DEGREES

No. 11. Table of Contents

Annual Report

National Academies STEM Workforce Summit

Target 2: Connect universities, colleges, secondary schools and primary schools

A Global Imperative for 2015: Secondary Education. Ana Florez CIES, New Orleans March 11th, 2013

Department of Geography Geography 403: The Geography of Sub-Sahara Africa

Management and monitoring of SSHE in Tamil Nadu, India P. Amudha, UNICEF-India

Educational system gaps in Romania. Roberta Mihaela Stanef *, Alina Magdalena Manole

EFA and the Institute of Education, University of London : implicit and explicit engagements

Baku Regional Seminar in a nutshell

James H. Williams, Ed.D. CICE, Hiroshima University George Washington University August 2, 2012

Asia-Pacific Regional Education for All Report. A Synthesis of the National EFA Reports

Higher education is becoming a major driver of economic competitiveness

EXECUTIVE SUMMARY. TIMSS 1999 International Mathematics Report

A 90 Year Quest for Excellence in Education!

APPENDIX 2: TOPLINE QUESTIONNAIRE

What Do Teachers Know and Do? A Report Card on Primary Teachers in Sub-Saharan Africa

ESTABLISHING NEW ASSESSMENT STANDARDS IN THE CONTEXT OF CURRICULUM CHANGE

Meeting on the Recognition of Prior Learning (RPL) and Good Practices in Skills Development

International activities of the American Society for Microbiology (ASM)

SMASE - WECSA ASSOCIATION 10 th Anniversary

Collaborative Partnerships

New Education Division Documents No. 13. Post-basic Education in Partner Countries

Department: Basic Education REPUBLIC OF SOUTH AFRICA MACRO INDICATOR TRENDS IN SCHOOLING: SUMMARY REPORT 2011

IS THE WORLD ON TRACK?

BASIC EDUCATION IN GHANA IN THE POST-REFORM PERIOD

California Professional Standards for Education Leaders (CPSELs)

HAAGA-HELIA University of Applied Sciences. Education, Research, Business Development

PIRLS. International Achievement in the Processes of Reading Comprehension Results from PIRLS 2001 in 35 Countries

CONSULTATION ON THE ENGLISH LANGUAGE COMPETENCY STANDARD FOR LICENSED IMMIGRATION ADVISERS

Summary results (year 1-3)

EDUCATION. Graduate studies include Ph.D. in from University of Newcastle upon Tyne, UK & Master courses from the same university in 1987.

CONVENTION ON INTERNATIONAL TRADE IN ENDANGERED SPECIES OF WILD FAUNA AND FLORA

In September 2000, heads of all 191 member states of the United Nations committed

RELATIONS. I. Facts and Trends INTERNATIONAL. II. Profile of Graduates. Placement Report. IV. Recruiting Companies

RAISING ACHIEVEMENT BY RAISING STANDARDS. Presenter: Erin Jones Assistant Superintendent for Student Achievement, OSPI

Asian Development Bank - International Initiative for Impact Evaluation. Video Lecture Series

INSTRUCTION MANUAL. Survey of Formal Education

Introduction to the HFLE course

UNESCO Bangkok Asia-Pacific Programme of Education for All. Embracing Diversity: Toolkit for Creating Inclusive Learning-Friendly Environments

Internet Society (ISOC)

CHAPTER 3 CURRENT PERFORMANCE

Financing of Higher Education in Latin America Lessons from Chile, Brazil, and Mexico

Measuring up: Canadian Results of the OECD PISA Study

The Survey of Adult Skills (PIAAC) provides a picture of adults proficiency in three key information-processing skills:

EXECUTIVE SUMMARY. TIMSS 1999 International Science Report

DOES OUR EDUCATIONAL SYSTEM ENHANCE CREATIVITY AND INNOVATION AMONG GIFTED STUDENTS?

No educational system is better than its teachers

Politics and Society Curriculum Specification

Standards and Criteria for Demonstrating Excellence in BACCALAUREATE/GRADUATE DEGREE PROGRAMS

Improving the impact of development projects in Sub-Saharan Africa through increased UK/Brazil cooperation and partnerships Held in Brasilia

BENCHMARK TREND COMPARISON REPORT:

August 14th - 18th 2005, Oslo, Norway. Code Number: 001-E 117 SI - Library and Information Science Journals Simultaneous Interpretation: Yes

TIMSS Highlights from the Primary Grades

Curriculum vitae University of Saarland Sociology, American Studies, Economics

Addressing TB in the Mines: A Multi- Sector Approach in Practice

The Oregon Literacy Framework of September 2009 as it Applies to grades K-3

FACTORS AFFECTING TRANSITION RATES FROM PRIMARY TO SECONDARY SCHOOLS: THE CASE OF KENYA

PUPIL PREMIUM POLICY

National and Regional performance and accountability: State of the Nation/Region Program Costa Rica.

Abstract. Janaka Jayalath Director / Information Systems, Tertiary and Vocational Education Commission, Sri Lanka.

Developing an Assessment Plan to Learn About Student Learning

UPPER SECONDARY CURRICULUM OPTIONS AND LABOR MARKET PERFORMANCE: EVIDENCE FROM A GRADUATES SURVEY IN GREECE

Global Conference on Literacy

RECOMMENDED CITATION: Pew Research Center, October, 2014, People in Emerging Markets Catch Up to Advanced Economies in Life Satisfaction

Transcription:

Quality of Education and Learning Outcomes Aaron Benavot School of Education/Educational Administration & Policy Studies University at Albany-SUNY Education in the Post-2015 Development Agenda Region 1 Consultation UNESCO, Paris, France December 5-6, 2013

Conceiving Quality of Education: begin by viewing quality as multidimensional 1. An enabling learning environment with basic conditions for learning Material conditions: safe nearby schools, sufficient classrooms, heating/ventilation, latrines, potable water, textbooks and exercise books, libraries (reading corners) & instructional resources like maps, globes, science instruments, labs, computers, access to internet. Non-material conditions: teachers are present in classroom; sufficient instructional time; appropriate language of instruction; relevant and feasible (not overloaded) curriculum; gender-neutral, culturally sensitive textbooks 2. Teacher-related factors: the prevalence of motivated, knowledgeable, welltrained and experienced teachers who receive fair compensation for their teaching efforts 3. Classroom processes: dominant instructional practices (teachercentred/child-centred); opportunities to process and apply of diverse knowledge and perspectives; use of teaching practices aligned with student needs 4. Learning outcomes: the acquisition of knowledge, skills, competences, attitudes and values as well as a desire for further learning 5. Internal efficiency: reduced dropout rates, grade repetition, high completion These components are integrated in different ways in diverse models of learning.

Typical model of Student Learning (1)

Model of Student Learning Outcomes (2) Enabling Inputs Expenditures Physical infrastructure Human resources (teachers,..) School governance/autonomy Intended curriculum Teaching & learning materials Classroom Environment/ Learning Activities Outcomes Subject based knowledge Literacy and numeracy skills Creative and emotional skills Values and behavior Social benefits Context Global diffusion of educational models/practices Economic and labor market conditions (Int l & Nat.) Socio-cultural traditions, Historical influences National governance and management strategies Public resources available for education Public expectations of education Parental support/ school involvement National standards Influence of peers Changes of teaching profession

Low educational expenditure per pupil in public schools Insufficient instructional time: do countries reach the recommended 850-1,000 yearly instructional hours? Uneven availability of textbooks and other instructional materials: textbooks per pupil Computer availability; internet access by school or classroom Lack of qualified teachers, poor teaching training, poor conditions of employment High pupil-teacher ratios (PTR): do countries have more than 40 students per teacher; are PTRs are rising in countries where education has expanded rapidly? Outcome measure: Low completion or high drop-out rates in basic education: percentage of children who complete primary or compulsory school cycle Indicators of quality: from quality as inputs to quality as outcomes In the past, researchers and governments mainly measured and monitored inputs (see typical proxy measures below); today much more emphasis on outcomes, but only certain kinds of learning outcomes

Despite the dominant turn to viewing quality as outcomes; still little monitoring and few comparable measures of actual classroom life Many models of learning emphasize an input-output process. These models situate learning in a production process: inputs throughputs/process outputs/outcomes Despite growing policy emphasis on learning outcomes--typically measured in school-based assessments, some comparable the nature of the educational process (what happens inside classrooms) is often ignored, or left unmeasured. This is the black box of learning, producing a weak evidentiary base as which classroom activities produce or less learning short term and long-term. To understand what happens inside classrooms, detailed information is needed about the intended and implemented curriculum, including use of textbooks and instructional materials. Also need: context-sensitive observations of the pedagogy, teaching methods and nature of teacherstudent interactions. Interestingly, regardless of whether new instructional technologies (e.g., laptops, notebooks, software) actually contribute to learning, they may tell us more about what happens inside classrooms

International & Regional Assessments Sponsor- Survey Int l Assessment of Educational Progress IEA-Trends in International Mathematics and Science Study (TIMSS) IEA-Progress in International Reading Literacy Study (PIRLS) Year(s) of Assessment Number of Assessments Subject/ Content Area 1988,1990 2 Mathematics, Science 1964,1970,1982,1984,1995, 1999, 2003, 2007, 2011 1970, 1990, 2001, 2006, 2011 IEA- SCE, CIVED and ICCS 1968 1973; 1994-2002 2009 8-9 Mathematics, Science (grades 4, 8 and 12) 4-5 Reading (grades 4, 8 and 12) Civics, citizenship education, social studies IEA-other assessments: SITES, PPP, COMPED, CES, SFFL OECD-Programme for International Student Assessment (PISA) UNESCO-Laboratorio (TERCE) Latin America CONFEMEN-PASEC Francophone Africa + UNESCO-SACMEQ Anglophone Africa 1960s-1990s 2000, 2003, 2006, 2009, 2012 Technology, Pre-primary education, Computers, Classroom environment, Foreign languages 5 Mathematics, Science and Reading Literacies (15 yr old) 1997; 2006; 2012 3 Mathematics, Reading and Sciences (3 rd & 6 th grades) 1995-2006 Once or, in some cases, twice Mathematics, Reading (grade 5) 1999, 2002, 2007 3 Mathematics, Reading (grade 6)

Which learning is being assessed in international learning assessments? TIMSS, PIRLS and PISA are all summative assessments, mainly for stakeholders external to the school e.g., policy analysts, ministry officials and inspectors, curriculum developers, and educational researchers TIMSS, PIRLS and PISA place greater emphasis on cognitive knowledge, skills and competences, and on the lower levels or stages of learning, mainly in three specific curricular areas: language, mathematics and sciences TIMSS, PIRLS and PISA are all low-stakes assessments, and have little direct impact on a child s progress in the educational system and in life.

Unanticipated Worldwide Increase in National Learning Assessments,1995-99 to 2000-06 Percent of countries in each region that have carried out at least one national assessment between 1995-99 and 2000-06 Overall, growing share of developing countries have conducted NLAs: from 28% to 50% Sub-Saharan Africa Arab States Central Asia East Asia/Pacific South/West Asia Latin America Caribbean N. America W. Europe Central Eastern Europe 15 11 15 11 24 25 33 33 54 44 69 55 64 59 65 77 1995-1999 2000-2006 Sources: EFA Global Monitoring Report (2008); Benavot and Tanner (2007)

Even with growing preponderance of learning assessments, many unanswered questions remain about Whether, and to what extent, assessment results are publicly disseminated If, and how, assessments are used by policy makers, schools, teachers, teacher trainers to improve teaching practices and provision of instruction The extent to which assessments bring about real change in the conditions and practices of local schools ( closing the feedback loop ) Whether the knowledge emerging from such assessments is applicable/ relevant to specific contexts Whether assessments are available for secondary analyses.

Limitations of assessing learning outcomes Assessed learning outcomes primarily focus on cognitive achievement in language, mathematics and sometimes science. Less attention to other subject areas or competences: e.g., foreign language, history, social studies, health education, ESD, moral education, and to non-cognitive learning outcomes Few studies follow student achievements over time (longitudinal). Most assessments are cross-sectional: single time point and grade/age level Certain factors excluded from models: Emphasis on inputs, less on actual school conditions, classroom processes, opportunity to learn (OTL); range of socio-demographic factors (eg home language, family size) Correlation vs. causation: few randomized studies of causal factors and little multi-level research despite nested nature of schooling Limited coverage of non-oecd countries in international assessments (less of a problem in regional and national assessments) Studies assess learning outcomes among schooled children and do not test (or compare) the knowledge levels of similarly aged young people who are currently - or permanently - out of school (ex Pratham)

Thank you! For more information: Contact Aaron BENAVOT abenavot@albany.edu To download papers, go to: albany.academia.edu/aaronbenavot

Critical Perspectives on Learning Assessments (1) International and regional assessments, and most national assessments, focus on a narrow range of school-based knowledge and skills--in language, mathematics and, in some cases, science and civics. In most educational systems, students are exposed to a much broader array of subject matter and knowledge domains for example, history, geography, environmental studies, social studies, art, music, literature, religious/moral education, physical education which are rarely assessed, but which constitute a significant percentage of a student s school week. If countries undertook assessments in these other subjects, would country rankings remain the same? Probably not. Should sub-par or average international performance in only one to three areas of the curriculum be the basis for calls for educational reform? Many learning assessments collect data on multiple dimensions of learning including knowledge, skills, attitudes and behavior, although reports of assessment results tend to focus almost exclusively on those pertaining to the knowledge or cognitive dimension. Would international rankings of countries look substantially different if attitudinal and/or behavioral dimensions of student learning were given more visibility?

Critical Perspectives on LAs (2) Almost all learning assessments are one-time (or periodic) snapshots of what students know, what levels of skill proficiency they have attained, and how knowledge and proficiencies are distributed. Almost no comparative assessments follow the same students over time. As such, they provide indirect (and incomplete) information about learning gains or losses over time, and changes in the unequal distribution of learning levels. Would longitudinal evidence about learning levels and inequalities show different patterns and relationships than the ones currently being reported? Almost all assessments do not collect information about what happens inside the classroom. LAs tend to emphasize input variables, on the one hand, and learning outcomes, on the other. They ignore variables that capture classroom-based learning processes and teaching practices. If effective teachers and teaching practices are the key to improved learning, as many argue, then how can assessments that compile little information about teachers and classroom teaching generate important information on policy reform levers?

Critical Perspectives on LAs (3) Assessments invariably concentrate on achievement in one curricular subject at a time. In doing so, they incorrectly assume that the actual coverage of a subject (the implemented curriculum) is relatively independent of the coverage in other subjects. In reality, school decisions about allocating instructional time to different school subjects are part of a zero-sum framework. If schools decide to devote more instructional time to, say, mathematics or science (highly valued subjects), this reduces time available to presumably less valued subjects, say, in the arts or the humanities. In short, the interdependence of subject emphases in school curricula--usually overlooked in comparative learning assessments--can affect content coverage and influence student learning. Learning assessments are either curriculum-based (TIMSS, PIRLS) or crosscurricular in their orientation (PISA). Many analysts are quick to draw policy conclusions about the need for specific curricular reforms when some assessments like PISA lack systematic information about curricular subjects, and their implementation in the school and classroom. This puzzling tendency underscores the need for caution when drawing curriculum-oriented policy implications from certain learning assessments.

Critical Perspectives on Learning Assessments (4) International assessments only examine learning outcomes among schooled children and youth. They do not test the knowledge levels of similarly aged young people who are currently--or permanently--out of school. Thus if policy makers are interested in an overall picture of what young people know, and whether they have acquired the skills and proficiencies necessary for adult roles, then the exclusion of non-tested populations bias (usually upwards) the national average. This is especially salient in countries in which participation rates are low or dropout rates are high.

Overall, int l. regional and national LAs: Seek to measure learning levels on a system-wide basis, for a specified age-based group of students (e.g. 4 th graders or 8 th graders) Provide stakeholders with (select) systematic information about the quantity and quality of learning that has occurred. Usually designed to compare learning levels against a given curricular standard or achievement/ competence level Are predominantly curriculum-based and subject-specific. Only a few examine cross-curricular competences (like PISA) Examine knowledge of the official intended curriculum (official syllabus, required topics, authorized textbooks, etc.) and not the actual implemented curriculum Focus on cognitive learning outcomes: how much knowledge and skills students have acquired from what they should have been taught in a specific subject area Are mainly created, administered and analysed by national ministries of education or national evaluation/research institutes. Sometimes assisted by regional and int l agencies. Funding can come from multiple sources.

Note: the data on National Learning Assessments excludes information on: 1) School leaving examinations, often given at key transition points in a national education system: End of primary cycle basic education upper secondary 2) Entrance examinations to gain admission to universities, colleges and other tertiary-level institutions 3) Formative assessments and testing within schools While National Learning Assessments are related to int l. and regional assessments exercises, the latter have different histories, sponsors, purposes and designs

NLAs: Characteristics and Patterns Historical trend: Between 1995-99 and 2000-06 the percentage of countries in the world that had carried out at least one NLA more than doubled: from 28% to 67% In both periods, the prevalence of NLAs is strongest in developed countries, followed by developing countries and finally countries in transition Regional variation: lowest levels in Sub-Saharan Africa and Central Asia. Highest levels in North America and Western Europe. In all regions, however, increases in % of countries conducting NLAs Subjects assessed: Of countries that have conducted at least one NLA, over 90% have assessed language (reading) and mathematics. About 50% have assessed learning in sciences. Social studies, history, geography or civics was assessed in about 40% and foreign languages in 18% of NLAs. These patterns changed little over time, though some increase in foreign language assessment (from 14% to 21%) Subject by region: Strong emphasis on sciences in LAC, SWA & East Asia. Relatively more assessments of social science subjects in LAC and SWA. Most frequently assessed grades: Grades 4-6, then Grades 1-3 and 7-9.

Despite growth in int l, regional and national assessments, many countries have never participated in, or conducted, an assessment For example, in sub-saharan Africa: Angola, Burundi, Cape Verde, Comoros, Dem Rep of Congo, Equatorial Guinea, Guinea-Bissau, Liberia, Rwanda, Sao Tome and Principe, Sierra Leona, Somalia And in Ivory Coast and Eritrea, more than 10 years since the last learning assessment was conducted

Expansion of Assessments of Adult Skills and Competencies Name and Sponsor of Adult Assessment International Adult Literacy Survey (IALS): OECD and Stats Canada Adult Literacy and Life Skills Survey (ALL) Year of Assessment Number of participating countries 1994, 1996, 1998 20 2003 6 China Adult Literacy Survey 2001 1 Literacy Assessment and Monitoring Programme (LAMP): UNESCO Programme for the International Assessment of Adult Competencies (PIAAC): OECD 2005-2010?? 2011-2013 24 Skills Assessed Prose literacy, Document Literacy and Quantitative Literacy Prose literacy, Document Literacy and Quantitative Literacy Prose literacy, Document Literacy and Quantitative Literacy Functional literacy and numeracy skills Literacy, Reading components, Numeracy, and Problem solving in technology-rich environments

In theory, and if designed well, learning assessments can have numerous potential benefits. They Chart trends /progress in (select aspects of) student learning Identify socio-demographic disparities in learning outcomes Monitor impact of policies aimed at improved learning e.g., teacher training, curriculum development, textbook revision Improve allocation of resources to support low-performing schools Provide information to parents and other stakeholders to ensure that schools are more accountable

However, in developing world, institutional capacity and scientific quality of learning assessments are often weak Problematic item construction-validity and reliability of items Non-representative sampling procedures Lack of standardization in test administration and grading Many countries under-estimate the complexity of conducting assessments and how to ensure that results are policy relevant (Braun and Kanjee 2006). Multi-lateral agencies e.g., World Bank, UNESCO and IDB--and bilateral donors increasingly support capacity building for conducting learning assessments. Ex: at World Bank, the percentage of education projects providing support for learning assessments increased from 0% before 1988 to 70% during 1990-1994 and about 60% of projects funded since 1995 (Larach and Lockheed 1992; Nielsen 2006)

Why is the world awash in Learning Assessments? Explanations (1) Shifts in (political/ideological) discourse (Coleman report): no longer enough to provide equality of opportunity, need to examine (in)equalities or disparities in outcomes Methodological: the scientific basis for measuring certain kinds of learning outcomes have improved; The technology for conducting learning assessments is cheaper and more readily available More national governments are willing to fund/ support standardized learning assessments, as a prelude to, or consequence of, participation in comparative assessments The business of learning assessments and testing is a lucrative activity: many corporations are making considerable profits from designing, carrying out and analysing assessments

Explanations (2) Growing interest in quality issues in international education policy debates: institutionalization of Education for All agenda (1990 and then in 2000); countries expected to produce national plans to achieve EFA goals (and MDGs) One EFA goal focuses on Quality.(see below) Noticeable shift in indicators of quality education: from measures of inputs (e.g., expenditures, trained teachers, textbooks, pupilteacher ratios, instructional time) to measuring learning outcomes (see below) International education discourse places a premium on accountability and evidence-based policy. Learning assessments provide evidence to evaluate system effectiveness and the impact of various educational reforms, interventions, programs. International agencies and NGOs increasingly provide incentives (ideological, material) for countries to participate in learning assessment exercises. For many, education, and learning outcomes, are seen as directly linked to development progress.

National commitment to international EFA agenda has partly spurred growth in international, regional and national assessments Goal 2 Ensuring that by 2015 all children, particularly girls, children in difficult circumstances and those belonging to ethnic minorities, have access to and complete free and compulsory primary education of good quality Goal 6 Improving all aspects of the quality of education and ensuring excellence of all so that recognized and measurable learning outcomes are achieved by all, especially in literacy, numeracy and essential life skills Note: Goal 6 refers to all levels/types of education, and says nothing about comparability

Quality in education is multidimensional Concept. It refers to a learning environment characterized as: enabling equitable effective and efficient (both internal and external)

Explanations (3): World Polity (neo-institutional) argument, see Kamens and McNeely (2010) in the Comparative Education Review: Science has become the mode of understanding society; and the idea that societies, like organizations, can be successfully "managed" to achieve goals. Widespread agreement across nations about the desired outcomes of education legitimates international efforts to make mass education more accountable to society. Thus, national assessments and international testing acquire a good deal of external legitimacy. Intensification of global competitiveness both rhetoric and reality. Greater country mobility (both up and downwards) in the world economic system depends on the mobilization of a skilled and competent workforce Other reasons?

From Explanations to Critical Perspectives What are the limitations and weaknesses of learning assessments as presently conceived? What areas and dimensions of learning are missing from current assessment exercises? Under what conditions can clear and valid policy implications be drawn from learning assessments?

Critical Perspectives on Learning Assessments (1) International and regional assessments, and most national assessments, focus on a narrow range of school-based knowledge and skills--in language, mathematics and, in some cases, science and civics. In most educational systems, students are exposed to a much broader array of subject matter and knowledge domains for example, history, geography, environmental studies, social studies, art, music, literature, religious/moral education, physical education which are rarely assessed, but which constitute a significant percentage of a student s school week. If countries undertook assessments in these other subjects, would country rankings remain the same? Probably not. Should sub-par or average international performance in only one to three areas of the curriculum be the basis for calls for educational reform? Many learning assessments collect data on multiple dimensions of learning including knowledge, skills, attitudes and behavior, although reports of assessment results tend to focus almost exclusively on those pertaining to the knowledge or cognitive dimension. Would international rankings of countries look substantially different if attitudinal and/or behavioral dimensions of student learning were given more visibility?

Critical Perspectives on LAs (2) Almost all learning assessments are one-time (or periodic) snapshots of what students know, what levels of skill proficiency they have attained, and how knowledge and proficiencies are distributed. Almost no comparative assessments follow the same students over time. As such, they provide indirect (and incomplete) information about learning gains or losses over time, and changes in the unequal distribution of learning levels. Would longitudinal evidence about learning levels and inequalities show different patterns and relationships than the ones currently being reported? Almost all assessments do not collect information about what happens inside the classroom. LAs tend to emphasize input variables, on the one hand, and learning outcomes, on the other. They ignore variables that capture classroom-based learning processes and teaching practices. If effective teachers and teaching practices are the key to improved learning, as many argue, then how can assessments that compile little information about teachers and classroom teaching generate important information on policy reform levers?

Critical Perspectives on LAs (3) Assessments invariably concentrate on achievement in one curricular subject at a time. In doing so, they incorrectly assume that the actual coverage of a subject (the implemented curriculum) is relatively independent of the coverage in other subjects. In reality, school decisions about allocating instructional time to different school subjects are part of a zero-sum framework. If schools decide to devote more instructional time to, say, mathematics or science (highly valued subjects), this reduces time available to presumably less valued subjects, say, in the arts or the humanities. In short, the interdependence of subject emphases in school curricula--usually overlooked in comparative learning assessments--can affect content coverage and influence student learning. Learning assessments are either curriculum-based (TIMSS, PIRLS) or crosscurricular in their orientation (PISA). Many analysts are quick to draw policy conclusions about the need for specific curricular reforms when some assessments like PISA lack systematic information about curricular subjects, and their implementation in the school and classroom. This puzzling tendency underscores the need for caution when drawing curriculum-oriented policy implications from certain learning assessments.

Critical Perspectives on Learning Assessments (4) International assessments only examine learning outcomes among schooled children and youth. They do not test the knowledge levels of similarly aged young people who are currently--or permanently--out of school. Thus if policy makers are interested in an overall picture of what young people know, and whether they have acquired the skills and proficiencies necessary for adult roles, then the exclusion of non-tested populations bias (usually upwards) the national average. This is especially salient in countries in which participation rates are low or dropout rates are high.

Conclusions Learning assessments have become a worldwide phenomenon, a highly institutionalized component of almost all national education systems The spread of LAs has been especially noticeable in the so-called developing world during the past 10-15 years This global transformation reflects changes in the policy discourse of international agencies, in social science research of education, in the strengthening of accountability demands of international NGOs. It is imperative that interested parties both within and outside of the education system develop a critical perspective towards learning assessments, and their limitations and biases.

References Heinz-Dieter Meyer and A. Benavot (editors) In preparation, The Power, Politics and Culture of Global Educational Benchmarking: Who Succeeds at Pisa and Why? A. Benavot. 2012, Policies toward quality education and student learning: Constructing a critical perspective. Innovation-The European Journal of Social Science Research 25 (1): 67-77. A. Benavot. 2011, Improving the provision of quality education: Perspectives from textbook research. Journal of International Cooperation in Education 14 (2): 1-16. David Kamens and A. Benavot. 2011, National, regional and international learning assessments: Trends among developing countries, 1960-2009. Globalisation, Societies and Education 9 (2) 285-300. David Kamens and Connie McNeely. 2010. Globalization and International Testing and National Assessment. Comparative Education Review 54, 1: 5-27. A. Benavot and Erin Tanner. 2007, The growth of national learning assessments in the world, 1995-2006. Background paper for the Education for all global monitoring report 2008: Education for all by 2015: will we make it? Paris: UNESCO.

Learning outcomes as measured in assessments are becoming the predominant means to assess quality International assessment programs have increased in number, frequency and the number of participating countries. Participation very extensive in Region 1 countries In Latin America and Sub Saharan Africa regional assessments are strong. In some regions (East, South, West and Central Asia, Arab states) program participation is low. International Association for the Evaluation of Educational Achievement-IEA coordinates several international studies: TIMSS, PILRS, CIVIC (grades 4 & 8 & 12) 65 participating countries OECD hosts PISA (15 years old): includes OECD and some non-oecd countries 67 participating countries CONFEMEN hosts PASEC (grade 2 & 5) includes mainly Francophone African countries 21 participating countries Laboratorio or LLECE hosts SERCE covers Latin America (grades 3 & 6) 16 participating countries UNESCO-IIEP hosts SACMEQ includes Anglophone African countries (grade 6) 15 participating countries

Background In the decades following WWII, international policy discussions of education focused almost exclusively on establishing compulsory attendance provisions in newly independent countries and increasing access to primary or basic education. Eradicating literacy was an important policy priority, but not linked to the education agenda. In recent decades, especially since 2000, there is an growing focus on quality education school quality and qualityrelated issues. However, little international or academic consensus as to what quality in education entails: many stress the importance of learning outcomes, though disregard the enabling conditions of learning. In fact the international community will likely agree to some kind of access plus learning target goal for the post-2015 period

International policy Statements on quality education World Declaration on EFA: Meeting Basic Learning Needs 1990 The focus of basic education must be on actual acquisition and outcomes, rather than exclusively upon enrolment, continued participation in organized programmes and completion of certification requirements. Active and participatory approaches are particularly valuable in assuring learning acquisition and allowing learners to reach their fullest potential. It is, therefore, necessary to define acceptable levels of learning acquisition for educational programmes and to improve and apply systems of assessing learning achievement. (Article IV) 38

International policy Statements on quality education (2) Dakar Framework for Action: Education for All Meeting Our Collective Commitments 2000 Quality is at the heart of education, and what takes place in classrooms and other learning environments is fundamentally important to the future well-being of children, young people and adults. A quality education is one that satisfies basic learning needs, and enriches the lives of learners and their overall experience of living. (para 42) 39

The push to assess learning: multiple rationales from different stakeholders Ostensibly learning assessments should: Gather information about what students know and can do (in relation to what they have been taught) Motivate students to improve their knowledge and skills Motivate and encourage teachers to meet the identified needs of students Provide evidence to stakeholders of how well students have learned and how well teachers have taught Highlight curriculum expectations of authorities Provide feedback that enables teachers, students and parents to improve and guide instruction

Learning Assessments: Societal and governmental purposes Learning assessments should also Help gov ts. gauge (select) learning levels and the pace of progress over time and to hold schools and/or districts to account where progress is slow and interventions are needed Help gov ts set policy targets--measurable learning outcomes--and establish benchmarks for reform efforts Provide a legitimate platform to assess the impact of reforms over time Highlight what is possible and feasible to achieve from education reform and which policy levers within (or outside) the educational sector are effective in facilitating learning Provide employers with notion of the skills and knowledge of future workers

Outline of Presentation Goal of this Talk: Discuss the nature and worldwide spread of learning assessments (LA) international, regional and national from historical, global and critical perspectives 1. Introduction: Background, models of learning 2. The push to assess learning: organizational and societal rationales and societal 3. Historical trends/ Current patterns in international regional and national learning assessments 4. Explanations for the spread of LAs 5. Critical perspectives 6. Conclusions

Growing participation of developing countries in international assessments, 1959-2009, by year

Growing participation of developing countries in regional assessments, 1959-2009, by year

Classification of developing countries by types of learning assessments conducted, 1960-2008 International Assessment* (1960-2008) Regional Assessment** (1990-2008) Has conducted at least one National learning assessment (1995-2008) Has NOT conducted a National learning assessment or information is missing Participated in at least one international assessment Participated in at least one regional assessment South Africa; Swaziland; Djibouti Argentina; Brazil, Chile; Columbia; Dominican Republic; El Salvador; Guatemala; Honduras; Mexico; Panama; Paraguay; Peru; Venezuela Botswana; Zimbabwe Did not participate in any regional assessment Ghana; Nigeria Egypt; Jordan; Lebanon; Oman; Qatar; United Arab Emirates Indonesia; Korea, Republic of; Malaysia; Philippines; Singapore; Thailand, India Belize; Uruguay Azerbaijan; Georgia; Turkey MOST DEVELOPED COUNTRIES ARE HERE Cyprus; Tunisia; Yemen China; Macao; Papua New Guinea Iran Trinidad and Tobago Kazakhstan; Kyrgyzstan; Tajikistan; Uzbekistan

Classification of developing countries by types of learning assessments conducted, 1960-2008 International Assessment* (1960-2008) Did not participate in any international assessments Regional Assessment** (1990-2008) Participated in at least one regional assessment Did not participate in any regional assessment Has conducted at least one National learning assessment (1995-2008) Burkina Faso; Cameroon; Central African Republic; Guinea; Kenya; Lesotho; Madagascar; Malawi; Mauritius; Mozambique; Namibia; Niger; Senegal; Seychelles; Uganda; Zambia Mauritania Bolivia; Costa Rica; Cuba; Ecuador; Nicaragua Eritrea; Ethiopia; Gambia Algeria; Kuwait; Morocco; Saudi Arabia Myanmar; Cambodia; Cook Islands; Fiji; Kiribati; Lao, PDR; Samoa, Western; Solomon Islands; Tonga; Tuvalu; Vanuatu; Vietnam Bangladesh; Maldives; Pakistan Bahamas; Guyana; Haiti; Jamaica; St. Kitts/ Nevis Has NOT conducted a National learning assessment or missing Benin; Chad; Congo; Gabon, Mali, Tanzania; Togo Angola; Burundi; Cape Verde; Comoros; Zaire; Cote d Ivore; Equatorial Guinea; Guinea-Bissau; Liberia; Rwanda; Sao Tome and Principe; Sierra Leone; Somalia Bahrain; Iraq; Libya; Palestinian Autonomous Territories; Sudan; Syria, Brunei; East Timor; North Korea; Marshall Islands; Micronesia; Nauru; Niue; Palau; Tokelau, Afghanistan; Bhutan; Nepal; Sri Lanka; Taiwan Antigua/ Barbuda; Aruba; Barbados; Bermuda; Grenada; St. Lucia; St. Vincent/Grenadine; Suriname, Armenia; Turkmenistan

Initial summation re quantity of LAs There has been a significant increase in the number, frequency and country coverage of international and regional learning assessments Concurrently, and especially after 1990, there has been a huge increase in the number (and percentage) of countries conducting national learning assessments In fact: Out of more than 880 assessments carried out in developing world between 1960 and 2009, national assessments predominate. The table below breaks down all learning assessments by assessment type.