Learning Oriented Assessment A systemic approach
For a complete list of titles please visit: /elt/silt Also in this series: European Language Testing in a Global Context: Proceedings of the ALTE Barcelona Conference July 2001 Edited by Cyril J. Weir and Michael Milanovic IELTS Collected Papers: Research in speaking and writing assessment Edited by Lynda Taylor and Peter Falvey Changing Language Teaching through Language Testing: A washback study Liying Cheng The Impact of High- stakes Examinations on Classroom Teaching: A case study using insights from testing and innovation theory Dianne Wall Impact Theory and Practice: Studies of the IELTS test and Progetto Lingue 2000 Roger Hawkey IELTS Washback in Context: Preparation for academic writing in higher education Anthony Green Examining Writing: Research and practice in assessing second language writing Stuart D. Shaw and Cyril J. Weir Multilingualism and Assessment: Achieving transparency, assuring quality, sustaining diversity Proceedings of the ALTE Berlin Conference, May 2005 Edited by Lynda Taylor and Cyril J. Weir Examining FCE and CAE: Key issues and recurring themes in developing the First Certiicate in English and Certiicate in Advanced English exams Roger Hawkey Language Testing Matters: Investigating the wider social and educational impact of assessment Proceedings of the ALTE Cambridge Conference, April 2008 Edited by Lynda Taylor and Cyril J. Weir Components of L2 Reading: Linguistic and processing factors in the reading test performances of Japanese EFL learners Toshihiko Shiotsu Aligning Tests with the CEFR: Relections on using the Council of Europe s draft Manual Edited by Waldemar Martyniuk Examining Reading: Research and practice in assessing second language reading Hanan Khalifa and Cyril J. Weir Examining Speaking: Research and practice in assessing second language speaking Edited by Lynda Taylor IELTS Collected Papers 2: Research in reading and listening assessment Edited by Lynda Taylor and Cyril J. Weir Examining Listening: Research and practice in assessing second language listening Edited by Ardeshir Geranpayeh and Lynda Taylor Exploring Language Frameworks: Proceedings of the ALTE Kraków Conference, July 2011 Edited by Evelina D. Galaczi and Cyril J. Weir Measured Constructs: A history of Cambridge English language examinations 1913 2012 Cyril J. Weir, Ivana Vidaković, Evelina D. Galaczi Cambridge English Exams The First Hundred Years: A history of English language assessment from the University of Cambridge 1913 2013 Roger Hawkey and Michael Milanovic Testing Reading Through Summary: Investigating summary completion tasks for assessing reading comprehension ability Lynda Taylor Multilingual Frameworks: The construction and use of multilingual proiciency frameworks Neil Jones Validating Second Language Reading Examinations: Establishing the validity of the GEPT through alignment with the Common European Framework of Reference Rachel Yi-fen Wu Assessing Language Teachers Professional Skills and Knowledge Edited by Rosemary Wilson and Monica Poulter Second Language Assessment and Mixed Methods Research Edited by Aleidine J Moeller, John W Creswell and Nick Saville
Learning Oriented Assessment A systemic approach Neil Jones Consultant, Cambridge English Language Assessment and Nick Saville Cambridge English Language Assessment With input from Angeliki Salamoura Cambridge English Language Assessment
University Printing House, Cambridge CB2 8BS, United Kingdom Cambridge University Press is part of the University of Cambridge. It furthers the University s mission by disseminating knowledge in the pursuit of education, learning and research at the highest international levels of excellence. Information on this title: /9781316507889 UCLES 2016 This publication is in copyright. Subject to statutory exception and to the provisions of relevant collective licensing agreements, no reproduction of any part may take place without the written permission of Cambridge University Press. First published 2016 Printed in XXXX A catalogue record for this publication is available from the British Library Library of Congress Cataloging-in-Publication Data Names: Jones, Neil, Dr., author. Saville, Nick, editor. University of Cambridge. Local Examinations Syndicate. Title: Learning oriented assessment : a systemic approach / Neil Jones and Nick Saville. Description: Cambridge ; New York : Cambridge English, [2016] Series: Studies in language testing ; 45 Includes bibliographical references and index. Identiiers: LCCN 2016000619 ISBN 9781316507889 (paperback) Subjects: LCSH: Language and languages--examinations. Language and languages--europe. Education, Higher--Evaluation. Educational tests and measurement. Communication in education. Language--Examinations Europe--Languages. Classiication: LCC P53.4.J66 2016 DDC 418.0076--dc23 LC record available at http://lccn.loc. gov/2016000619 Cambridge University Press has no responsibility for the persistence or accuracy of URLs for external or third-party internet websites referred to in this publication, and does not guarantee that any content on such websites is, or will remain, accurate or appropriate. Information regarding prices, travel timetables, and other factual information given in this work is correct at the time of irst printing but Cambridge University Press does not guarantee the accuracy of such information thereafter.
To all the learners we have taught from 1970 to 2015
Contents List of tables and igures Abbreviations Acknowledgements Series Editors note x xi xii xiv 1 Learning Oriented Assessment: An overview 1 1.1 The organisation of this volume 6 1.2 What is learning? 7 1.3 What is language learning? 8 1.4 What is to be learned? 9 1.5 The roles of assessment in learning 10 1.6 Aligning large-scale and classroom assessment 12 1.7 In summary 14 2 The roots of Learning Oriented Assessment 15 2.1 Cambridge English and Learning Oriented Assessment 16 2.2 Origins and emphases of learning-oriented approaches 16 2.3 Deining learning-oriented assessment: Process, policy or principles? 24 2.4 Speciic learning-oriented methods 28 3 What is learning? 30 3.1 Constructivism 30 3.2 An appropriate model of cognition and learning 32 3.3 Task-based interaction 33 3.4 Roles of teachers and learners 41 3.5 In summary 46 4 What is language learning? 48 4.1 Languages as a special case 48 4.2 The Common European Framework of Reference for Languages 49 4.3 Natural language acquisition 51 4.4 Second Language Acquisition research 51 4.5 In summary 57 vii
Learning Oriented Assessment 5 What is to be learned? 58 5.1 The desired outcomes of learning 58 5.2 The nature of language proiciency: Construct deinition 62 5.3 The content of learning: Curricular objectives 65 5.4 In summary 66 6 The role of large-scale assessment in learning 68 6.1 Proiciency testing: The importance of criterion reference 69 6.2 Scale construction 70 6.3 Item Response Theory 71 6.4 Item banking 72 6.5 Performance assessment 74 6.6 Validity and reliability of large-scale assessment 75 6.7 Large-scale assessment: Evidence of and for learning 78 6.8 In summary 79 7 Learning-oriented assessment in the classroom 81 7.1 The nature of classroom learning-oriented assessment 81 7.2 Domain-speciic and generalisable learning skills 82 7.3 Learning-centred and content-centred activities 83 7.4 Classroom evidence of and for learning 84 7.5 Learning-oriented assessment: An ecological model 88 7.6 In summary 91 8 Aligning large-scale and classroom assessment 93 8.1 Alignment of goals 93 8.2 Aligning interpretations of standards 94 8.3 Construct-based alignment of assessment 97 8.4 Evidence within a Learning Oriented Assessment model 100 8.5 The validity of Learning Oriented Assessment 103 9 Implementing Learning Oriented Assessment 106 9.1 Educational policy-making: The global scale 106 9.2 Worlds of diference: Predictable implementation issues 107 9.3 Asset Languages: A cautionary tale 109 9.4 Technology to the rescue? 110 9.5 New contexts of learning 111 9.6 Positive impact by design 113 9.7 Steps in implementation 117 9.8 A conclusion 118 viii
Contents Appendices Appendix 1 Assessment for learning: 10 principles (Assessment Reform Group 2002) 122 Appendix 2 The Teaching and Learning Research Programme (TLRP): 10 principles of efective pedagogy (Teaching and Learning Research Programme 2007) 125 Appendix 3 James and Pedder s (2006) hypotheses on assessment for learning 127 References 128 Author index 140 Subject index 144 ix
List of tables and igures Figure 1.1 A complementary relationship between large-scale and classroom assessment Figure 1.2 Four worlds of learning Figure 1.3 The four worlds linked by a focus on task Figure 1.4 Evidence for learning: The macro level Figure 1.5 Evidence for learning: The micro level Figure 4.1 CASP general and speciic principles Figure 5.1 Illustration of a construct (Khalifa and Weir 2009:43) Figure 6.1 Item banking approach to scale construction and use (adapted from Jones and Saville 2007) Figure 6.2 A socio-cognitive framework (adapted from Weir 2005b) Table 7.1 Formative assessment cycles (adapted from Wiliam 2006) Figure 7.1 The classroom within a Learning Oriented Assessment model Figure 8.1 Four worlds of learning revisited Figure 9.1 Steps in implementation: From policy to evaluation of outcomes x