Teaching, testing and the CEFR: more than just levels José Noijons, Project Coordinator RELANG International Conference Towards CEFR Proficiency Guidelines-based Certification of Arabic as a Foreign Language University of Genoa, 6-7 September 2018
Languages Language Reading Speaking Writing Dutch Native Native Native English Excellent Excellent Excellent French Good Fair Fair German Good Good Fair Italian Arabic
RELANG Relating language curricula and examinations to the common European reference levels of language proficiency Promoting quality assurance in education and facilitating mobility European Centre for Modern Languages (ECML) and European Commission Cooperation on Innovative Methodologies and Assessment in Language Learning
What is the CEFR CEFR: Common European Framework of Reference, launched in 2001 by the Council of Europe Description of the process of mastering an unknown language by type of competence and sub-competence, using descriptors for each competence or sub-competence Descriptors without reference to any specific language Descriptors specify progressive mastery of each skill, graded on a sixlevel scale (A1, A2, B1, B2, C1, C2). A common basis for the elaboration of language syllabuses, curriculum guidelines, examinations, textbooks, etc. across Europe Basis for language learners in order to use a language for communication so as to be able to act effectively
Objectives of the CEFR Promoting plurilingualism Supporting plurilingual profile of individual learners Developing and reviewing the content of language curricula Defining positive can do descriptors adapted to the age, interests and needs of learners Designing and developing textbooks and teaching material Supporting teacher education and cooperation among teachers of different languages Enhancing quality and success in learning, teaching and assessment Facilitating transparency in testing and the comparability of certifications
CEFR Table 1 C2 Proficient User Independ ent User Basic User C1 B2 B1 A2 A1 Can understand with ease virtually everything heard or read. Can summarise information from different spoken and written sources, reconstructing arguments and accounts in a coherent presentation. Can express him/herself spontaneously, very fluently and precisely, differentiating finer shades of meaning even in more complex situations. Can understand a wide range of demanding, longer texts, and recognise implicit meaning. Can express him/herself fluently and spontaneously without much obvious searching for expressions. Can use language flexibly and effectively for social, academic and professional purposes. Can produce clear, well-structured, detailed text on complex subjects, showing controlled use of organisational patterns, connectors and cohesive devices. Can understand the main ideas of complex text on both concrete and abstract topics, including technical discussions in his/her field of specialisation. Can interact with a degree of fluency and spontaneity that makes regular interaction with native speakers quite possible without strain for either party. Can produce clear, detailed text on a wide range of subjects and explain a viewpoint on a topical issue giving the advantages and disadvantages of various options. Can understand the main points of clear standard input on familiar matters regularly encountered in work, school, leisure, etc. Can deal with most situations likely to arise whilst travelling in an area where the language is spoken. Can produce simple connected text on topics which are familiar or of personal interest. Can describe experiences and events, dreams, hopes and ambitions and briefly give reasons and explanations for opinions and plans. Can understand sentences and frequently used expressions related to areas of most immediate relevance (e.g. very basic personal and family information, shopping, local geography, employment). Can communicate in simple and routine tasks requiring a simple and direct exchange of information on familiar and routine matters. Can describe in simple terms aspects of his/her background, immediate environment and matters in areas of immediate need. Can understand and use familiar everyday expressions and very basic phrases aimed at the satisfaction of needs of a concrete type. Can introduce him/herself and others and can ask and answer questions about personal details such as where he/she lives, people he/she knows and things he/she has. Can interact in a simple way provided the other person talks slowly and clearly and is prepared to help.
CAVEATS! The CEFR does not set out to tell practitioners what to do, or how to do it. It raises questions but does not provide ready-made anwers. It is not the function of the CEFR to lay down the objectives that users should pursue or the methods they should employ. This does not mean that the Council of Europe is indifferent to these issues. Indeed, a number of specialists who have contributed to language policy projects have put a great deal of thought and work into the principles and practice of language learning, teaching and assessment over the years
The CEFR Model of Language Use Elements in action-oriented approach: Actions performed by persons - individuals and social agents A range of competences, both general and in particular communicative language competences Various contexts under various conditions and constraints to engage in language activities Language processes to produce and/or receive texts in relation to themes in specific domains
The user/learner s Competences
Language use and the user/learner
Linking to the CEFR Five inter-related sets of procedures of linking curricula and examinations to the CEFR 1. Familiarisation 2. Specification 3. Standardization & Benchmarking 4. Standard setting 5. Validation
Opportunities of Specification To describe and analyse in a detailed way the content of a curriculum, an examination or a test To provide evidence of the quality of the curriculum, examination or test to provide evidence of the relation between curricula, examinations/tests and the CEFR to provide guidance for developers of curricula and examinations to increase the transparency about the content and quality of curricula and examinations and their relationship to the CEFR
Claims through Specification and Standardization Claimed link to CEFR is based on specification only no indication of performance level or score required for CEFR-level in question Claims to be substantiated through standardisation Performance levels or item-difficulties judged in relation to CEFR levels.
Steps in the Standardization Phase 1. Adequate familiarization with the CEFR. 2. Training in rating productive skills tables and scales in the CEFR or the linking manual & scales or specific rating scales 3. Training in rating receptive tasks tables in the linking manual & specifications developed for the examinations or tests in question 4. Benchmarking performance samples 5. Standard setting of receptive tasks
Benchmarking for Productive Skills In holistically rated tests, the judgment on the CEFR level is direct Assistance needed for raters in giving valid judgments Main tool used for this special type of standard setting: benchmarking Benchmarking: providing one (or more) typical sample(s) to illustrate performance at a given level Benchmarking and standard setting are procedures requiring group decisions to be carefully prepared by appropriate training
Standard Setting for Receptive Skills For tests with a numerical score, performance standards have to be set Receptive skills (reading, listening) Underlying competences (grammar, vocabulary) A performance standard is the boundary (cut-off score) between two levels on the scale reported by a test Process to arrive at a cut-off score: standard setting
How to arrive at Standards? Group decisions (panel) Group is familiar with CEFR Test content has been specified in terms of the CEFR Standard setting procedures have been formalized Careful selection and training of panel members
Validity of the Linking Process Does the curriculum teach what it claims to teach in terms of the CEFR? Does the examination test what it intends to measure? Does the test accurately reflect the syllabus on which it is based AND reflect the descriptors in the CEFR? Does the content specification reflect all areas to be assessed in suitable proportions?
In Other Words Is your B1 my B1?