14th EALTA Conference Language testing and assessment Its role in mobility and social integration 1-3 june 2017, Sèvres, France TRINCHERI, Miranda / Consorcio ELSE BROWN, Ana / Laboratorio de Idiomas, UBA IES Lenguas Vivas
CELU certification: Introduction, development CELU / CELPE-Bras: regional integration in SA Construct, theoretical view Test design: integrated tasks Task assessment Analysis: tasks and its parts Conclusions: impact
Argentina s official certification of SpanishL2. Officially acknowledged by the Ministry of Education and by the Ministry of Foreign Affairs of Argentina. Started in 2004 and been growing since. Administered by the ELSE Consortium. More than 50 exam centres
Bakhtin s perspective on social use of language (Bajtin, 1982). Critical language testing (Shohamy,1998) Performance test (Alderson, 1995; Mc Namara, 2000; Schlatter et.al., 2005) Impact (Bachman, 1996; Watanabe, 2004). Proficiency (Dell'Isola et.al., 2003)
Dell Isola et.al., 2003 Speakers can participate in different activities within their resources with or without help from others. Different levels of achievement show different levels of proficiency in language use" An exam of language use
Based on a concept of language as a combined action of participants with a social aim and also, considering language and culture as inseparable, the idea of proficiency that underlies the exam consists of the adequate use of language in order to perform acts in the real world. Celpe-Bras considers context and aim of the participants in the interaction. (Brasil, 2015, p.9) Reciprocal agreement between Argentina and Brasil governments to acknowledge both exams. (05/08/2005)
Format Part Skills Timing Activities Written Reading Listening Writing 3 hours Listening and writing with a particular purpose to a specific audience No explicit questions about the language or its grammar are asked Reading Oral Listening Interacting 15 to 20 minutes Reading short texts, commenting on them while interacting with the examiner 8
CELU levels Intermediate Advanced Language users at the Intermediate level can speak and write in Spanish quite fluently and naturally, in simple family, working and social environments. They may lack accuracy when facing unknown situations or when having to express subtleties of meaning. They manage to communicate in working as well as academic contexts. Language users at the Advanced level can speak and write in Spanish with ease and spontaneity in a wide range of social situations. They can use the language appropriately in working as well as academic contexts. Full descriptions: http://www.celu.edu.ar/en/content/levels
Genre organizes language experience. Skills, as in real life, can be assessed from an integrated perspective. Tasks expect adequate answers according to a genre, a role and a purpose. Action-oriented and contextualized tasks.
You are one of the artists that participated in the contest described in the article and you didn t win. Write an e-mail to the judges to express your disagreement with the results. Usted es uno de los artistas que se presentó al concurso que se menciona en la siguiente noticia y no ganó. Escríbale un correo electrónico al presidente del jurado para expresar su disconformidad por el resultado.
You were part of the expedition team mentioned in the following text. Tell on your travel blog how the expedition to the garbage island was and comment on your experience. Usted formó parte del equipo de exploradores mencionado en el siguiente texto. Cuente en su blog de viajes cómo fue la expedición a la isla de la basura y comente su experiencia.
Each task is assessed by two independent raters. Raters attend standardisation sessions for every exam. Holistic rating scales with five different performance levels. Expected answer A sample of answers is collected and their level is agreed among raters.
Basic (2) Advanced (5) The text doesn t fulfill genre or purpose. Fulfills the purpose of the task. Relationship between his/her role and the addressee is not adequate or is different from the one required. Recontextualizes inadequately or wrongly input information. Or the texts lacks cohesion and coherence. Linguistic resources are very limited. Builds an adequate relationship between his/her role and the addressee. Finds a position within the genre and produces a personal text. Retrieves information from the input. Rewords and organizes input information, following required genre characteristics. The text is coherent and cohesive. Lexicon and structures are varied and adequate. Could be occasional inadequacies which not affect reading fluency.
In which ways the candidate plays an active role in the language that it s being tested? We define active role as social actions that are embedded in the linguistic choice the candidate make. For example, the means chosen to fulfill the purposes of the text. Methodology Analysis of tasks (2014/2015) -Required genres -Roles -Purposes
4 exams, 16 written tasks CELU 114 CELU 214 CELU 115 CELU 215 June 2014 November 2014 June 2105 November 2015
INPUT OUTPUT ROLE ADDRESSEE PURPOSE DOMAIN
ACT INPUT OUTPUT ROLE ADDRESSEE PURPOSE DOMAIN 1 Interview Article School principal Specialized magazine Spread PUBLIC 2 News E-mail (f) Artist Judges Express disagreement PRIVATE 3 Article E-mail (inf) Friend Friend Comment and advice PRIVATE 4 News Letter to the editors Motorist Newspaper Give opinion PUBLIC
The exam offers the candidate a range of social processes to interact with The role and the purpose assigned in the task allow the candidate to interact with different domains: public and private, personal and professional All the tasks are designed to promote linguistic and social activity interaction as a mean of assessment
ELSE Consortium: 3 universities (2004), 38 universities (2017) Training of examiners all over the world, academic network focused on SL2 assessment Research: periodic conferences, studies and publications CELU Archive: it gathers all the exams and it s used by researchers. Currently, more than 18,000 written texts and 2500 interviews being coded. Consortium activities: language courses, class materials, workshops, etc.
Critical language testing assumes that the act of testing is not neutral. Rather, it is both a product and an agent of cultural, social, political, educational and ideological agendas that shape the lives of individual participants, teachers and learners (Shohamy, 1998, 336).
Alderson, J. C., Clapham, C., & Wall, D. (1995). Language test construction and evaluation. Ernst Klett Sprachen. Bajtin, M. M. (1982). Este tica de la creacio n verbal. Mexico: Siglo veintiuno editores. Bachman, L. y Palmer, A. (1996). Language testing in practice: designing and developing useful language tests, Oxford, Oxford University Press. Brasil (2015). Certificado de Proficiência em Língua Portuguesa para Estrangeiros: Manual do Aplicador. Secretaria de Educação Superior (SESu). Brasília: MEC. Available at: http://www.ufrgs.br/acervocelpebras/arquivos/manuais/manual-do-aplicador-2015 CELPE Bras Web site http://celpebras.inep.gov.br/inscricao/ CELU Web site http://www.celu.edu.ar/ Dell'Isola, R. L., Scaramucci, M. V., Schlatter, M., & Júdice, N. (2003). A avaliação de proficiência em português língua estrangeira: o exame CELPE-Bras. Revista Brasileira de Linguística Aplicada, 3(1), 153-164. Mc Namara, T (2000) Language testing, Oxford, Oxfod University Press. Schlatter, M., Almeida, A. D. N., Fortes, M. S., & Schoffen, J. R. (2005). Avaliação de desempenho e os conceitos de validade, confiabilidade e efeito retroativo. A redação no contexto do vestibular, 11-35. Shohamy, E. (1998). Critical language testing and beyond. Studies in educational evaluation, 24(4), 331-345. Watanabe, Y. (2004). Methodology in washback studies en Cheng, L; Watanabe,Y; Curtis, A (eds), Washback in language testing, research contexts and methods, London, Lawrence Erlbaum.
Ana Brown / Laboratorio de Idiomas, UBA & IES Lenguas Vivas analaura.brown@gmail.com Miranda Trincheri / Universidad de Buenos Aires & Consorcio ELSE mirandatrincheri.celu@gmail.com