Breaking through the Ceiling Effect : Predictors of the Impact of Immersion Learning

Breaking through the Ceiling Effect : Predictors of the Impact of Immersion Learning Experiences on Student Attainment of Advanced Russian L2 Proficiency Gina M. Peirce Department of Linguistics University of Pittsburgh December 2011

Introduction. Constraints on U.S. students acquisition of advanced foreign language proficiency within the traditional learning context of a four-year university classroom instructional program are a well-known problem in the field of Second Language Acquisition (SLA). This problem is particularly acute for languages that are categorized by the Defense Language Institute as relatively difficult for native English speakers to learn, such as Slavic and East Asian languages and Arabic, among others. For example, Rifkin (2005) found that students who learned Russian only in the traditional domestic classroom context at the University of Wisconsin-Madison typically scored at the Intermediate Mid level on the American Council on the Teaching of Foreign Languages (ACTFL) scales of speaking and reading proficiency at the conclusion of their fourth-year language courses. Some exceptional U.S. students in traditional language programs attain scores of Intermediate High, but very few are able to enter the Advanced proficiency range. While Russian is considered a critical language by the U.S. government because an increased number of highly proficient speakers are needed to fill federal jobs, most of these positions require minimum scores of 2 on the Interagency Language Roundtable (ILR) proficiency scale which is approximately equivalent to Advanced level proficiency on the ACTFL scale for a candidate to meet the job requirements. Effectively conducting research with original language sources in the context of an academic career typically necessitates Advanced level proficiency as well. Therefore, a serious mismatch exists between the preparation provided by traditional U.S. university language programs and the level of proficiency that is required for college graduates to pursue a professional academic or governmental career utilizing their second language skills. According to Rifkin, traditional classroom Russian language learning is, most likely, 1

constrained by a ceiling just below the advanced level. Without an immersion experience, students of Russian will likely find it difficult, if not impossible, to break through this ceiling into advanced level proficiencies (13). After comparing proficiency outcomes data from traditional classroom instruction with data from the prominent domestic immersion program offered by the Russian language summer school at Middlebury College in Vermont, as well as other research studies on proficiency outcomes from study abroad immersion programs, Rifkin concludes that language instructors should vigorously advocate for their students to participate in both domestic immersion experiences and study abroad programs (15), which he views as offering the only real possibility for overcoming the so-called ceiling effect that constrains L2 acquisition by university students. Data from a large-scale study by Davidson (2010) of proficiency outcomes of study abroad programs in Russia and other countries of the former Soviet Union provide additional support for this contention, with particular emphasis on the benefits of long-term experiences in which students spend a full academic year immersed in the target language and culture. Several other researchers such as Freed, Segalowitz, and Dewey (2004), Hernández (2010), and Serrano, Llanes, and Tragant (2011) have made explicit comparisons of the L2 proficiency outcomes of traditional classroom instruction, domestic immersion programs and/or study abroad immersion programs. Additional researchers have proposed various psycholinguistic factors as predictors of the extent of L2 proficiency gains in each type of learning context, such as Sunderman and Kroll (2009), DeKeyser (2010), and Golonka (2006), or have reported on particularly effective instructional methods within a certain context, such as Brown (2009). This paper will review existing literature that compares the effects of different types of university-level instructional contexts on attainment of advanced L2 proficiency, with 2

the central objective of identifying factors that most strongly predict the effectiveness of different learning contexts and of specific instructional programs in enabling U.S. students to cross the barrier between intermediate and advanced levels of proficiency in the Russian language or, in other words, to break through the ceiling effect. In addition, the paper will suggest directions for future research on this topic. Literature Review. A considerable body of scholarship recognizes the instructional context as a significant factor affecting both acquisition rate and ultimate attainment of L2 learners. For example, a state-of-the-art article by Teresa Pica notes that 1985 and 1998 studies by Merrill Swain identified learners access to input that was meaningful, copious, and comprehensible as an important benefit of L2 immersion learning (Pica, 2009, 482). However, Swain s analysis found that immersion programs led to a greater advantage in learners development of receptive reading and listening skills than in their production accuracy. Pica also reviews studies of SLA from the perspective of information-processing theory, noting in particular a 1997 study by Robert DeKeyser focusing on the automatization of L2 knowledge. DeKeyser argued that a sequence of explicit instruction in grammatical rules (as is typically delivered in a domestic classroom context), followed by opportunities for learners to practice and apply these rules (which are abundant in a study abroad context), could most effectively facilitate learners acquisition of implicit knowledge of morphosyntactic rules in the L2 and their ability to accurately apply these rules in a wide range of communicative situations (Pica, 2009, 483). A study by Freed, Segalowitz, and Dewey (2004) was among the first to directly compare L2 proficiency gains by U.S. university students in traditional classroom, domestic immersion, and study abroad contexts. This study compared gains in oral fluency by groups of students who studied French for one semester in a traditional classroom or study abroad context or for one 3

summer term in an intensive domestic immersion program (which appeared to match the profile of the Middlebury College intensive summer language school examined by Rifkin in connection with Russian instruction). All students completed pretests and posttests in the form of oral interviews in French, as well as a written Language Contact Profile instrument * detailing the extent of their contact with and use of both French and English outside of the classroom. For the domestic immersion group, Freed, Segalowitz, and Dewey found statistically significant gains between pretest and posttest levels of five out of nine measures of speaking performance. The study abroad group made statistically significant gains in overall speech fluidity although their average gain was less than half that of the domestic immersion group but they did not make statistically significant gains on any single oral fluency measure. The group receiving traditional domestic classroom instruction failed to make any gains in overall speech fluidity and actually showed a significant loss on one oral fluency measure. The authors attribute these results in part to the fact that the domestic immersion group reported significantly more out-of-class contact hours in French than the other two groups, while the study abroad group reported significantly more out-of-class contact hours in English than the domestic immersion group. (The total number of classroom contact hours in French was similar for the domestic immersion and study abroad groups.) This study casts doubt on the frequently held, and yet too often unexamined, assumption that students participating in language immersion programs abroad necessarily receive more exposure to the target language and make greater oral proficiency gains than their peers who study the L2 in a program located in their home country. More recently, Hernández (2010) compared oral proficiency outcomes of study abroad * An online language contact reporting instrument, designed for submission of weekly reports by study abroad students on the extent of their target language utilization in a variety of situations, was presented by Dan E. Davidson, President of American Councils for International Education, and colleagues on November 18, 2011 at the annual conference of the Association for Slavic, East European and Eurasian Studies. This new instrument may enable more fine-grained analysis of out-of-class L2 usage in future studies of language immersion programs. 4

and at-home language learners, consisting of U.S. university students who spent a semester studying Spanish through either a study abroad program in Spain or traditional domestic classroom instruction. The students completed pretest and posttest Simulated Oral Proficiency Interviews (SOPIs), a questionnaire on integrative and instrumental motivations for language study, and a Language Contact Profile on their out-of-class contact hours in Spanish (unsurprisingly, the study abroad group reported more than twice as many hours as the at-home group). On the pretest SOPI, all students scored in the ACTFL Intermediate range; but on the posttest, 75% of the at-home students compared with only 20% of the study abroad students showed no oral proficiency gains. Among the study abroad students, 40% attained Advanced ratings on the posttest, and 25% gained two sublevels over their pretest scores. Hernández s statistical analyses showed that students within each group who had higher integrative motivation scores interacted significantly more with Spanish outside of class than did other students. Such interaction accounted for 48% of variance in proficiency gains for the study abroad students, and yet it was not a significant predictor for the at-home students. Hernández concludes that these results suggest the traditional domestic classroom learning context did not support the development of advanced language competence (659) and did not provide students with sufficient access to authentic language use in communicative contexts in order to foster significant second language acquisition growth (660). However, his study does not address the domestic immersion context, which was found by Rifkin as well as Freed, Segalowitz, and Dewey to be highly effective in fostering L2 proficiency gains. Therefore, Hernández s work also does not support the belief that study abroad is necessarily superior to all other language learning contexts. A study by Serrano, Llanes, and Tragant (2011) compared L2 proficiency outcomes of a 5

study abroad immersion program with those of both domestic intensive (25 hours of classroom instruction per week) and semi-intensive (10 hours per week) immersion programs. They investigated the effects of each of these three learning contexts on the development of Spanish university students written and oral production in English through measures of fluency, syntactic complexity, lexical complexity, and accuracy. All subjects completed a pretest and posttest with oral narrative and written composition tasks. MANCOVA analysis of the test results showed no significant differences in speaking or writing proficiency outcomes between the study abroad and domestic intensive groups, but the study abroad group scored significantly higher than the domestic semi-intensive group on measures of oral and written fluency and lexical complexity. While these results suggest that the more concentrated intensive immersion program led to greater proficiency gains than the semi-intensive program in the domestic setting (given that both domestic programs provided the same number of total classroom contact hours), methodological limitations of this study cast doubt on the comparison between study abroad and domestic intensive programs. Serrano, et al. did not gather data on the study abroad group s outof-class contact hours with the target language as in the studies by Freed, et al. and Hernández; though the authors specified that both of their domestic immersion groups had few opportunities for out-of-class contact with the L2, unlike Freed s domestic immersion group which had extensive extracurricular language practice opportunities. The absence of data on total hours of L2 exposure for the study abroad group in the article by Serrano, et al. impedes examination of the effects on proficiency outcomes of instructional environment features other than time on task. Sunderman and Kroll (2009) took an innovative approach to comparing L2 proficiency gains in study abroad and domestic instructional contexts, focusing on individual differences in 6

cognitive processing abilities. They hypothesized that individual working memory capacity influences a learner s ability to process the L2 while simultaneously suppressing the L1 in a study abroad environment. Their subjects consisted of two groups of U.S. university students, one of which had studied Spanish exclusively in a domestic instructional context, while the other recently participated in a semester or summer study abroad program. Students performed a reading span task to test their working memory capacity, as well as comprehension and production tasks to test their Spanish proficiency through reaction time and accuracy measures. The group that had studied abroad was significantly faster and more accurate on both L2 proficiency tasks, and particularly the production task. However, regression analysis also showed that working memory significantly affected the comprehension task results independently of study abroad experience, while production test data indicated that study abroad experience led to significantly increased L2 accuracy only for students who demonstrated a working memory capacity above a threshold score on the reading span test. For students with lower levels of internal cognitive resources, study abroad experiences resulted in an estimated effect on production accuracy that was not different than zero. In other words, it was as if these individuals had not studied abroad (Sunderman and Kroll, 2009, 92). The authors speculate that for these students, managing the cross-linguistic competition in the immersion environment becomes too challenging of a task and the individual simply reverts to speaking L1 or suffers other costs to language processing (95). Sunderman and Kroll s findings highlight the intriguing possibility that no single instructional context is most conducive to L2 proficiency gains for all students, and that the optimal language program type varies with the cognitive profiles of individual learners. A mixed methods study by DeKeyser (2010) further emphasized the importance of cognitive resources and an information-processing perspective for examining L2 proficiency 7

gains. DeKeyser hypothesized that learners development of production accuracy in an immersion setting is largely a function of their monitoring of the explicit knowledge of L2 grammar and vocabulary that they have previously acquired through classroom instruction, which is cognitively demanding and requires a high level of motivation. Through questionnaires, interviews, and participant observation of a group of U.S. university students on a six-week immersion program in Argentina, he collected qualitative data on the students perceptions and monitoring of their own L2 knowledge and the effects of feedback from interactions with native Spanish speakers on their learning. In addition, quantitative data on students oral accuracy was obtained from recorded interviews in Spanish, revealing that their pre-program written proficiency tests measuring explicit L2 knowledge but not their pre-program language aptitude tests were a significant predictor of individual post-program oral production accuracy. DeKeyser observed that despite two or more years of previous classroom instruction in Spanish, few students had sufficient declarative knowledge of grammar to allow for effective monitoring, and students entering the immersion program with greater explicit knowledge felt they learned more from informal interactions with native Spanish speakers than did those starting with less explicit knowledge (84). He reported that the students generally low levels of L2 production and comprehension led to a loss of motivation and a reversion to greater use of English near the end of the six-week program. DeKeyser concluded that L2 learners must acquire explicit knowledge of basic target language structures through classroom instruction before they can achieve significant proficiency gains through study abroad immersion experiences. His findings vividly illustrate the point that the popular concept of fast and effortless improvement in proficiency [through study abroad] is vastly exaggerated at best, and perhaps more myth than reality (80). 8

DeKeyser s work corroborates a previous study by Golonka (2006), who investigated metalinguistic variables as predictors of U.S. students ACTFL Oral Proficiency Interview (OPI) gains from a pretest score of Intermediate High to a posttest Advanced rating over the course of a semester study abroad immersion program in Russia. Golonka found the most effective predictors of students crossing of the Advanced threshold to be their pre-program explicit grammatical knowledge, measured by the American Council of Teachers of Russian (ACTR) Qualifying Grammar Test; and their vocabulary, accuracy, and self-monitoring (rates of selfcorrection of errors and sentence repair), measured by the pretest OPI. She reports that this model accounted for 47% of variance in proficiency gains during the study abroad program, predicting with 86.4% accuracy which students ended up in the group of gainers and which in the group of null gainers. While the Sunderman and Kroll, DeKeyser, and Golonka studies do not address the possibility of differential proficiency outcomes from study abroad and domestic immersion programs, their accumulated findings show that any thorough examination of the effectiveness of immersion learning contexts must not neglect the role of individual cognitive differences, along with metalinguistic factors such as self-monitoring, in L2 acquisition. Another promising, yet underexplored, area of research analyzes domestic nonimmersion instructional programs that have achieved unusually strong L2 proficiency outcomes. For example, a study by Brown (2009) reported the results of an advanced Russian course taught for one semester at Brigham Young University, which engaged students in parliamentary-style debate and Model United Nations along with thematically related writing, reading and listening activities to enhance their proficiency. (It should be noted that these subjects differed from the broader population of U.S. university students in that most had extensive informal language experience from living in a Russian-speaking country, presumably as missionaries, but relatively 9

little prior formal instruction.) Five of the 14 students in the course, all of whom began with an oral proficiency rating of Advanced Mid or higher, gained one sublevel between pretest and posttest ACTFL OPIs, and seven of the 14 gained two or more sublevels on ACTFL Written Proficiency Tests (WPTs) from their initial levels of Intermediate High through Advanced Mid. Brown concludes from these impressive results that innovative curricular design and development in the university foreign language classroom can equal if not exceed uptake that occurs in extended immersion environments (534). Directions for Future Research. To examine the validity of such claims, further research is needed in order to integrate studies of immersion and non-immersion language instructional programs and consider the transferability between contexts of key features of demonstrated successful models of each type. This research should focus on the population of U.S. university students who are native speakers of English, are enrolled in instructional programs in Russian as a second language, and score in the ACTFL Intermediate range of speaking proficiency in Russian. Research designs should incorporate analysis of existing published and unpublished data on subjects from the target population, as well as collection of new data from multiple sources. Key questions in need of further investigation include: 1) Controlling for hours of instruction, which type of L2 learning environment (traditional domestic classroom, domestic immersion program, or study abroad immersion program) is most conducive to producing proficiency gains among students from the target population?; and 2) Within each of these three language program types, which factors are the strongest predictors of the percentage of students from this population who will attain Advanced level proficiency in Russian during the program? Promising research instruments for future studies include pre-program and post-program Oral Proficiency Interviews (OPIs) administered to students by ACTFL-trained and certified 10

testers on either an official or unofficial (in-house) basis; written pretests of explicit grammatical knowledge, which are typically utilized for course placement purposes in L2 immersion programs; and written posttests where available. Telephone or in-person interviews with language program coordinators could also be employed to collect qualitative data on pedagogical approaches (sheltered versus unsheltered immersion, explicit grammatical instruction, etc.) and the extent and types of opportunities for student exposure to the L2 outside of the classroom. Following these data collection procedures, ANOVA statistical analysis and post-hoc tests (where appropriate) should be conducted to test for significant differences among proficiency outcomes from traditional domestic classroom, domestic immersion, and study abroad immersion environments. In addition, ANOVA statistical analysis and post-hoc tests (where appropriate) should be conducted to test for significant differences among proficiency outcomes from programs utilizing different pedagogical methods within each type of instructional context. Regression analysis should be used to investigate relative effect sizes of potential predictors of student proficiency gains, such as: extent and types of student L2 exposure outside of the classroom; instructional methods; pretest scores on oral interviews, written placement tests, and any other assessments of pre-program communicative and grammatical competence; and student demographic characteristics. Reports on research findings should highlight any program design factors that are found to have relatively large effect sizes and are transferable across different instructional contexts. Anticipated outcomes of such research would include: 1) comparison of the demonstrated effectiveness of domestic non-immersion, domestic immersion, and study abroad immersion programs in producing gains among students who begin at the ACTFL Intermediate level of speaking proficiency in Russian; 2) identification of factors predicting the extent of student 11

proficiency gains within each type of instructional context; and 3) interpretation of the above findings within the frameworks of previous psycholinguistic research on individual differences in cognitive resources of L2 learners such as Sunderman and Kroll (2009), metalinguistic factors in L2 acquisition such as Golonka (2006) and DeKeyser (2010), and application of Vygotskian concepts of proximal development and scaffolding to language teaching such as Brown (2009). This research would advance understanding among linguistics scholars, language instructors, and university administrators of the psycholinguistic attributes of immersion experiences that facilitate student attainment of advanced L2 proficiency. It could enhance teaching and learning by identifying curriculum features (in addition to time on task) that contribute to the effectiveness of immersion and non-immersion programs in helping students to reach advanced L2 proficiency levels and that distinguish highly effective from less effective programs, as well as identifying design components of highly effective immersion programs that are transferable to non-immersion contexts and vice versa. Publicizing the results of such research could also strengthen the case for expanded financial support of well-designed study abroad and domestic language immersion experiences by colleges, universities, and governmental and nongovernmental funding agencies. Dissemination of findings on effective program models and pedagogical approaches to language teaching would promote improvements in the design of traditional classroom-based curricula, as well as domestic and study abroad immersion programs. This in turn could help to increase the numbers of U.S. students attaining advanced L2 proficiency in each of these instructional contexts, leading to an increased supply of university graduates with the necessary qualifications to pursue academic, governmental or other professional careers using their skills in critical less commonly taught languages, including Russian. 12

References Brown, N. A. (2009). Argumentation and debate in foreign language instruction: A case for the traditional classroom facilitating advanced-level language uptake. The Modern Language Journal, 93(4), 534-549. Davidson, D. E. (2010). Study abroad: When, how long, and with what results? New data from the Russian front. Foreign Language Annals, 43(1), 6-26. DeKeyser, R. (2010). Monitoring processes in Spanish as a second language during a study abroad program. Foreign Language Annals, 43(1), 80-92. Freed, B., Segalowitz, N., & Dewey, D. (2004). Context of learning and second language fluency in French: Comparing regular classroom, study abroad, and intensive domestic immersion programs. Studies in Second Language Acquisition, 26(2), 275-301. Golonka, E. M. (2006). Predictors revised: Linguistic knowledge and metalinguistic awareness in second language gain in Russian. The Modern Language Journal, 90(4), 496-505. Hernández, T. A. (2010). Promoting speaking proficiency through motivation and interaction: The study abroad and classroom learning contexts. Foreign Language Annals, 43(4), 650-670. Pica, T. (2009). Second language acquisition in the instructional environment. In W. C. Ritchie, & T. K. Bhatia (Eds.), The New Handbook of Second Language Acquisition (2nd ed., pp. 473-501). Bingley, UK: Emerald Group Publishing Limited. Rifkin, B. (2005). A ceiling effect in traditional classroom foreign language instruction: Data from Russian. The Modern Language Journal, 89(1), 3-18. Serrano, R., Llanes, A., & Tragant, E. (2011). Analyzing the effect of context of second language learning: Domestic intensive and semi-intensive courses vs. study abroad in Europe. System: An International Journal of Educational Technology and Applied Linguistics, 39(2), 133-143. Sunderman, G., & Kroll, J. (2009). When study-abroad experience fails to deliver: The internal resources threshold effect. Applied Psycholinguistics, 30(1), 79-99. 13