WORLD LANGUAGE PROGRAM EVALUATION

WORLD LANGUAGE PROGRAM EVALUATION Carmel Clay Schools 2013-2014 1 P a g e

Acknowledgements This document is the result of the efforts put forth by a dedicated committee of parents, teachers, and administrators. The intent of their work was to provide a data-driven, objective evaluation of Carmel Clay Schools K-12 World Language program for the purpose of providing information to guide the continuous school improvement process. Sincere appreciation is expressed to: Committee members for their understanding of the global society in which we live, their appreciation for the role of world language in schools, and for their commitment and sustained efforts throughout this project Parents, students, teachers, counselors and administrators for providing the data that formed the basis of the committee s work. Thank you to teachers for participation in the instructional audit and focus groups as well as students and counselors for participation in focus groups. Dr. Amy Dudley and Dr. Linda Thompson for guidance and planning throughout the program evaluation process. Brenda Cook and April O Cull, curriculum and instruction assistants, for organizing and preparing materials to facilitate the committee s work. Rachel Sever Supervisor of Learning September, 2014 2 P a g e

Carmel Clay Schools World Language Program Evaluation Table of Contents Introduction... 4 Review of Literature... 5 References... 8 Data Collection... 9 Findings... 10 Recommendations... 16 3 P a g e

Introduction In the fall of 2013, a committee of teachers, administrators and parents was convened for the purpose of conducting an evaluation/needs assessment of the Carmel Clay Schools world language program. Throughout the 2013-2014 school year, the committee met to engage in the following activities: 1. Study current educational literature on best practices in the teaching of world language. 2. Examine the Indiana Academic Standards (IDOE, 2013), Performance Descriptors for Language Learners (ACTFL, 2012), and World Readiness Standards for Language Learning (ACTFL, 2013). 3. Identify critical questions to guide the research process: a. How well is our curriculum vertically articulated? b. How do our instructional practices relate to best practices? c. How well are our assessments aligned to best practices? d. How effectively does our current program prepare students to be college and career ready? e. To what extent are teachers able to collaborate to ensure vertical and horizontal articulation? 4. Design a plan to generate data to answer the critical questions. 5. Collect data from the following sources: a. World language achievement data, including end-of-course exam scores, course grades, IB exam scores, AP exam scores, and IU Honors Study Abroad Program acceptance b. Instructional audits were conducted voluntarily and anonymously over a 2- week period, involving 21 teachers from all four secondary schools c. World language course enrollment d. Anonymous, disaggregated Standard for Success (SFS) data from the world language department chairs e. Focus groups with counselors, teachers, and students at all four secondary schools f. Review of current curriculum and common assessments 6. Analyze data and information. 7. Formulate a report including findings and recommendations. 4 P a g e

Review of Literature As local diversity and global interconnectedness continues to increase, students who graduate high school with experience speaking and understanding multiple languages are poised to participate more fully in their community and world. Dual language proficiency not only provides opportunities for students to engage in international business and travel ventures, but it can also open doors for productive collaboration in daily life. This literature review provides a summary of research on best practices in curriculum, instruction, and assessment in world language programs as well as current information on college and career readiness as it relates to world languages. Best Practices in Curriculum and Instruction World language instruction should focus on the goals articulated in the 5 C s of the American Council on the Teaching of Foreign Languages (ACTFL): Communication, cultures, connections, comparisons, communities. These standards advocate the importance of linguistic and cultural competence, wherein students are prepared to: Communicate effectively in more than one language in order to function in a variety of situations and for multiple purposes Interact with cultural competence and understanding Connect with other disciplines and acquire information and diverse perspectives in order to use the language to function in academic and career-related situations Develop insight into the nature of language and culture in order to interact with cultural competence Communicate and interact with cultural competence in order to participate in multilingual communities at home and around the world (ACTFL, 2013). When designing world language programs to meet these lofty goals, the curriculum must be articulated to ensure learners are able to make continuous progress in their language learning. According to Met (2007), program articulation is a collaborative effort among professionals, involving the alignment of proficiency-based standards and assessments. There must be regular communication among all levels so each level of instruction builds upon the preceding one (Sandrock & Webb, 2003; Met, 2007). This communication among teachers can yield great benefits for learners. Schools have shown dramatic turnaround in student achievement when teams of teachers analyze student work against standards, identify common targets for improving instruction, and plan professional development to align lessons across grade levels (Sandrock, 2010). All in all, well-articulated programs yield an increase in student learning because gaps and overlaps can be eliminated (Met, 2007). In addition to implementing a well-articulated curriculum, the research on foreign language instruction emphasizes the importance of creating an authentic learning environment, wherein students in both exploratory and single language courses are immersed in the target language and engaged in performance-based activities that mirror real-world, culture-rich interactions. Although instruction should ensure that learners develop both a rich repertoire 5 P a g e

of formulaic expressions and a rule-based competence, vocabulary and grammar should be taught as a means to accomplish a communicative task (Sandrock & Webb, 2003; Ellis, 2008). In turn, learners focus predominantly on meaning. Successful instructed language learning requires extensive second language input and output so students learn the language with all senses rather than worksheets and drills (Ellis, 2008; Costa, 2009). This approach also builds student interest, which is the best motivator of success in learning a language (Sandrock & Webb, 2003). In addition, instruction at all levels of proficiency should incorporate higher-level thinking skills of Bloom s taxonomy and should relate to essential questions (Brim, 2013; Costa, 2009). Best Practices in Assessment Language learning is most meaningful when curriculum and instruction are geared toward a common performance-based assessment. Met (2007), indicates that the most effective way to develop clearer operational definitions of what learning is, what evidence we will accept of learning, and how we will know learning when we see it is to develop instruments such as common final performance assessments. In world language programs, assessments should be: Authentic: Reflect tasks that individuals do in the world outside the classroom, Performance-Based: Reflect how students use the language and cultural knowledge in communicative tasks, Based on the three modes of communication: o interpretive (reading, listening, viewing) o interpersonal (speaking + listening OR writing + reading) o presentational (writing, speaking, visually representing), Integrated: Blend communication with other goal area(s) of the standards, Show developmental progress of proficiency (novice, intermediate, advanced), and Blend with classroom instruction and experiences: Teaching to the test (in a positive sense) (Sandrock, 2010). When curriculum and instruction are designed with the end performance in mind, or backward-designed, students and teachers have a common, meaningful focus for their work leading up to the assessment. Students should set their own goals, self-evaluate and reflect on progress toward those goals (Costa, 2009; Brim, 2013; Sandrock, 2010; Zemelman, 2012, Duncan, 2014). Portfolios which include performance-based assessment integrate knowledge and skills rather than either one in isolation; they also encourage students to selfassess and take ownership in their learning (Brim, 2013). Portfolios not only encourage student self-assessment, these assessments also lend themselves to teachers collecting formative assessment data. Formative data provides information about learning that has already been mastered and learning that still needs to take place in order for students to achieve the performance goals. When educators engage in a collaborative process to examine student work during the formative stage of learning, they can assess the effectiveness of their instruction, more accurately understand student learning, and outline strategies that assist students in improving the quality of their work 6 P a g e

prior to the final performance assessment (Zemelman, 2012). Wiggins (2012) describes effective teacher feedback as goal referenced, tangible/transparent, actionable, userfriendly, timely, ongoing, and consistent. These seven aspects of teacher feedback lead to greater learning for students (Wiggins, 2012). College and Career Readiness The term college and career readiness as it pertains to world languages describes the degree to which students are proficient in a target language and can interact with cultural competence and understanding. This proficiency and competence is developed in direct relation to the length of time of study and level of immersion in the language (Sandrock & Webb, 2003). One definition describes readiness as the content knowledge, skills, and habits that students must possess to be successful in postsecondary education or training that leads to a sustaining career. A student who is ready for college and career can qualify for and succeed in entry-level, credit-bearing college courses without the need for remediation or developmental coursework (EPIC, 2013). Prerequisite skills and capabilities for college and career readiness related to the world language education include basic proficiency in a second language and basic awareness of other cultures (EPIC, 2013). Although this definition of college and career readiness emphasizes basic proficiency, the Indiana Academic Standards and ACTFL World-Readiness Standards place substantial emphasis on the communicative skills and cultural awareness necessary for multi-lingual involvement in our increasingly globalized society. In addition, the Partnership for 21 st Century Skills (www.p21.org) outlines a portrait of global awareness that moves beyond basic proficiency and awareness by encouraging learners to explore and understand their world on a deeper level. The Partnership defines global awareness as: Using 21 st century skills to understand and address global issues Learning from and working collaboratively with individuals representing diverse cultures, religions and lifestyles in a spirit of mutual respect and open dialogue in personal, work and community contexts Promoting the study of non-english language as a tool for understanding of other nations and cultures. Overall, world language programs are challenged to not only teach grammar and vocabulary for effective communication, but also prepare learners to understand and engage in a myriad of global perspectives in our twenty-first century world. Learners are most prepared to achieve dual-language proficiency and embark upon current local and global issues when they are provided the opportunity to participate in a world language program with wellarticulated curricula, extensive immersion in the target language and culture, and performance-based experiences with descriptive feedback from proficient teachers and realworld partners. 7 P a g e

References ACTFL American Council on the Teaching of Foreign Languages. World-Readiness Standards for Learning Languages. Retrieved October 15, 2013 from http://www.actfl.org/publications/all/world-readiness-standards-learninglanguages. Brim, C. (2013). The Learner Engaged and Empowered: How to Create an Authentic and Personal Language Experience. The Language Educator, 8(4), 28-31. Costa, G. C. (2009). Foreign Language Instruction and the Sense-Sational Habits of Mind. In A. Costa & B. Kallick (Eds.), Habits of Mind Across the Curriculum: Practical and Creative Strategies for Teachers. Alexandria, VA: ASCD. Duncan, G. (2014). Embracing the paradigm shift in learning and assessment. The Language Educator, 9(2), 18-19. Ellis, R. (2008). Principles of Instructed Second Language Acquisition. Washington, DC: Center for Applied Linguistics, CALdigest. Retrieved October 14, 2013 from http://www.cal.org/resource-center/briefs-digests. EPIC Educational Policy Improvement Center. (2013). The Definition: Understanding College and Career Readiness. Retrieved March 17, 2014 from https://www.epiconline.org/issues/college-career-readiness/definition.dot. Met, M. (2007). Achieving True Articulation in Foreign Language Learning. The Language Educator, 2(5), 54-59. Partnership for 21 st Century Skills. (2003). Learning for the 21 st Century. Washington, DC: Retrieved May 15, 2014 from http://www.p21.org/storage/documents/p21_report.pdf. Sandrock, P. and Webb, E. (2003). Learning Languages in Middle Schools. National Council of State Supervisors of Foreign Languages. Retrieved October 14, 2013 from http://www.ncssfl.org/papers/finalmiddleschool.pdf. Sandrock, P. (2010). The Keys to Assessing Language Performance: A teacher s manual for measuring student progress. Alexandria, VA: The American Council of the Teaching of Foreign Languages. Wiggins, G. (2012). Seven Keys to Effective Feedback. Educational Leadership, 70(1), 10-16. Zemelman, S., Daniels, H., & Hyde, A. A. (2012). Best practice: Bringing standards to life in America s schools. Portsmouth, NH: Heinemann. 8 P a g e

Data Collection The world language program evaluation committee collected data from the following sources to answer the critical questions. 1. World language achievement data, including end-of-course exam scores, course grades, IB exam scores, AP exam scores, and IU Honors Study Abroad Program acceptance 2. Instructional audits conducted voluntarily and anonymously over a 2-week period, involving 21 teachers from all four secondary schools 3. World language course enrollment 4. Anonymous, disaggregated Standard for Success (SFS) data from the world language department chairs 5. Focus groups with counselors, teachers, and students at all four secondary schools 6. Review of current curriculum and common assessments 9 P a g e

Findings Data from all sources were analyzed to answer the five critical questions. The following findings emerged from the data analysis: How well is our curriculum vertically articulated? 1. Students who take Level 2 of a world language in their 9 th grade year maintain a consistent level of achievement from the middle school to high school; however, multiple stakeholders describe the transition as challenging. Achievement data In 2013-2014, 96% of students who completed French I in middle school earned a B- or higher in Q1 of Level 2 French at CHS and 0% earned a D or F. In 2013-2014, 88% of students who completed German I in middle school earned a B- or higher in Q1 of Level 2 German at CHS and 0% earned a D or F. In 2013-2014, 84% of students who completed Spanish I in middle school earned a B- or higher in Q1 of Level 2 Spanish at CHS and 1% earned a D or F. Focus group data During the high school student focus group, students stated they felt more prepared for the high school if they had world language during the 4 th quarter; they felt less prepared if they had world language during 3 rd quarter of middle school. During all student, teacher, and counselor focus groups, participants stated that the required quarter on/quarter off world language coursework in middle school brings challenges. In particular, the middle school schedule causes difficulties for students to retain information, does not provide equitable learning opportunities for students who move into the school after the 6 th grade year, does not provide the opportunity for students to try more than one language or exit the language in 8 th grade if they are unsuccessful in the 6 th or 7 th grade, and causes scheduling difficulties when students need ENL, Success, or Resource classes. During the counselor focus groups at all four secondary schools, the majority of counselors expressed they receive concerns from many parents and students about the transition between Level I and Level II. Major concerns included: low grades in 9 th grade as well as differences in teaching styles and materials in Level I Spanish compared to Level II Spanish. During the high school counselor focus group, counselors expressed that some students felt overwhelmed and unprepared at the transition from Level II to Level III. 10 P a g e

Instructional Audit data The most commonly marked classroom practice on the instructional audit data is review of previously taught concepts, which occurred 6 times on average during the 2-week audit period. 2. The World Language Placement Test (WLPT) administered at the end of Level 1 in middle school Spanish, French, and German does not accurately predict student performance in level 2. In 2013-2014, 47% of students who completed French I in middle school earned a B- or higher on the WLPT, but 96% earned a B- or higher in Q1 of Level 2 French at CHS. In 2013-2014, 55% of students who completed German I in middle school earned a B- or higher on the WLPT, but 88% earned a B- or higher in Q1 of Level 2 German at CHS. In 2013-2014, 71% of students who completed Spanish I in middle school earned a B- or higher on the WLPT, but 84% earned a B- or higher in Q1 of Level 2 Spanish at CHS. 3. Many students do not continue to level 2 after taking Level 1 at the middle school. In 2013-2014, 45% of students who took Spanish 1 in middle school did not take Spanish II at CHS the following school year. In 2013-2014, 50% of students who took German I in middle school did not take German II at CHS the following school year. In 2013-2014, 47% of students who took French I in middle school did not take French II at CHS the following school year In 2013-2014, 15% of students who took Spanish I in middle school repeated Spanish I at CHS. In 2013-2014, 15% of students who took German I in middle school repeated German I at CHS. In 2013-2014, 10% of students who took French I in middle school repeated French II at CHS. How do our instructional practices relate to best practices? 1. Gradual release of responsibility is present in varying degrees throughout the world language department. In the instructional audit, 48% of teachers noted using whole group instruction and small group learning equally during the 2-week audit period. In the instructional audit, 24% of teachers noted using small group learning more often than whole group instruction during the 2-week audit period. In the instructional audit, 28% of teachers noted using whole group instruction more often than small group learning during the 2-week audit period. 2. Technology use is limited during World Language classroom instruction. 11 P a g e

Instructional audit In the instructional audit, 52% of teachers noted student use of technology 0-2 times during the 2-week audit period. In the instructional audit, 38% of teachers noted student use of technology 3-5 times during the 2-week audit period. In the instructional audit, 10% of teachers noted student use of technology 6-8 times during the 2-week audit period. SFS data In 2012-2013, 64% of world language teachers observed were noted as effectively integrating technology as a tool to engage students in academic content. In 2012-2013, 21 observations in world language classrooms noted the effective practice of effectively integrating technology as a tool to engage students in academic content. How well are our assessments aligned to best practices? 1. Checking for understanding/formative assessment is consistently implemented throughout the world language department. SFS Data In 2012-2013, RISE competency 2.4 Checking for Understanding was observed 297 times as Effective or Highly Effective in the world language classrooms. In 2012-2013, 68 observations in world language classrooms noted, Teacher uses a variety of methods to check for understanding that are successful in capturing an accurate pulse of the class s understanding. In 2012-2013, 63 observations noted, Teacher doesn t allow students to optout of checks for understanding and cycles back to these students. In 2012-2013, 56 observations in world language classrooms noted, Teacher checks for understanding at almost all key moments (when checking is necessary to inform instruction going forward). Instructional Audit In the instructional audit, 100% of teachers indicated they used checking for understanding/formative assessment and 57% of teachers noted formative assessment as occurring 5 or more times during the 2-week audit period. In the instructional audit, 90% of teachers indicated they used formative written assessments during the 2-week audit period. In the instructional audit, 76% of teachers indicated they used student selfassessment during the 2-week audit period. 2. Level 1 Assessments are inconsistent with the exception of the end-of-course exam/world Language Placement Test (WLPT). 12 P a g e

The WLPT is the only common assessment among middle and high school teachers teaching Level I French, Spanish, and German. Some Level I assessments include a performance component, but no common performance rubric exists. How effectively does our current program prepare students to be college and career ready? 1. A significant number of students at CHS have basic proficiency/college readiness upon graduation. In the 2013-2014 school year, 82% of seniors at Carmel High School completed a Level 2 or higher world language course by the end of their 12 grade year. Of the Level II or higher courses taken by the 2014 senior class at Carmel High School, 92% of semester two course grades were a C or higher. 2. A significant number of students at CHS are prepared to participate in multilingual communities at home and around the world. CHS Course completion: In the 2013-2014 school year, 24% of seniors at Carmel High School completed a Level IV or higher world language course. In the 2012-2013 school year, 28% of seniors at Carmel High School completed a Level IV or higher world language course. In the 2011-2012 school year, 29% of seniors at Carmel High School completed a Level IV or higher world language course. CHS Grades: Of the Level IV or higher courses taken by the 2014 senior class at Carmel High School, 97% of semester two course grades were C or higher. Of the Level IV or higher courses taken by the 2013 senior class at Carmel High School, 95% of semester two course grades were C or higher. Of the Level IV or higher courses taken by the 2012 senior class at Carmel High School, 97% of semester two course grades were C or higher. IU Honors study abroad program acceptance: During a three-year period from fall of 2012 to spring of 2014, 49 students who recently completed level IV of a World Language at Carmel High School, were accepted to the Indiana University Honors Study Abroad Program in Foreign Languages for high school students. o In 2014, 15 students (7 French, 5 Spanish, 2 German, 1 Chinese) were accepted. o In 2013, 14 students (4 French, 9 Spanish, 1 German) were accepted. o In 2012, 20 students (5 French, 15 Spanish) were accepted. 13 P a g e

3. The majority of CHS students who take an Advanced Placement (AP) or International Baccalaureate (IB) test earn a score worthy of College/University credit. Advanced Placement During a three-year period from the fall of 2010 to the spring of 2013, 72% of the students enrolled in an AP world language class took the AP test. During a three-year period from the fall of 2010 to the spring of 2013, the number of students who took an AP world language test increased from 55 in the 2010-2011 school year to 104 in the 2012-2013. During a three-year period from the fall of 2010 to the spring of 2013, the average score on an AP world language exam was 3.68. The score needed to obtain college placement is a 3. International Baccalaureate During a three-year period from the fall of 2010 to the spring of 2013, 50% of the students enrolled in an IB world language class took the IB test. During a three-year period from the fall of 2010 to the spring of 2013, the number of students who took an IB world language test decreased from decreased from 11 to 6. During a three-year period from the fall of 2010 to the spring of 2013, the average score on an IB world language exam was 4.6. A score of 4 is considered passing for an IB exam. To what extent are teachers able to collaborate to ensure vertical and horizontal articulation? 1. Common teacher collaboration time during the school day for analyzing student data to inform instruction is limited. During teacher focus groups, teachers report there is no common prep time during the school day to collaborate so they find collaboration time before/after school, during lunch, via email or over summer. According to 2012-2013 SFS data, 38% percent of world language teachers were noted to have collaborated with peers. 2. Teacher autonomy during collaboration time at the high school is limited. CHS teachers have 45 minutes of collaboration time during their Professional Learning Community (PLC) time on late-start Wednesdays. Teachers at CHS state that the majority of PLC time during the 2012-2013 and 2013-2014 school years is focused on a selection of four school-wide professional development topics, with little room for deviation from the topic of focus. 14 P a g e

CHS world language teachers state a desire to make part of PLC time more meaningful by allowing for teacher-driven collaboration. 3. Teachers perceive a lack of vertical articulation between the middle school and high school as well as a lack of horizontal articulation within languages and levels. Middle school teachers note that communication and data from the high school teachers is affirming and helpful in diagnosing areas of student weakness and areas of student mastery. High school teachers, specifically Level 2 of Spanish, French, German, indicate that feedback and communication with middle school teachers is helpful in ensuring seamless transition of students between Level 1 in middle school and Level 2 at the high school. Middle school and high school teachers report a lack of collaboration between the middle school and high school levels. Singleton teachers report a feeling of isolation when they are the only teacher of that language in a building/level. High school teachers state that discussion of vertical articulation does not happen under the current structure of PLC time. 15 P a g e

Recommendations The World Language Program Evaluation committee proposes the following recommendations to facilitate a model of continuous improvement in world language curriculum, instruction, and assessment. These recommendations are designed not only to support the current level of student achievement but to provide a vehicle for attaining the highest level of achievement for all students. 1. Establish a vertically aligned scope and sequence, which will prepare learners to be college and career ready through engagement in best practices. These best practices identified in the literature include, but are not limited to: a. Authentic performance-based assessments, which foster cultural competence and align to standards b. Formative assessment and student self-assessment c. Immersion in the target language d. Higher-level thinking at all levels of proficiency e. Meaningful technology use 2. Level 1 of all world languages should be consistent among secondary schools and provide continuous immersion in the language. 3. All middle school students should have an option to take Level 1 of a World Language or participate in an exploratory experience. 4. Increase job-embedded teacher collaboration time: a. Collaboration time is needed to analyze student data to inform instruction and share best practices both vertically and horizontally. b. Collaborative professional development experiences are needed to streamline the transition between levels. c. Collaboration time should have some flexibility to address needs as they arise. 16 P a g e