Collaboration to Achieve Whole-School SEL Across a Large, Urban District Jen Dirga Open Circle Michelle Porche Wellesley Centers for Women BU School of Education Implementation funded by Partners HealthCare Evaluation funded by NoVo Foundation
What is Open Circle? Open Circle, established in 1987, is a leading provider of evidence-based professional development and curricula for social and emotional learning (SEL) in Kindergarten through Grade 5. 2
Emotion drives attention and attention drives learning. Robert Sylwester (1995) Professor Emeritus of Education University of Oregon 3
Open Circle Goals Skill Development Help students develop skills for recognizing and managing emotions, empathy, relationships and problem solving Learning Environment Help students feel safe, cared for, and engaged in learning at school 4
CASEL Five Social and Emotional Learning involves the development of skills in 5 areas Manage emotions and behaviors Recognize emotions, Show understanding and empathy for others Make constructive choices about personal and social behavior Form positive relationships, work in teams, effectively manage conflict 5
Whole-School Approach 6
Curricula Open Circle Curriculum Grade K Open Circle Open Circle Curriculum Grade 1 Open Circle Open Circle Curriculum Grade 2 Open Circle Teachers implement gradedifferentiated SEL curricula during twice-weekly, 15- minute classroom meetings Open Circle Curriculum Grade 3 Open Circle Open Circle Curriculum Grade 4 Open Circle Open Circle Curriculum Grade 5 Open Circle The entire school community models and uses SEL vocabulary, concepts and strategies throughout the school day and beyond 7
Key Open Circle Skills to help decrease incidents of bullying Self Awareness/ Self Management Social Awareness and Positive Relationships Problem-Solving Grade 3-5 Open Circle lessons: What is Bullying Behavior What is a Bystander Being an Ally 8
Professional Development Grade-level teachers receive 4 days of training, 2+ coaching visits Administrators, counselors, specialist subject teachers, support staff, and family and community outreach coordinators receive 1 day of training Open Circle s PD is designed to be interactive and experiential with multiple cycles of learning, practice, and reflection 1+ peer coaches and SEL leadership team of 8 cross-departmental staff members receive additional PD 9
Background SEL programs shown to improve social skills, behavior, and academic achievement Most studies conducted at the classroom level Few examine whole-school and district-level reform through SEL
Aims Process outcomes and program impact for 2-year, grade K-5 SEL rollout across a large urban district: 23 schools (one-third of elementary-level district schools) 400 classrooms (universal every K-5 classroom) Over 7,000 students (universal every K-5 student)
District Setting Setting 8.5% suspension rate 7.4% retention in grade 8.9% dropout Student population 78% low-income 30% limited English proficiency 20% special education status 36% Black/African American; 40% Hispanic; 13% White; 9% Asian; 2% other/multi-racial
Factors to Consider What are the critical factors to successful district implementation? How is implementation related to school climate, teacher practice, and students social and emotional skill development and behavior? 13
Evaluation Sample 23 Participating Schools 34 schools expressed interest District selected schools, prioritizing schools serving lowest-income populations 19 schools joined in Year 1 (3 of which dropped) and 7 schools joined in Year 2 793 Staff Members Trained 413 teachers 332 counselors, special subject teachers, support staff 48 administrators 5 family/community outreach coordinators
Evaluation Measures Staff surveys School/Classroom Climate; Students Prosocial and Negative Behaviors Implementation of Open Circle Curriculum Learning from training/coaching Program satisfaction Observation tools Classroom Assessment Scoring System (CLASS; La Paro, Pianta & Stuhlman, 2004) Open Circle Fidelity Checklist Student survey Social Skills Improvement System (SSIS; Gresham & Elliott, 2008) District data Teacher Report Card Rating of Social Development Measures Sociodemeographic data
Evaluation Procedures Implementation observations 344 teachers Pre-Post online surveys 250 teachers, 177 specialist/para, 27 administrators, 4 family coordinators Pre-Post classroom observations 12 Grade 3; 13 Grade 5 Pre-Post student surveys 247 Grade 3; 239 Grade 5 Qualitative interviews with 17 staff
Training Participation 100% Percent of 23 Participating Schools 80% 60% 40% 20% 0% Teachers Specialists & Paraprofessionals One or More Administrators Behavioral Health Staff All/Nearly All Trained Over 50% Trained Less than Half Trained 17
Teacher Training and Coaching Attendance 100% Percent of Teachers (n=413) Completing Training/Coaching 80% 60% 40% 20% 0% Completed All Half or More Few or None 4 Teacher Training Sessions 2 Coaching Sessions for Teachers 18
Teacher Implementation (n=123) 92% reported facilitating Open Circle Meetings for 15+ minutes 81% reported facilitating Open Circle Meetings one or more times per week 83% reported completing at least three-quarters of all curriculum lessons 19
Teacher Fidelity Coaches Fidelity Checklist for Teachers in first year of Implementation (n=316) Not Skilled 7% Somewhat Skilled 31% Skilled 63% Note: Total adds up to more than 100% because of rounding. 20
Teacher SEL Integration (n=121) I integrated SEL throughout the day I modeled SEL skills throughout the day I posted SEL visuals in my classroom I used SEL community-building activities I read or assigned SEL-themed books I sent SEL content to families 0% 20% 40% 60% 80% 100% Very Frequently Frequently Occasionally 21
Whole-School Implementation Special Subject Teachers and Support Staff Infusion (n=76) I used SEL vocabulary in my interactions with students I encouraged my students to practice SEL skills I used SEL community-building activities I infused SEL in school-wide activities (e.g. assemblies) I posted SEL visuals in my work area I read SEL-themed books with students I collaborated with grade-level teachers on an SEL-related project 0% 20% 40% 60% 80% 100% Very Frequently Frequently Occasionally 22
Program Satisfaction I would recommend Open Circle to a colleague. Agree Strongly Agree Teachers (n=124) 44% 44% Specialists/Paraprofessionals (n=76) 43% 32% Administrators (n=24) 46% 50% How satisfied are you with Open Circle in your school? Very Satisfied Extremely Satisfied Teachers (n=124) 45% 21% Specialists/Paraprofessionals (n=76) 36% 17% Administrators (n=24) 75% 17% 23
Impacts Across Staff (Teachers/Specialists/Administrators: n=228) Open Circle Percent Agreement Provided a common vocabulary 89.0% Increased my students demonstration of prosocial behaviors 93.9% Reduced my students demonstration of negative behaviors 89.9% Helped to create a safer environment in my school 92.1% Helped to create a more caring environment in my classroom/school 93.4% Helped to create a more highly engaged environment in my classroom/school Enabled me to spend less time managing disruptive behaviors and more time on academics 88.6% 86.8% Enabled me to make fewer office referrals for student behavior problems 51.8% 24
Student Behaviors & Climate PreSurvey Mean PostSurvey Mean Paired t-test ProSocial 3.37 3.61 t = -6.00, p <.0001 Behaviors 1 Negative 2.13 2.24 t = -2.24, p <.0267 Behaviors 1 Classroom Climate 2 4.75 4.68 t =1.56, p <.1211 1 Teachers, Specialists, Paraprofessionals (n=138) 2 All staff (n=190) 25
Perceived Impact of Training & Coaching Positive Training/Coaching Impact Correlated to Improved Outcomes Training Post School/Classroom Climate.34*** (n=204) Post Prosocial Behaviors.33*** (n=204) Post Negative Behaviors -.14* (n=204) Coaching.30*** (n=127).20* (n=127) -.19* (n=127) * p <.05 *** p <.001
CLASS Observation Results Spring 2014 (n=26 teachers/classrooms) CLASS Emotional Support (M=2.02) CLASS Classroom Organization (M=5.39) CLASS Instructional Support (M=3.88) Meeting Fidelity.78***.76***.63*** Carryover Fidelity.70***.54***.67*** Student Engagement.62***.68***.50*** CLASS observation scores highly correlated with observed fidelity scores and student engagement. (p<.001) CLASS scoring:1-2= low range, 3-5=middle range; 6-7=high range. 27
Student Surveys SSIS (n=342) Cooperation 3.20 (0.53) Empathy 3.18 (0.61) Self Control 2.68 (0.69) Bullying 1.36 (0.51) Pre Post Paired t-test 3.19 (0.53) 3.12 (0.60) 2.68 (0.70) 1.28 (0.44) t=-0.24 t=-1.39 t= 0.03 t=-2.36* 28
Preliminary Multi-level Modeling Results Analytic sample: (N=19,487) Students nested in classrooms, nested in schools The estimated variance components indicated that 11% of the variation in SEL scores was between schools, 23% was between classes, and 65% was between students Suggestion of cumulative effect two years of Open Circle and comparison non-open Circle higher on SEL report card compared to one year of Open Circle (controlling for sociodemographics) Socio-demographics explained more of the variation in student scores than participation in any SEL program
Qualitative Results Teacher feedback focused on positive experiences with Open Circle (n=28), ways Open Circle helps students (18), the importance of making Open Circle a whole-school initiative (n=23), and requests to extend the Open Circle Curriculum into middle school (n=4). The majority of comments expressed excitement about Open Circle. Lessons are teacher friendly, well thought out and easy to implement. Open Circle gives us some common language, a basic routine, and a set of tools to work on social and emotional learning. Open Circle can go a long way toward creating a safe and supportive school culture. the lessons have been very useful to me in helping students develop acceptance and respect for each other, resulting in a supportive community environment in the class. I think that Open Circle has made a lot of difference in how my class settled down without Open Circle, I would not have been able to teach. Barriers expressed included: need for greater administrative support, variation in level of buy-in by teachers, in follow through with training, and implementation.
Success Factors Cultivate many SEL champions at district-level and throughout schools Leverage relationships and funding from external community groups Follow up and follow through Overloaded schools often need extra support for implementation Expect a range of needs Schools vary widely in their levels of commitment, responsiveness and capacity for SEL work 31
Next Steps Preliminary results indicate that this whole-school SEL program is highly scalable Key success factors have been initial and ongoing principal and district administrator support and teacher buy-in Latent class analysis with full district data to determine if Open Circle is more effective with particular groups of students Longitudinal analysis with a third year of district data Analysis of combined district and observation data
Conclusion Achieved extensive, wholeschool SEL professional development and implementation across a significant share of schools in large urban district in two years 33
Thank You! www.open-circle.org info@open-circle.org (781) 283-3277 35