Brigham Young University BYU ScholarsArchive All Faculty Publications 2009-07-01 The Effects of School-Wide Positive Behavior Support on School Climate: A Middle School Logitudinal Study K. Richard Young Richard_Young@byu.edu Ryan H. Shatzer See next page for additional authors Follow this and additional works at: https://scholarsarchive.byu.edu/facpub Part of the Teacher Education and Professional Development Commons Original Publication Citation OSEP Conference, Washington, D.C. ( July 29) BYU ScholarsArchive Citation Young, K. Richard; Shatzer, Ryan H.; Young, Ellie L.; Caldarella, Paul; and West, Richard E., "The Effects of School-Wide Positive Behavior Support on School Climate: A Middle School Logitudinal Study" (2009). All Faculty Publications. 1252. https://scholarsarchive.byu.edu/facpub/1252 This Presentation is brought to you for free and open access by BYU ScholarsArchive. It has been accepted for inclusion in All Faculty Publications by an authorized administrator of BYU ScholarsArchive. For more information, please contact scholarsarchive@byu.edu.
Authors K. Richard Young, Ryan H. Shatzer, Ellie L. Young, Paul Caldarella, and Richard E. West This presentation is available at BYU ScholarsArchive: https://scholarsarchive.byu.edu/facpub/1252
The Effects of School-Wide Positive Behavior Support on School Climate: A Middle School Longitudinal Study K. Richard Young, BYU Ryan H. Shatzer, BYU Richard P. West, USU Ellie L. Young, BYU Paul Caldarella, BYU
Funding U.S. Department of Education Field Initiated Research Projects OSEP Grant # H324CO30124B Written by Brigham Young University in collaboration with participating school district 2003 2008
Contents Introduction Methods Results and Discussion i Questions
Questions What strategies and resources are needed to implement a successful SWPBS program in middle schools? How can you measure the effects of SWPBS in middle schools?
INTRODUCTION
Positive Behavior Support (PBS) Build Positive Relationships ~5% Tertiary Prevention: Students with High-Risk Behavior Data Based Decision Making PBS Establish Clear Expectations ~15% Secondary Prevention: Students with At-Risk Behavior Rif Reinforce Appropriate Behavior Teach Appropriate Skills Primary Prevention: School-Wide ~80% of Students
School Climate A set of internal characteristics that distinguishes one school from another and influences the behavior of its members Encompasses the dimensions of (1) Values (2) Educational practices (3) Personal interactions (Hoy and Hannum, 1997) (National Research Council, 2004)
School Climate Our definition includes perceptions of... Physical Aspects of the School Programs and Resources Instructional Management Relationships between Staff, Students, and the Community Parent and Teacher Support
Outcomes of Healthy School Climate Improved Student Achievement Standardized test scores Reading levels GPA Academic and school adjustment Decreased Student Misbehavior Student drug use Aggression and school violence Anti-social behavior Absences and suspensions Student delinquency (Brand et al., 2008; Esposito, 1999; Hoy & Hannum, 1997) (Astor et al., 2002; Aveyard et al., 2004; Gottfredson et al., 2005; Sprott, 2004)
METHODS
Build Positive Rlti Relationships Data Based Decision Making PBS Establish Clear Expectations Reinforce Appropriate Behavior Teach Appropriate Skills
Tertiary-Level Intervention One-on-one one interventions PBS Framework Secondary-Level Intervention (Achievement Plus Class) Social Skills Instruction: Prevention Plus and peer praise notes Organizational Skills: Planners and goal setting Learning Strategies: Study skills and accessing supports Emotional Resiliency: Strong Kids and journaling Behavioral Self-Management: Conflict resolution techniques Experiential Learning: Ropes course and service-learning School-Wide Intervention (SWPBS) Faculty training by PBS support staff Proactive screening for students at-risk for ED School-wide assemblies, activities, and celebrations Posting school rules and common classroom rules School-wide instruction of social skills Teacher praise notes to students Administrative Intervention Data-based decision making: Indicators of School Quality (ISQ) This PBS framework was implemented over 4 years at the treatment middle school (2004-2007)
Sample Treatment School Control School n % n % Teachers 50 56 Students 1063 1331 Gender Ethnicity Male 548 51.6% 693 52.1% Female 515 48.4% 638 47.9% Caucasian 934 87.9% 1210 90.9% Other 129 12.1% 1% 121 91% 9.1% Special Education 141 13.3% 148 11.1% Reduced Lunch 402 37.8% 366 27.5% * The numbers reported are school averages over the four years of the intervention
School Records Measures Grade Point Average (GPA) Absences Office Discipline Referrals (ODRs) Indicators of School Quality (ISQ) Teacher ratings of school climate (7 categories) Student ratings of behavior support (5 categories) Supplemental Questionnaire Teacher ratings of school climate (3 factors)
Indicators of School Quality (ISQ) Conducted over 9 Years 1,300 Schools 33 Utah School Districts i t 14 States + 2 International 70,000 Teachers 500,000 Families 700,000 Students
Indicators of School Quality (ISQ) The Surveys are short include all parents, students, faculty, and other school staff employ age and culturally ll appropriate language evaluates several environment domains The Reports are easy to read provide data to compare each school to a normative group of schools provide a progress summary summarize the statistical and practical adequacy of the response to the survey
Indicators of School Quality Status Report Parent Support Parent Support Parents volunteer often at the school Parents know what happens at school Enough parents participate at parent/teacher conferences Parents support extracurricular activities Teacher Excellence Teachers are knowledgeable about the subjects they teach Teachers care about students as individuals Teachers maintain discipline in their classrooms Teachers are well organized Teachers enjoy teaching Student t Commitment t Students are well behaved Enough students participate in extracurricular activities Students are excited about learning Students have pride in their school Administration Administration is accessible to parents, students, and staff Administration is supportive of teachers Administration is well organized Administration applies appropriate discipline at the school Administration has high expectations for all students Instructional Quality This school prepares students for future employment This school provides a quality education Instruction at this school is innovative Instruction at this school challenges students Resource Accessibility Staff has access to enough ongoing training Counselors are accessible to students Students have adequate computer access The school has enough quality textbooks Students have enough extracurricular opportunities Safety Students and staff feel safe at school Students feel safe going to and from school The school is clean and in good repair The school grounds and hallways are well supervised Parent Teacher Student Staff
ISQ Predicts Academic Achievement 3 rd Grade 5 th Grade 8 th Grade 11 th Grade Parent Teacher Student Parent Teacher Student Parent Teacher Student Parent Teacher Student Parent Support Teacher Excellence Student Commitment School Leadership Instructional Quality Resource Management School Safety
Elements of Behavior Support Elements Positive Relationships Clear Expectations Social Building Skills Academic Rewards & Recognition Items Is there an adult at school whom you trust to ask for help? Are you frequently confused about what is expected of you at school? Do you make friends easily? Do you do well on tests and assignments? Are you recognized enough for your efforts to succeed? Build Positive Relationships Data Based Decision Making PBS Establish Clear Expectations Reinforce Appropriate Behavior Teach Appropriate Skills
RESULTS & DISCUSSION
Data Analysis Analysis of Variance (ANOVA) Linear Trend Were there significant improvements over the four years in the treatment middle school? Interaction Effects Were there significant improvements in the treatment school, relative to the control school?
School Record Data 2004-20072007
School Records Summary Academic Achievement GPA Absences Tardies* Unexcused Absences* ODRs Disorderly Conduct ODRs* ODRs *Significant interactions
GPA 3.40 3.35 3.30 3.25 3.20 3.15 Treatment Control 3.10 305 3.05 3.00 2004 2005 2006 2007 PBS School: Significant increase in GPA, but no interaction effect Linear trend: F (1, 4457) = 11.81, p <.001, d =.14
Tardies 9.0 8.0 7.0 60 6.0 5.0 4.0 3.0 Treatment Control 2.0 1.0 0.0 2004 2005 2006 2007 PBS School: Significant interaction: F (3, 10758) = 77.51, p <.001 Decrease in tardies: Linear trend: F (1, 4822) = 70.08, p <.001, d =.32 Periods per student per year
Unexcused Absences 18 16 14 12 10 8 Treatment Control 6 4 2 2004 2005 2006 2007 PBS School: Significant interaction: F (3, 10758) = 12.04, p <.001 Decrease in absences: Linear trend: F (1, 4822) = 5.85, p <.05, d =.11 Periods per student per year
Disorderly Conduct ODRs 1.4 1.2 1.0 0.8 0.6 Treatment Control 0.4 02 0.2 0.0 2004 2005 2006 2007 PBS School: Significant interaction: F (3, 10758) = 14.01, p <.001 Decrease in ODRs: Linear trend: F (1, 4822) = 11.27, p <.001, d =.14 Per student per year
Practical Significance It was estimated that during the last year of PBS the treatment school saved: 167 student and administrative hours due to the reduced number of ODRs 4,498 days in the classroom due to the reduced number of absences 213 hours of class time due to reduced tardiness *Based on 15 minutes spent per ODR and 5 minutes late per tardy
Indicators of School Quality (ISQ) Teacher Version: 2004-20072007
ISQ Results Summary Teacher Version Teacher Excellence* Instructional Quality* School Leadership* Student Commitment Parent Support Resource Management * School Safety *Significant interactions
Teacher Excellence 4.7 4.6 4.5 Treatment 44 4.4 Control 4.3 4.2 2004 2005 2006 2007 PBS School: Significant Interaction: F (3, 306) = 2.75, p <.05 Significant Linear Trend: F (1, 145) = 4.71, p <.05, d =.48
Instructional ti Quality 4.4 4.2 4.0 Treatment Control 3.8 3.6 2004 2005 2006 2007 PBS School: Significant Interaction: F (3, 306) = 3.13, p <.05 Significant Linear Trend: F (1, 145) = 8.81, p <.01, d =.64
Sh School ll Leadership 4.8 4.6 4.4 Treatment 42 4.2 Control 4.0 3.8 2004 2005 2006 2007 PBS School: Significant Interaction: F (3, 306) = 6.17, p <.001 Significant Linear Trend: F (1, 145) = 19.69, p <.001, d = 1.15
Student t Commitment t 4.2 4.0 38 3.8 3.6 3.4 Treatment Control 3.2 3.0 2004 2005 2006 2007 PBS School: Non-significant Interaction: F (3, 307) = 1.29, p =.28 Significant Linear Trend: F (1, 145) = 11.56, p <.001, d =.74
Parent Support 4.0 3.8 3.6 Treatment 34 3.4 Control 3.2 3.0 2004 2005 2006 2007 PBS School: Non-significant Interaction: F (3, 307) = 0.37, p =.77 Significant Linear Trend: F (1, 145) = 4.07, p <.05, d =.50
Resource Management 4.2 4.0 3.8 Treatment 36 3.6 Control 3.4 3.2 2004 2005 2006 2007 PBS School: Significant Interaction: F (3, 306) = 3.09, p < 0.05 Significant Linear Trend: F (1, 144) = 11.67, p <.001, d =.77
Sh School lsft Safety 4.8 4.6 44 4.4 4.2 4.0 Treatment Control 3.8 3.6 2004 2005 2006 2007 PBS School: Non-significant Interaction: F (3, 306) = 2.11, p =.10 Non-significant Linear Trend: F (1, 144) = 0.44, p =.51
Behavior Support Items Student Survey: 2005-20072007
Positive Relationships 100% 95% 90% 85% 80% 75% Treatment Control 70% 65% 60% 2005 2006 2007 Positive Relationships: Is there an adult at school whom you trust to ask for help? Percentages are those students that marked Yes
Clear Expectations ti 100% 95% 90% 85% 80% 75% Treatment Control 70% 65% 60% 2005 2006 2007 Clear Expectations: Are you frequently confused about what is expected of you at school? Percentages are those students that marked No
Social Skills 80% 75% 70% 65% 60% 55% Treatment Control 50% 45% 40% 2006 2007 Building Social Skills: Do you make friends easily? Percentages are those students that marked Yes
Academic Skills 100% 95% 90% 85% 80% 75% Treatment Control 70% 65% 60% 2005 2006 2007 Building Academic Skills: Do you do well on tests and assignments? Percentages are those students that marked Yes
Rewards and Recognition 80% 75% 70% 65% 60% 55% Treatment Control 50% 45% 40% 2005 2006 2007 Rewards and Recognition: Are you recognized enough for your efforts to succeed? Percentages are those students that marked Yes
Supplemental Questionnaire Teacher Survey: 2004-20072007
Factor Analysis Item Factor 1 Factor 2 Factor 3 Students show respect for each other.82 Students use appropriate social skills.81 Students share responsibility for making school safe.75 Students are motivated to use appropriate p social skills.66 Student Pro-social Behavior Students learn important social skills.64 Everyone at school treats others with positive regard.62 Behavior problems dealt with appropriately.57 School develops links to community.83 School involves families.79 Students have opportunities to express feelings.51 Adults invite discussions of safety.51 School uses positive means for student cooperation.51 School Communication/ Collaboration Student with special needs receive special help.80 School identifies needs of high-risk students.76 School strives for academic success.65 Only factor loadings over.50 are shown All eigenvalues > 1; Explains 59.38% of the variance Educational Assistance α =.90.82.76
Supplemental l Questionnaire i Summary Teacher Survey Student Pro-social Behavior* School Communication/Collaboration* Educational Assistance* *Significant interactions
Student t Pro-Social Behavior 4.2 4.0 3.8 Treatment 36 3.6 Control 3.4 3.2 2004 2005 2006 2007 PBS School: Significant Interaction: F (3, 399) = 13.84, p <.001 Significant Linear Trend: F (1, 185) = 38.63, p <.001, d = 1.20
School Communication/Collaboration 4.4 4.2 4.0 3.8 Treatment Control 3.6 3.4 2004 2005 2006 2007 PBS School: Significant Interaction: F (3, 400) = 5.55, p <.001 Significant Linear Trend: F (1, 185) = 15.14, p <.001, d =.78
Educational Assistance 4.8 4.6 4.4 Treatment 42 4.2 Control 4.0 3.8 2004 2005 2006 2007 PBS School: Significant Interaction: F (3, 398) = 5.20, p <.05 Significant Linear Trend: F (1, 184) = 10.64, p <.001, d =.65
Overview School-wide PBS showed moderate improvements in school climate School records (GPA, attendance, ODRs) Indicators of School Quality (ISQ) Supplemental questionnaire Gradual, continuous improvement Limitations it ti Measures of treatment fidelity
QUESTIONS
Questions What strategies and resources are needed to implement a successful SWPBS program in middle schools? How can you measure the effects of SWPBS in middle schools?
References Astor, R. A., Benbenishty, R., Zeira, A., & Vinokur, A. (2002). School climate, observed risky behaviors, and victimization as predictors of high school students' fear and judgments of school violence as a problem. Health Education & Behavior, 29(6), 716-736. 736. Aveyard, P., Markham, W. A., Lancashire, E., Bullock, A., Macarthur, C., Cheng, K. K., et al. (2004). The influence of school culture on smoking among pupils. Social Science & Medicine, 58(9), 1767-1780. 1780. Brand, S., Felner, R. D., Seitsinger, A., Burns, A., & Bolton, N. (2008). A large scale study of the assessment of the social environment of middle and secondary schools. Journal of School Psychology, 46(5), 507-535. 535. Esposito, C. (1999). Learning in urban blight: School climate and its effect on the school performance of urban, minority, low-income children. School Psychology Review, 28(3), ( ) 365-377. 377. Gottfredson, G. D., Gottfredson, D. C., Payne, A. A., & Gottfredson, N. C. (2005). School climate predictors of school disorder. Journal of Research in Crime and Delinquency, 42(4), 412-444. 444. Horner, R. H., & Sugai, G. (2004). School-wide Positive Behavior Support: Building Sustainable Support. www.pbis.org. Hoy, W. K., & Hannum, J. W. (1997). Middle school climate: An empirical assessment of organizational health and student achievement. Educational Administration Quarterly, 33(3), 290-311 311. Merrill, K (2003). Strong Teens: A curriculum for promoting social and emotional resiliency in students from grades 9-12. The Oregon Resiliency Project. University of Oregon. National Research Council (2004). Engaging Schools: Fostering High School Students' Motivation to Learn. Division of Behavioral and Social Sciences and Education. Washington, DC: The National Academies Press. Sprott, J. B. (2004). The development of early delinquency: Can classroom and school climates make a difference? Canadian Journal of Criminology and Criminal Justice, 46(5), 553-572. 572. Taylor, M., West, R., & Smith, T. (2006). The Indicators of School Quality (ISQ) Survey Manual.. Logan, UT: Center for the School of the Future. Young, K., Marchant, M., & Wilder, L. K. (2003). School-based interventions for students with emotional/behavioral disorders. In P. Allen-Mears and M. W. Fraser (Eds.) Interventions with children and adolescents: An interdisciplinary perspective. (pp. 175-204). Boston: Allyn & Bacon. Young, K., & Shatzer, R. (2008, July). School-wide Positive Behavior Support: Effects on Middle School Teachers Perception of School Quality. Poster presented at the Office of Special Education Programs (OSEP) Conference, Washington D.C.
http://education.byu.edu/pbsi 301 MCKB Brigham Young University Provo, UT 84602 (801) 422-3694 byu.pbsi@gmail.com richard_young@byu.edu ryan_shatzer@byu.edu rich.west@usu.edu ellie_young@byu.edu paul_caldarella@byu.edu