1. Factors affecting sustainability of SWPBS 2. Current research on sustainability of SWPBS

Transcription

1 Sustaining School-wide PBS: The Principal s Perspective Kent McIntosh, University of British Columbia Andrew Krugly, Dewey Elementary School (IL) PBIS Forum, October 9, 2009 Handouts: Agenda Kent McIntosh 1. Factors affecting sustainability of SWPBS 2. Current research on sustainability of SWPBS Andrew Krugly 1. Sustainability at Dewey Elementary 2. Tips for Principals Questions and Answers Handouts: Do you have a deep understanding of the principles of sustainability? Common perception that sustainability is a ethereal, theoretical concept (Vaughn et al, 2000) We all have experiences with it The same principles of individual behaviour still apply to systems Antecedent Behaviour Consequence Ongoing Challenge Select Practices & Implement with Fidelity Student Outcomes

2 Definition iti Sustainability Durable implementation of a practice at a level of fidelity that continues to produce valued outcomes (Han & Weiss, 2005) A SWPBS Sustainability Study (Doolittle, 2006) Sample: 285 schools with SET scores Differences between schools that implemented and those that did not Differences between schools that sustained and those that did not Implementers vs. Non-implementers Maintainers i vs. Non-maintainers i SET Subscale SET Overall SET Subscale SET Overall Expectations Defined Expectations Taught Reward System Responding to Violations Monitoring and Decision-Making Management (team and admin) Met SET ( 80%) Expectations Defined Expectations Taught Reward System Responding to Violations Monitoring and Decision-Making Management (team and admin) Maintained SET for 5 years District-Level Support (Doolittle, 2006) Never met SET (<80%) District-Level Support (Doolittle, 2006) Met and lost SET in 5 years

3 PBS will sustain IF it remains: A PRIORITY for faculty, staff, and administrators EFFECTIVE for ALL students EFFICIENT for school personnel ADAPTIVE to change Antecedent Behaviour Consequence Select Identifying Practices Ongoing Student & Implement Modifying Challenge Outcomes with Practices Fidelity Antecedent Consequence Behaviour Priority Effectiveness Valued Outcomes Data- Based Prob. Solving Identifying & Modifying Practices Valued Outcomes Data- Based Prob. Solving Continuous Regeneration Identifying & Modifying Practices Practice Implementation Practice Implementation Efficiency Behaviour

4 Priority Valued Outcomes Data- Based Prob. Solving Continuous Regeneration Identifying & Modifying Practices Effectiveness Priority Valued Outcomes PRIORITY Importance in comparison to other practices Incorporation into core system components Connection to other initiatives Practice Implementation Efficiency Priority Valued Outcomes ENHANCING PRIORITY Maximize visibility Present data to people p with resources Describe effects of abandoning support for the practice Get into written policy Braid project with other initiatives Show how practice can lead to other outcomes of new initiatives iti Summary Statement Setting Events Presence of PBS Coach Triggering Antecedents Prevalence of Problem Behaviour Desired Alternative Behaviour Implement SWPBS Acceptable Alternative Typical Consequence Maintaining Consequence Reductions In Problem Behaviour

5 Priority Valued Outcomes Data- Based Prob. Solving Continuous Regeneration Identifying & Modifying Practices Effectiveness EFFECTIVENESS Extent to which the practice results in desired outcomes Choice of practices should be based on proven effectiveness Effects must be attributed to the practice Identifying & Modifying Practices Effectiveness Practice Implementation Efficiency ENHANCING EFFECTIVENESS Select practices that are likely to produce the Identifying desired outcomes & Modifying (i.e., evidence-based Practices practices) Ensure fidelity of implementation Share data that show how adoption is related to effects Effectiveness Priority Valued Outcomes Data- Based Prob. Solving Continuous Regeneration Practice Implementation Identifying & Modifying Practices Effectiveness Efficiency

6 EFFICIENCY Relationship between continued effort and continued effectiveness Weighed against other potential practices ENHANCING EFFICIENCY A durable practice should become more efficient over time Easier on implementers Repetition builds fluency Easier to modify materials than create them Easier on resources Fewer visits from external consultants Fewer release days Practice Implementation Practice Implementation Efficiency Efficiency Priority Valued Outcomes Data- Based Prob. Solving Continuous Regeneration Practice Implementation Identifying & Modifying Practices Effectiveness CONTINUOUS REGENERATION Iterative monitoring of fidelity, outcomes, and context Use of data Adaptation over time while keeping critical features intact Ongoing investment in building local capacity Data- Based Prob. Solving Continuous Regeneration Efficiency

7 ENHANCING CONTINUOUS REGENERATION Data- DATA-BASED Based Prob. DECISION MAKING Solving Adjust practices for a Continuous changing environment Effectiveness Efficiency Relevance Expand to new areas Connect with a community of practice Cultivate local expertise Regeneration Cautions for Continuous Regeneration When you keep it fresh avoid lethal mutations Consider the critical features of what makes SWPBS effective Reward systems recognition of their success Not a scrap of paper without recognition Not insincere praise Not the same for everyone! Priority Valued Outcomes Data- Based Prob. Solving Continuous Regeneration Identifying & Modifying Practices Effectiveness A Measure to Assess SWPBS Sustainability Practice Implementation School-wide PBS Universal Behavior Sustainability Inventory School Teams (SUBSIST) Efficiency

8 Overview The SUBSIST is an online survey assessing the variables that enhance or prevent sustainability of SWPBS Developed through an expert panel and a pilot study Goal: identify the most important steps for sustainability McIntosh, K., MacKay, L. D., Hume, A., Doolittle, J. D., Vincent, C. G., Horner, R. H., et al. (2009). Development and validation of a measure to assess factors related to sustainability of school-wide positive behavior support. Manuscript submitted for publication. Survey Description 8 broad factors associated with sustainability of SWPBS: (1) Priority (5) Efficiency (2) Building Leadership (6) Use of Data (3) External Leadership (7) Capacity Building (4) Effectiveness (8) Potential Barriers Pilot Study Results (MacKay & McIntosh, 2009) Strong internal consistency, test-retest and inter-rater reliability Moderate correlation with the School-wide Evaluation Tool Analysis of perception data Most important items for initial implementation External Coaches 1. A school administrator regularly attends and participates in SWPBS team meetings 2. The school administrators ensure that the SWPBS team has regularly scheduled time to meet 3. There are adequate district resources (funding and time) allocated for SWPBS Team Leaders 1 The school administrators (building principal or vice principal) actively 1. The school administrators (building principal or vice-principal) actively supports SWPBS 2. The school administrators ensure that the SWPBS team has regularly scheduled time to meet 3. There are adequate district resources (funding and time) allocated for SWPBS

9 Most important items for sustainability External Coaches 1. The school administrators (building principal or vice-principal) actively supports SWPBS 2. The district administration actively supports SWPBS (e.g., Describes SWPBS as a top priority, provides clear direction) 3. SWPBS is embedded into school and/or district policy (e.g., Action plans, school improvement plans, mission/vision statements Team Leaders 1. The school administrators (building principal or vice-principal) actively supports SWPBS 2. The school administrators ensure that the SWPBS team has regularly scheduled time to meet 3. A school administrator i t regularly l attends and participates i t in SWPBS team meetings Largest discrepancies for sustainability 1. The critical features of SWPBS are implemented as intended (i.e., They are used with fidelity) External Coach (N = 11) Team Leader (N = 14) M = 4.91, SD =.30 M = 4.50, SD =.52 M diff =.41, U = 45.5, z = -2.13, p =.085, r = There is regular measurement of fidelity of implementation (e.g., Team checklist, SET, Benchmarks of Quality) External Coach (N =11) Team Leader (N = 14) M = 4.73, SD =.47 M = 4.29, SD =.61 M diff =.34, U = 47, z = -1.87, p =.107, r =.37 More important for sustainability than initial implementation SWPBS has been expanded to other areas (e.g., Classrooms, buses, students with intensive needs, parenting workshops Z = -3.50, p <.001 SWPBS is viewed as a part of systems already in use (As opposed to being an add-on system) Z = -3.50, p <.001 The school team has a high level of expertise in implementing at the schoolwide level Z= -3.35, 35 p <.001 Current Research Main study now open for participation School team leaders (internal coaches) Coaches (external coaches) Non-sustaining school leaders/coaches: $100 gift certificate for completion Sustaining i school leaders/coaches: Draw for 3 $100 gift certificates per school sustainpbis@gmail.com to complete the SUBSIST

10 Contact t Information Kent McIntosh: kent.mcintosh@ubc.ca Now Hiring: Associate Professor Assistant Professor Graduate Students t Come join us in school psychology! Selected References Curtis, M. J., Castillo, J. M., & Cohen, R. (2008). Best practices in system-level change. In A. Thomas & J. P. Grimes (Eds.), Best practices in school psychology V (pp ). Bethesda, MD: National Association of School Psychologists. Doolittle, J. H. (2006). Sustainability of positive behavior supports in schools. Unpublished doctoral dissertation, University of Oregon. Gersten, R., Chard, D. J., & Baker, S. (2000). Factors enhancing sustained use of research-based instructional practices. Journal of Learning Disabilities, 33, McIntosh, K., Horner, R. H., & Sugai, G. (2009). Sustainability of systems-level evidence-based practices in schools: Current knowledge and future directions. In W. Sailor, G. Sugai, R. H. Horner, G. Dunlap (Eds), Handbook of positive behavior support (pp ). New York: Springer. Sindelar, P. T., Shearer, D. K., Yendol-Hoppey, D., & Liebert, T. W. (2006). The sustainability of inclusive school reform. Exceptional Children, 72, Vaughn, S., Klingner, J., & Hughes, M. (2000). Sustainability of research-based practices. Exceptional Children, 66,

11 SUSTAINABILITY OF PBIS AT DEWEY SCHOOL Evanston/Skokie School District 65 Andrew Krugly - Principal Kruglya@district65.net

12 Dewey School

13 Demographic Data 58.1 % White 15.9% African American 18% Hispanic 5.9% Asian/Pacific Islander 30.6% Low Income 13.9% Limited English Proficient 0.5% Chronic Truancy Rate 12.1 Mobility Rate 398 Students

14 Special Characteristics Enrollment increased by 75 students during and by 50 more students for school year. TWI Two-Way Immersion, Bilingual Classrooms We house kids from a battered women s shelter Inclusive kindergarten setting Full day kindergarten Just opened 3.5 million dollar addition with new library and new offices Currently under phase II construction - renovations of the older section of the school

15 Strong Initial Development Create a Very Strong Core Implement with Fidelity Principal Leadership and Modeling Staff Training Refreshers Regular Universal Team Meetings Use Data for EVERYTHING Sustainability: 8 Years Later Have Support in Place for the Principal Social Worker, Psychologist, PBIS Coach MAKE IT FUN FOR STUDENTS AND STAFF - CELEBRATE SUCCESS

16 Initial Development Use of External Coach (IL PBIS) 3 Day Training (Feb./Mar. 2002) Complete Universal Team One member from every team in the building Grade level, fine arts, special education Principal Training was useful Balance of theory and practice Training allowed our team to work through OUR plan

17 Initial Development TIME to get buy in (Apr. Aug. 2002) Staff PTA Leadership Parent Body at Large Develop and implement a plan with FIDELITY Adapt and embed into our school fabric Adapt features - NOT THE CORE this was the key to our success

18 The Core 4 Key Components adapted but present Matrix of Expectations Cool Tools Rewards Tiger Tickets Consequences ODRs (Office Discipline Referrals) Green Team/Universal Team Meetings This is our internal structure that allows us to keep moving forward

19 Principal Modeling Stay Positive Stay Consistent - Majors and Minors Sometimes adults and systems need to change NOT children MY approach to discipline HAD TO CHANGE Goal - Change behavior, not just punish

20 Critical Ideas for Successful Implementation and Sustainability Responding to staff challenges Continue to build staff skills by building on previous training - more than a one-shot deal Make the professional development fun and interesting Trainer needs to connect with the staff Use of data and examples Take your time getting to implementation Need about 80% buy-in for the critical components of the practice Continue to work with staff blockers so there is no sabotage Staff Training Refreshers

21 Data Based Decisions EVERYTHING! Data Collection/Progress Monitoring Tool - SWIS 9:00 ODR Spike - Caused Change in Morning Entrance Procedure (Universal) 9:15 ODR Spike - Caused Creation of Homework check at end of the day (Targeted) Which weekly Cool Tools to teach Gather data from children about celebrations Constant Goal less ODRs and higher student achievement

22 Data Shows Fidelity Scores on the SET In 2009 we switched from the SET to the BOQ to measure fidelity.

23 DATA SHOWS AREAS FOR IMPROVEMENT Disproportionality This graph allows comparison of the proportion of students within a school by ethnic grouping (number of students within an ethnic group divided by total enrollment) with the proportion of office discipline referrals from each ethnic group (number of office discipline referrals from an ethnic group divided by the total number of office discipline referrals). The graph allows assessment of the extent to which the proportion of referrals over-represents any specific ethnic group.

24 DATA SHOWS SUCCESS Lower Major and Minor Infractions

25 Targeted ODR Data September 2002 June 2003 Disruptions 146 September 2003 June 2004 Disruptions 35 Decrease of 76% from September 2004 June 2005 Disruptions 30 Decrease of 14% from September 2005 June 2006 Disruptions 20 Decrease of 33% from September June 2007 Disruptions 31 Increase of 55% from September June 2008 Disruptions 36 Increase of 14% from September 2002 June 2003 Homework 159 September 2003 June 2004 Homework 142 Decrease of 11% from September 2004 June 2005 Homework 93 Decrease of 35% from September 2005 June 2006 Homework 76 Decrease of 18% from September June 2007 Homework 82 Increase of 8% from September June 2008 Homework 90 Increase of 9% from 06-07

26 Targeted ODR Data Defiance/Disrespect Major/Minor Total September 2004 June /49 70 September 2005 June /59 82 Increase of 17% from September June /71 87 Increase of 6% from September June /48 64 Decrease of 27% from 06-07

27 DATA SHOWS SUCCESS Lower Suspensions Students Suspended Suspension Days ** ** Two of the four students who received out of school suspensions were new to Dewey School during and have not grown up with our culture of positive behavior. Two of the out-of-school suspensions involved extreme behavior that required one eight-day suspension and one ten-day suspension. The student who received the ten-day suspension was new to Dewey School.

28 DATA SHOWS SUCCESS Less Special Education Identification Tier 2 Support Plans Moved to Case Study Special Education Placement

29 DATA SHOWS SUCCESS Higher Student Achievement - ISAT READING Trends Total White African American Low Income IEP TWI - E TWI - S

30 DATA SHOWS SUCCESS Higher Student Achievement - ISAT Math Trends Total White African American Low Income IEP TWI - E TWI - S

31 Maintain fidelity during implementation. Adapt to the needs of your school, BUT.. Don t mess with the core. If you change the core, you cannot achieve the desired results. Use the data. Lessons Learned Have a process for collecting, analyzing, and using it. Use the data to make decisions. Use the data to get buy-in from others (teachers and parents). Opponents can t argue with data. Make changes and adaptations in practice based on data analysis. HAVE FUN WITH THIS IN YOUR SCHOOL. More fun = more investment and more success.

32 Questions???