10-11 Evaluation Report

Transcription

1 1-11 Evaluation Report Winter 212 About the Initiative The North Carolina Positive Behavior Intervention and Support Initiative is part of the North Carolina State Improvement Program funded through IDEA. Vision :: All schools in North Carolina will implement Positive Behavior Intervention and Support as an effective and proactive process for improving social competence and academic achievement for all students. Mission :: Provide leadership, professional development, resources, and on-going support in order for schools to successfully implement Positive Behavior Intervention and Support. Action :: The North Carolina Positive Behavior Intervention and Support (PBIS) sites are working to integrate their Safe Schools Plans, Character Education efforts and strategies, and discipline efforts in order to make schools caring and safe communities for learning. What s Inside... Celebrating Statewide Achievements..1 Context...2 Fidelity Impact School Spotlight...7 Supporting Sustainability...8 PBIS High Schools...8 Celebrating Statewide Achievements Beginning with the 27-8 school year, schools participating in the North Carolina Positive Behavior Intervention and Support (PBIS) Initiative could apply for recognition by documenting ongoing administrator participation, an active PBIS team, and an in-school coach and by providing evidence of implementation progress. Three levels of implementation are recognized: PBIS Green Ribbon Schools :: Green Ribbon schools have completed Module I team training and begun PBIS implementation. They have attained at least a Level I on the Implementation Inventory and 8% total on the Schoolwide Evaluation Tool (SET). PBIS Model Schools :: Model schools have completed all of the requirements for Green Ribbon Schools and have completed Module 2 training. They have also achieved a Level 2 on the Implementation Inventory and 9% total SET score. PBIS Exemplar Schools :: Exemplar schools have completed requirements for Green Ribbon and Model schools and have completed all three team training Modules. They have scored a Level 3 or higher on the Implementation Inventory and 95% total on the SET. At least two consecutive years of required behavioral, attendance, and academic data shows improvement, and they have documented at least one additional data element that they are tracking as a team (e.g. EBS Self-Assessment, Staff Retention Data, Climate Surveys, Referral Information for Special Education, Direct Behavior Rating, etc.). PBIS Schools earning recognition Exemplar Model Green Ribbon PBIS 1

2 CONTEXT Positive Behavior Intervention & Support in North Carolina Graham Cherokee Clay Swain Macon Jackson Haywood Alleghany Northampton Gates Ashe Surry Stokes Warren Rockingham Caswell Person Hertford Halifax Watauga Wilkes Avery Yadkin Forsyth Bertie Franklin Guilford Nash Caldwell Edgecombe Davie Martin Madison Iredell Randolph Wake Wilson Burke Chatham Beaufort McDowell Catawba Rowan Pitt Buncombe Johnston Greene Lincoln Lee Rutherford Cabarrus Harnett Wayne Polk Gaston Stanly Moore Lenoir Transylvania Pamlico Craven Sampson Jones Hoke Union Anson Duplin Onslow Carteret Henderson Yancey Mitchell Cleveland Alexander Mecklenburg Davidson Montgomery Richmond Alamance Scotland Orange Durham Granville Cumberland Vance Chowan Pasquotank Perquimans Washington Camden Currituck Tyrrell Hyde Dare Robeson Bladen Pender One hundred of the 115 school districts in the state have at least one school participating in the North Carolina Positive Behavior Intervention and Support Initiative. Columbus Brunswick New Hanover PBIS = Green Steady growth has been evident in the number of schools that have implemented PBIS and 4% of the schools in NC are participating in the Initiative. Schools Participating in PBIS Initiative Implementation Year NC PBIS Initiative critical features leadership Team :: Representative stakeholders meet regularly to support effort. Coordination :: State and local coordinators manage day-to-day operations. Funding :: Implementation is supported with targeted state-approved funds. Visibility :: Ongoing local, state, and national dissemination efforts are evident. Political Support :: Leadership Team reports annually on activities and outcomes. Training Capacity :: Team of trainers is available to build and sustain practices. Coaching Capacity :: Individuals are available to support emerging practices. Demonstrations :: Large number of schools available for model demonstrations. Evaluation :: Process is in place for collecting and reporting information on: implementation of action plan, use of Statewide PBIS (SWPBIS), or impact on student outcomes. 2 Winter 212

3 Fidelity The North Carolina Positive Behavior Intervention and Support Initiative requests that participating schools complete and submit implementation and evaluation tools: Implementation Inventory Schoolwide Evaluation Tool (SET) or Benchmarks of Quality (BoQ) NC PBIS Office Discipline Referral (ODR) Reporting Spreadsheet The NC PBIS Initiative recommends that teams use other tools such as the EBS Survey, School Safety Survey, and Team Implementation Checklist, which are online at In addition, the Initiative uses other data already collected by the Department of Public Instruction, such as information on achievement, attendance, and suspensions. The NC PBIS Initiative provides additional tools for school use around these data points as well as a Data Collection Manual on the website. PBIS Schools regularly assess the extent to which key features of PBIS are being implemented and they use this information to develop action plans for refining and sustaining the process Average SET SCOres N= Average SET subscale scores varied across levels of schools (N=453) and were generally higher for elementary (n=318) than middle (n=17) or high (n=28) schools SET scores across school years While the number of schools completing the SET has more than doubled over the past four years, average scores have remained above the recommended standard of 8% across all subscales Levels of Implementation Across School Years 1-11 PBIS 3

4 Fidelity The Implementation Inventory yields fidelity of implementation data across the full PBIS continuum of behavior support. Level < 8% on all subscales Level 1 8% on Universal subscale Level 2 8% on Universal and Secondary subscales Level 3 8% on Universal, Secondary, and Tertiary subscales Level 4 8% on Universal, Secondary, Tertiary, Practices, Systems and Data subscales While more schools continue to submit Implementation Inventory Online (IIO) scores, fewer seem to be scoring at level 2. Secondary level item analysis reveals lower scores in training and support for the PBIS team and staff implementing secondary interventions, as well as need to improve secondary data collection practices. Number of Schools at each IIO Level (N = 161) 28-9 (N = 3) 29-1 (N = 386) (N = 49) Data also shows shifts in the proportion of schools scoring at each level. Over the last two years, over 5% of schools scored at or above Level 2 compared to less than 25% scoring at or above Level 2 in Proportion of schools at each IIO Level Level 4 Level 3 Level 2 Level 1 Level Over time, state averages have increased across all six subscales of the IIO. The largest increases have occurred in the Secondary & Data subscale averages. Average IIO Scores Winter 212

5 Impact The North Carolina Positive Behavior Intervention and Support Initiative is focused on improving social competence and academic achievement for all students. Information from impact evaluation indicators reflects the extent to which targeted outcomes are being and/or likely to be achieved ODR Rates per 1 students per school day 1.2 ODR/1 Students/Day For the 1-11 school year, ODR data was gathered through ODR spreadsheets. While the NC High School ODR rates appear to be above the national average for PBIS schools, this is due to the data of one school that reported ODR rates more than 2 times higher than the next highest reported HS rate, 7 times higher than the national average, and almost 9 times higher than the state average for all other PBIS high schools. When this outlier is removed, average reported ODR rates are below the national average. Out-of-school Suspension Rates in PBIS Schools Over Time ODR/1 Students Overall suspension rates continue to decrease in PBIS schools. Type of School Out-Of-School Suspensions in PBIS Schools 35 3 ODR/1 Students Rates of suspension in PBIS schools continue to decrease, and are lower in Exemplar schools PBIS 5

6 IMPACT More PBIS schools with suspension rates below the state average also had above average academic performance composites (greater than 74.75), and met expected growth, high growth, and AYP targets. PBIS School Performance and OSS Rates (State Average OSS rate/1 students = 15.7 & Performance Composite = 74.75) Average Performance Composites Exemplar PBIS schools had higher average performance composites, and more Exemplar schools met expected growth targets Percentage of Schools Meeting Expected Growth Winter 212

7 school spotlight Brunson Elementary School Forsyth County Schools 28-9, 29-1, Exemplar School Percentage of Students Passing Both Reading & Math EOGs cedar creek middle School franklin County Schools 28-9, 29-1, Exemplar School Percentage of Students Passing Both Reading & Math EOGs holly springs high School wake County Schools Green Ribbon School Percentage of Passing Scores on EOC Tests OSS/1 std 1-11 PBIS 7

8 Supporting Sustainability North Carolina s PBIS schools contribute to research and support sustainability in several ways. The data that schools contribute each year helps the state team support ongoing implementation. From time to time, coaches and schools are asked to respond to surveys to collect additional data. This data is used to adapt professional development and technical assistance to meet the changing needs of schools and districts. In addition, data collected from schools, districts, and coaches is routinely shared with the national PBIS community at conferences. During the school year, information from a coaching survey completed by school district coaches and coordinators was used to develop a poster for the 21 Illinois PBIS Implementers Forum and a presentation for the International Conference for Positive Behavior Support. One of the findings of the project was that the presence of a district leadership team was associated with higher school performance on the SET and having more schools earn recognition. Additional information is available from your regional coordinator. PBIS High Schools With a growing number of high schools participating in PBIS, we began to ask questions about changes in graduation rates at high schools implementing PBIS. Because graduation is the culmination of 4 years of high school (and the outcome of the foundations laid in elementary and middle school), we compared the state graduation rate with the graduation rates of high schools who Average Graduation Rates had been implementing PBIS for at least four years. Overall graduation 85 rates as well as graduation rates for 8 students with disabilities exceed the state rate in schools implementing 75 PBIS for at least four years, and the 7 highest average rates exist among high schools who earned model 65 recognition (no high schools earned 6 exemplar recognition in 21-11) Public Schools of North Carolina state board of education department of Public Instruction exceptional Children division The North Carolina Positive Behavior Intervention and Support Initiative is part of the North Carolina State Improvement Program funded through IDEA. For more information, contact: Heather Reynolds Behavioral Support and Special Programs Exceptional Children Division NC Department of Public Instruction 6356 Mail Service Center, Raleigh, NC (VOICE) (FAX) STAFF Exceptional Children Division Mary Watson, Director Laura Snyder, Assistant Director Behavior Support and Special Programs Joe Simmons, Section Chief Dywanda Pettaway, Program Assistant State PBIS Consultant Heather Reynolds PBIS Technical Assistance Partner Bob Algozzine, UNC-Charlotte Regional PBIS Coordinators Minnie Kidd, Region 1 Vacant, Region 2 Correy Watkins, Region 3 Richard Tedescucci, Region 4 Cayce McCamish, Region 5 Laura Winter, Region 6 Vacant, Region 7 Julie Weatherman, Region 8 8 Winter 212