Lessons Learned: Creating and Maintaining Evidence-based Practices within the Classroom and Across School Environments Reesha Adamson, Ph.D. Missouri State University Starting Point Educators cannot make students learn or behave Educators can create environments to increase the likelihood students learn and behave Environments that increase the likelihood are guided by a core curriculum and implemented with consistency and fidelity The Good News.. Research reviews continue to indicate that effective responses to significant behavioral challenges in school include: Social Skills Training Academic Restructuring Behavioral Interventions = instructional strategies - teaching 1
Designing School-Wide Systems for Student Success Academic Systems Behavioral Systems Intensive, Individual Interventions Individual Students Assessment-based High Intensity 1-5% 1-5% Intensive, Individual Interventions Individual Students Assessment-based Intense, durable procedures Targeted Group Interventions Some students (at-risk) High efficiency Rapid response 5-10% 5-10% Targeted Group Interventions Some students (at-risk) High efficiency Rapid response Universal Interventions All students Preventive, proactive 80-90% 80-90% Universal Interventions All settings, all students Preventive, proactive Positive Behavior Support Social Competence & Academic Achievement OUTCOMES Supporting Staff Behavior Supporting Decision Making PRACTICES Supporting Student Behavior Center for Adolescent Research in Schools (CARS) Center grant funded 2008-2013 Focused on secondary age students with intensive social, emotional, and behavioral problems Main purpose: develop and evaluate an intervention package 2
CARS Principal Investigators Lee Kern, Ph.D., Lehigh University Steve Evans, Ph.D., Ohio University Tim Lewis, Ph.D., University of Missouri Howard Wills, Ph.D. & Debra Kamps, Ph.D., University of Kansas Mark Weist, Ph.D., University of South Carolina Need for Research Poor outcomes among students with emotional and behavioral disorders High Drop Out Rates Poor Academic Achievement Poor post-school outcomes Few evidence-based interventions targeting high school age students CARS Overview Years 1-2: Develop and refine assessments & intervention package with small sample of students (n=38, 3 states) Year 3: Evaluate and refine intervention package with larger sample of students (n= 60, 6 states) Years 4-5: Evaluate efficacy of intervention with large sample of students (n=647, 5 states) Randomized control trial Supplemental Studies Years 1-5: Conduct supplemental research studies in core areas relevant to students with emotional and behavioral problems 3
Conceptual Framework Enhancing School Capacity Healthy Choices Academic Skills Social Skills Building Child Capacity STUDENT Mental Health Family Support Behavior Management School Connectedness Increasing Family and Community Supports Goals of Major Study Identify/develop effective education and mental health interventions for students with emotional and behavioral problems What can be put in place and put in place well? Maximize intervention feasibility What do we know works? Maintain evidence based best practices approach How can we start with what schools are already doing? Develop intervention within existing school resources Considerations for Intervention Development Interventions must: multi-component, diverse delivered by practitioners after relatively little training minimal on-going technical assistance Classroom and Mental Health Manuals Assessment & Resources matched to interventions Wants/Need Supports/Tools in Manuals 4
Participants Gender Frequency Percent Female 212 33.8% Male 415 66.2% Total 627 ------------ School Communities Caucasian Urban Rural Suburban African American Hispanic/La tino Other Grade Levels 40 35 30 25 20 15 10 5 0 8th 9th 1th 11th 5
Special Education and General Education Special Education General Education Outcome Measures Parent Measures Demographic Questionnaire Behavior Assessment System for Children-II (BASC 2) Index of Family Relations Impairment Rating Scale (IRS) Disruptive Behavior Disorders Scale (DBD) Stress Index for Parents of Adolescents (SIPA) Alabama Parenting Questionnaire Teacher Measures Impairment Rating Scale (IRS) Behavior Assessment System for Children-II (BASC 2) Classroom Performance Survey Outcome Measures Students Measures Behavior Assessment System for Children-II (BASC 2) WJIII (Letter-Word Identification, Reading Fluency, Passage Comprehension, Calculation, Math Fluency, Applied Problems) Reynolds Adolescent Depression Scale (RADS) Multi-Dimensional Anxiety Scale for Children (MASC) Student Engagement Instrument Youth Risk Behavior Survey (YRBS) 6
Outcome Measures Direct Observation Teacher Instructional Context (whole group, small group) Instructional delivery Opportunities to respond Positive/Negative/Corrective feedback Student Engagement Disruptions Additional Measures Treatment Integrity Social Validity Tracking of Services/Interventions received outside of the school day Overview of Intervention Components Intervention Focus Core Student Challenge Specific Strategy Enhancing School and Teacher Capacity Building Youth Competence Academic Skills Emotional/Behavioral Problems Classroom Structure (Expectations & Routines) Evidence-Based Academic Instruction (OTR, Accommodations) Positive Teacher-Student Interactions Connectedness Social Skills Academic Skills Mental Health Mentoring Organization and Study Skills Interpersonal Skills Securing Effective Therapy Increasing Family and Community Supports Behavior Academic Skills Social Skills Mental Health Parent Education Securing Effective Therapy & Supports 7
Enhancing School and Teacher Capacity Classroom Structure (Expectations & Routines) Evidence-Based Academic Instruction (OTR, Accommodations) Positive Teacher-Student Interactions High School Classroom Challenges Lack of effective differentiated instruction and classroom management found in many classrooms Wide range of instructional delivery set-ups driven largely by curriculum and credit requirements High School Classroom Challenges Embedding classroom/instructional strategies within current system Accountability for student with EBD success (Sped? Gen Ed?) Incentive to change instructional environments and practices to increase the likelihood of student success 8
A peek into High School Classrooms 74% of teacher time was coded as non-teaching Teacher Feedback Positive 0.16 per hour Corrective 0.16 per hour Negative 0.44 per hour Opportunities to Respond Group 2.5 per hour Individual 0.4 per hour Student Engagement Active 18% Passive 12% Off Task 14% Down Time 57% Classroom Structure: Expectations Post, Teach, Review, & Reinforce a few positively state classroom expectations that are: Stated in positive terms Observable and Measurable Simply stated and age appropriate Kept to a minimum of 3-5 Classroom Structure: Routines Provide structure and increase predictability of classroom environment. Same instructional procedures used to teach academics Present the rule or routine. Post in prominent positions in the classroom at student eye level. Discuss why the rule or routine is important. Ask for student demonstrations of examples and non-examples of the desired behavior. Provide opportunities to practice with feedback. Explain what will happen when the rule or routine is followed and what will happen if not followed. Once taught, provide frequent and consistent acknowledgement for compliance. 9
Evidence-based Academic Instruction: OTR/ Positive Teacher-Student Interactions Increase ratio of positive to negative interactions (4:1) Provide instruction that involves frequent opportunities to actively respond to academic requests (OTR): Response cards (RC) Computer Assisted Instruction (CAI) Class-wide Peer Tutoring (CWPT) Guided Notes Research Question Within a core instructional subject area of mathematics, what are the effects of differing types of OTR (guided notes, class-wide peer tutoring, and response cards) on student disruptive behavior and academic engagement? Methodology Alternating Treatment Design Academic Growth Treatment Integrity Social Validity 10
Methodology Baseline: Teacher-led Instruction (academic content, introduction of materials, reviewing previously learned material) Intervention Randomized Rotation of Intervention (ABC, BCA, CAB, ACB, BAC, CBA) Implementation of Discriminative Stimuli/Teaching of Expected Behaviors Replication/Best Treatment Most effective intervention continued Student One Student One Disruption Rate: Baseline: 0.58 per minute (0.26 to 1.2) Intervention: GN: 0.22 per minute (0.0 to 0.6) CWPT: 0.28 per minute (0.0 to 0.5) RC: 0.22 per minute (0.0 to 0.6) Best Treatment (RC): 0.07 per minute (0.0 to 0.2) 11
Student Two Student Two Disruption Rate: Baseline: 0.46 per minute (0.0 to 1.53) Intervention: GN: 0.46 per minute (0.0 to 1.5) CWPT: 0.36 per minute (0.0 to 1.5) RC: 0.42 per minute (0.06 to 1.0) Best Treatment (RC): 0.2 per minute (0.1 to 0.3) Student Three 12
Student Three Disruption Rate: Baseline: 0.46 per minute (0.0 to 1.53) Intervention: GN: 0.29 per minute (0.0 to 0.7) CWPT: 0.3 per minute (0.0 to 0.5) RC: 0.42 per minute (0.18 to 1.0) Best Treatment: 0.13 per minute (0.0 to 0.3) Evidence Based Academic Instruction: Choice Incorporate choice and preference in classroom activities Design relevant activities that will incorporate the students interests: Assess the content for importance or relevance. Point out to students the importance and worth of information. Select topics that reflect students interests. Related content to local issues or problems that are important or familiar to students. Begin and conclude classes with statements such as This is an important topic/skill because and Why was this important for us to learn about? Allow students to select from a list of options. Evidence Based Academic Instruction: Accommodations Need for Appropriate Accommodations Approximately 85% of secondary students with EBD have at least one class in the general education setting. Help students: Access grade level materials Access instruction Improve classroom performance Prevent students from falling farther behind academically and potentially dropping out. 13
Purpose of Guide Facilitate selection of one or more accommodations that are matched to the student s specific academic or behavioral needs. Increase the match between a particular student s difficulty and an appropriate accommodation. Match testing accommodations to instructional accommodations. Implementation of Guide Provide teachers who rated accommodations as feasible and acceptable with model on how to choose accommodations Identify student problem Select related accommodation Have teachers implement the accommodation Assess student Performance Gather teacher feedback and treatment acceptability data post impelemtation Accommodations Guide Model 14
15
16
Accommodations vs. Modifications Changes to how academic content is presented or assessed Accommodations do not change what the student is expected to master Modifications change what the student is expected to master Academic Services, Supports, and Modifications for Students with EBD (Wagner et al., 2006) Modification Percentage of Students Elementary Middle High More time to take tests 72.8 72.6 75.6 Tests read to students 45.6 40.0 26.5 Modified tests 43.7 46.3 24.2 More time to complete assignments 66.5 67.3 54.2 Modified assignments 47.9 41,4 20.8 Modified grading standards 37.3 27.3 14.5 Slower-paced instruction 51.0 46.5 19.0 Peer tutoring 17.0 10.5 8.2 Adult tutoring 15.3 8.4 7.6 Learning strategies/study skills 33.0 36.5 27.5 CARS IEP REVIEW Most Common Accommodations Extended Times Tests Read Aloud Tested in Small Group or Alternative Setting Often Determined by Disability Students with ADHD frequently received extended time Accommodation seldom linked to student need Testing accommodations not matched to instructional accommodations 17
CASE STUDY Externalizing behaviors Participant Description Low academic abilities 3 rd Grade academic functioning General education co-taught History class Teacher selected interventions (based on feasibility) Accommodations Positive Teacher/Student Interactions Model Implementation Step #1: Collect three samples of student work demonstrating frequent errors or low grades. Identify accommodations provided in IEP 18
Student IEP Accommodations Testing: Alternative setting Extended time for completion Read test to student (if requested) Assignments and Instruction: Lower difficulty level-shorten assignments Provided structured time to organize materials Frequent reminders of rules Check often for understanding/review Extended time for oral responses Extended time for written responses Maintain assignment notebook Bathroom break first 5 mins. of class, unless emergency Use lined paper for written assignments Assist or provide notes and study guides Extended to create assignments (1 ½ weeks or as agreed upon w/ teacher) Step #2: Identify general indicators of concern Writing, specifically difficulty organizing writing Couldn t complete short answer Attention to detail Chose wrong answers on multiple choice and selected obviously wrong answers Planning and time management Ran out of time to complete the test even when given multiple class periods Step # 3 Identify accommodations matched to student s needs Writing: Difficulty Organizing Writing Graphic Organizers Attention to Detail Assign a Peer Partner Use Graphic Organizers Chunk Large Assignments into Smaller Tasks Planning and Time Management Chunk Large Assignments into Smaller Tasks Graphic Organizers 19
Step # 4: Coordinate accommodations Model Accommodations Writing: Difficulty Organizing Writing Graphic Organizers Attention to Detail Assign a Peer Partner Use Graphic Organizers Chunk Large Assignments into Smaller Tasks Planning and Time Management Chunk Large Assignments into Smaller Tasks Graphic Organizers IEP Accommodations Testing- Alternative Setting Extended time for completion Read test to student (if requested) Assignments and Instruction- Lower Difficulty Level-Shorten Assignments Provided Structured Time to Organize Materials Frequent Reminders of Rules Check often for understanding/review Extended time for Oral Responses Extended time for Written Responses Maintain Assignment Notebook Bathroom Break first 5 mins. Of class, unless emergency Use lined paper for written assignments Assist or Provide Notes and Study Guides Extended to create assignments (1 ½ weeks or as agreed upon w/ Teacher) Step #5: Meet with the student for input and preferences Student agreed with teachers concerns. Student agreed to all suggested accommodations. Student requested: A different seat with less distractions during independent work. Materials if he forgot to bring them. Step #6: List accommodations to be implemented and evaluated. Prioritize Determine instruction or testing Define and describe conditions: 1) Use graphic organizers (I/T) All in class work, Teacher-created or pre-made organizers. 1) Change seat to reduce distractions (I/T) Student and teacher will meet to discuss alternative seat, can be changed if student does demonstrate classroom expectations. 1) Materials be given to student if forgotten (e.g. pencil) (I/T) Student must request material at the start of the class and give teacher collateral for the return of material at the end of the class period. 2) Assign a peer partner (I) All in-class work with a teacher chosen partner. 3) Chunk large assignments into smaller tasks (I) All in class assignments spanning multiple class periods with a teacher-created checklist of completed tasks. 20
Percentage of Time 2/7/2017 Step #7: Teach the accommodations Assign a peer partner What are the appropriate behaviors of working with a peer? What noise level can you work with a peer? What does helping vs. doing look like? Use graphic organizers How do I use graphic organizers (5 paragraph outline, flow chart, venn diagram, checklist, etc.)? Chunk large assignments into smaller tasks How do I use a checklist? How do I manage my time? Change seat to reduce distractions How do I transition? When is it appropriate to move? Materials given to student if forgotten (e.g. pencil) How do I know what materials I need? What is appropriate collateral? Step #8: Examine at least three samples of student work/tests to determine if there is a change in the performance trend Class Grade Increased from a 24% to a 77% Highest class grade he had received since beginning high school! Active Engagement 100 90 80 Baseline Graphic Organizers Change Seat Materials Given Peer Partner Chunking Assignments Reversal Graphic Organizers Change Seat outside classroom Materials Given Peer Partner Chunking Assignments 70 60 50 40 30 20 10 0 1 2 3 4 5 6 7 8 9 10 11 Data Observation Periods 21
Building Youth Competence Mentoring Organization and Study Skills Interpersonal Skills Securing Effective Therapy Mentoring Foundation of Check & Connect Core Intervention across Students Developed for high-risk urban students at the secondary level (Anderson, Christenson, Sinclair, Lehr, 2004; Evelo, Sinclair, Hurley, Christenson, Thurlow, 1996) Utilizes a monitoring system with two components Check Systematically assess the extent to which students are engaged in school. Connect Respond on a regular basis to students educational needs according to their type and level of risk for disengagement from school. Establish an adult mentor at school to enhance school engagement What is Check & Connect? Four components = 1.A mentor who works with students and families for a minimum of 2 years, 2.Regular checks using school data 3.Timely and individualized intervention - connect 4.Engagement with families 22
What is Check & Connect? Descriptive Characteristics Targeted (tier 2) or intensive (tier 3) intervention A structured mentoring intervention Empirically supported Clearly delineated elements What is Check & Connect? Core Elements (fundamental principles) Relationships Focus on alterable variables Personalized, data-based intervention Long-term commitment Participation in and affiliation with school Problem solving & capacity building Persistence-Plus Student Engagement Engagement is not a fixed attribute, but rather an alterable state of being Engagement is highly influenced by home, school, and peers influences can serve as inhibitors or facilitators. *Role of mentor is designed to work as broker of services for promoting & facilitating student engagement 23
Check & Connect Check aspect Systematic monitoring of warning signs for withdrawal Focus on data that is readily available Emphasis on alterable indicators those that can be addressed through intervention Connect aspect Creating a person-environment fit Support that is tailored to individual student needs, based on Level of engagement with school Associated influences of home and school Leveraging of local resources Organization and Study Skills Intervention requires consistent monitoring and routine check-ups Daily Planner Missing Assignment Tracker Organization Checklist 24
Securing Effective Therapy and Supports Mental Health Challenges Personnel with time Assessment Evidence-based interventions the fit within instructional context Choosing Mental Health Interventions 1 What is the Problem? 1.1 Gather teacher, parent and self report of difficulties related to impairment at school 1.2 Obtain relevant school records to assess impairment (e.g., office referrals) 1.3 Conduct Observations 1.4 Consult with CARS MH consultant 2 5 Interpretation Given the Information Gathered 2 3 4 Does Does Does student student student show show signs show signs signs of of of poor anxiety or withdrawal, social skills, worry? irritability, frequent negativity conflict or poor or sad problem mood? YES YES YES solving skills? 2.1 The C.A.T. 3.1 Coping 4.1 Project with Stress Interpersonal program Skills Group 16 Session Program 15 Session 28 thirty minute Program Session Group or Program Individual Group or Individual Group 5 Does student disrupt class, argue, or display other difficult to manage behaviors? YES 5.1 Referral to Special Education Teacher for classroom management or Teacher-Student communication concerns, and/or the development of an FBA/BIP 6 Prioritize Intervention 6.1 PRIORITY: Rank problems based on how much they are impairing school & academic functioning 6.2 FEASIBILITY: Indicate whether each of the interventions can be feasibly provided 6.3 ACCEPTABILITY: Indicate whether the student, teachers and/or parents are willing to participate 6.4 Record list of interventions that are feasible and acceptable in order or priority ranking 7 Implement the Intervention 7.1 Begin with the feasible intervention with the highest clinical priority and proceed through entire intervention 7.2 Measure outcomes and use data to inform modifications to interventions 5.2 De-Escalation Techniques Family Interventions Mental Health Interventions CBT for Anxiety/Worry CBT for Depressed Mood Interpersonal Skills Group for Social Skill Impairment Family Interventions 25
Resource Mapping Assessment of School Services Interview: Staff STAFF MEMBER DEGREE POTENTIAL MH TIME SPENT Assistant Principal MA, Curr & Instr -Parent Support Groups -Individual Counseling School Psychologist Ph.D., Sch Counseling -Group Counseling: Peer Conflict -Individual Counseling -Consultation w/parents Senior Counselor MA, Sch Counseling -Individual Counseling -Parent Ed 1 hr/month As needed As needed As needed 1 hr/week 1-2 hrs/week As needed Freshman Counselor MA, Sch Counseling -Individual Counseling 20 hrs/week School Counselor MA, Sch Counseling -Social Skills Knitting Group -Individual Counseling Security Coordinator Associate -Individual Counseling -Home Visits Graduation Coach MA, Literature -Social Skills Knitting Group -Individual Counseling -Open Office Parent Ed 1 ½ hrs/week 28 hrs/week As needed 4-5 hrs/week 1 ½ hrs/week 2 hrs/week As needed Transition Specialist MA, SpEd -Individual Counseling 1 hr/week School Nurse MA, Health Ed -Parent Support Group -Individual Counseling Social Worker MA, Soc Work -Parent Ed -Individual Counseling 2 hrs/month 2-3 hrs/week 1-2 hrs/week 1 hr/week Resource Mapping: ACTIVITY Individual Counseling Group Counseling Parent Related Activities TOTAL TIME SPENT (EXCLUDES AS NEEDED ) 53-56+ hours/week 1 ½ hours/week 5-6 hours/week RESOURCE MAPPING Are you aware of any staff or resources at the school which might be utilized to a greater degree to implement mental health interventions? STAFF MEMBER Assistant Principal School Psychologist Senior Counselor Freshman Counselor School Counselor Security Coordinator Graduation Coach Transition Specialist School Nurse Social Worker Other AvailableResources OTHER STAFF Officer Snyder, Social Worker Social Worker, Intervention Specialists School Counselor, ESL Aides School Nurse Social Worker, Other Counselors Officer Snyder New to School Social Worker, School Counselor Nurse, Graduation Coach, Counselor 26
Increasing Family and Community Supports Parent Education Securing Effective Therapy & Supports Parent Education Group Parent Training for Student Participants Focused on Structure and Support for the Home Behavior Contracting Access Outside Supports Support Group for Family Participants For more information www.ies-cars.org Reesha Adamson, Ph.D. radamson@missouristate.edu 27