ABSTRACT EXAMINING THE RELATIONSHIP BETWEEN QUALITY OF RTI IMPLEMENTATION AND READING ASSESSMENT DATA. by Kari E. Sanders

ABSTRACT EXAMINING THE RELATIONSHIP BETWEEN QUALITY OF RTI IMPLEMENTATION AND READING ASSESSMENT DATA by Kari E. Sanders The purpose of this study was to examine the relationship between the quality of self-reported RTI reading implementation and students reading scores on a state-wide achievement assessment. Specifically, the study focused on the relationship of implementation of the three tiers of RTI from 35 different schools who completed the RTIS-R scale and the reading scores obtained on the Ohio Achievement Assessment. Demographic variables were controlled for and data was analyzed using a hierarchical linear regression. Results suggested that Tier 2 RTI supports significantly contributed to performance on the Ohio Achievement Assessment above and beyond the effect explained by the percentage of economically disadvantaged students. This paper includes limitations of the study, implications for schools, and directions for future research.

EXAMINING THE RELATIONSHIP BETWEEN QUALITY OF RTI IMPLEMENTATION AND READING ASSESSMENT DATA A Thesis Submitted to the Faculty of Miami University in partial fulfillment of the requirements for the degree of Educational Specialist Department of School Psychology by Kari E. Sanders Miami University Oxford, Ohio 2013 Advisor Dr. Amity Noltemeyer Reader Dr. Michael Woodin Reader Dr. William Boone Reader Dr. Jessica Hoffman

Table of Contents Introduction 1 Description of RTI model.. 1 High quality reading instruction in an RTI model. 4 RTI implementation quality... 5 Rationale and purpose for the study.. 7 Methodologies... 7 Instrumentation.. 7 Participants. 9 Variables... 10 Procedures.. 10 Design.... 11 Data analysis.. 11 Results 12 Descriptive statistics.. 12 Regression analysis.12 Discussion...12 Limitations and future directions 14 Implications for practice 15 References.. 18 ii

Examining the relationship between quality of RTI implementation and reading assessment data An innovative method for the future of school psychology is Response to Intervention (RTI). RTI is a comprehensive framework for improving student outcomes that involves (a) screening all children for academic and/or behavioral problems, (b) monitoring the progress of children at risk for difficulties in these areas, and (c) providing increasingly intense interventions as needed based on results of ongoing progress monitoring (Fletcher and Vaughn, 2009). RTI was created to provide a system for identifying struggling students and providing prevention and intervention services necessary to provide help early rather than waiting for them to fall enough behind to qualify for special education services (Buffum et. al, 2010). Many components of RTI (e.g., progress monitoring, early intervention) are supported by the Individuals with Disabilities Education Act (IDEA, 2004) along with the No Child Left Behind act (NLCB, 2001). Although RTI has been used for both academic and behavior problems, it is perhaps most frequently utilized in the area of reading. Many students receiving additional supports in schools have the most difficulty improving their reading. It has been shown that learning difficulties are the cause of only 2-6% of reading failure issues (Bursuck & Blanks, 2010). The other failures may at least partially result from teaching or curriculum deficiencies that have the potential to be addressed by implementing an RTI model. Description of the RTI Model RTI is composed of three tiers. The first tier consists of a high quality core curriculum provided to the general student population. All students would be part of a universal screening process in this tier which involves their completion of a curriculum based measure assessment, possibly in conjunction with other screening measures. The students who are having difficulties in a particular subject area are identified through this assessment and will enter the second tier. 1

Students in the second tier should receive interventions according to the deficits emerging from screening measures. Some research has shown that small group interventions that are structured and timely are very effective for students (Buffum, Mattos, & Weber, 2010). These students are monitored every few weeks to assess their progress in the intervention. Students who continue to display concerns in their level or rate of progress may be identified for tier three intervention. The third tier consists of highly intensive, individualized intervention supports matched to student needs. In this tier, progress is monitored at least weekly. If a student does not improve after multiple rounds of increasingly intense high quality interventions that are implemented with fidelity, he or she may be referred for special education services. Research supports this three-tiered model of prevention and intervention within RTI (e.g., Batsche et al., 2006; Brown-Chidsey, Bronaugh, & McGraw, 2009; Burns & Gibbons, 2008; Mellard & Johnson, 2008;). However, RTI implementation is also distinguished by several components within each Tier. Although there are a plethora of such possible components that have been proposed, a thorough review of the literature by Noltemeyer, Boone, & Sansosti (under review) suggests six that are so central to RTI implementation that they emerge across studies and texts. These core components include: (1) Assessment: The RTI model involves screening the entire student population throughout the school year to identify at-risk students. This allows administrators and educators to catch problems early on. RTI also involves the use of progress monitoring measures. These measures help monitor how well the interventions are working for the student. (e.g., Brown Chidsey & Steege, 2005; Brown-Chidsey, Bronaugh, & McGraw, 2009; Mellard & Johnson, 2008) (2) Data-based decision making: Assessment data results are used to obtain an accurate view of how well the child is progressing and alter instruction accordingly. An example of data-based decision making would be using progress monitoring data to see if an intervention is actually producing 2

significant results for a student (e.g., Batsche et al., 2006; Mellard & Johnson, 2008). (3) High quality research-based instruction and intervention: It is essential for students to receive instruction and interventions that have been researched as successful. If the core curriculum in schools is research based and delivered effectively, it should be successful for at least 80% of the general education population. For the children who need more assistance, high quality interventions should be used to ensure that the children are receiving interventions that will work. (e.g., Brown-Chidsey & Steege, 2005; Gersten et al., 2009; Foorman & Torgeson, 2001; Bursuck & Blanks, 2010) (4) Treatment integrity: Treatment integrity occurs when an intervention or type of instruction is implemented as it was intended to be implemented. This means that all parties involved will make sure to execute their plan the best they can in order to achieve the greatest results. (e.g., Goss, Noltemeyer, & Devore, 2007; Glover, 2010) (5) Professional development: Educators and administrators will learn many new techniques for improving their students education. For example, they will learn to be more aware of their students progress and specific academic and behavioral needs. (e.g., Brown-Chidsey, Bronaugh, & McGraw, 2009; Hall, 2008) 6) Administrative leadership/schoolwide supports: To maximize the likelihood of success, support for RTI from the entire school staff and parents is needed. The more involvement from the administrators, teachers, staff, and parents yields successful results in academic and behavioral improvement. It is essential that all members are on board for the RTI framework and understand how it works. (e.g., Hall, 2008; IET Manual, 2007) Some research has documented positive outcomes that result from implementing the RTI system. The most important of these outcomes is improved academic results (Ardoin, Witt, Connell, & Koenig, 2005; Barnett, Daly, Jones, & Lentz, 2004; Flaum, 2009). RTI also has been shown to reduce special education referrals (Brown-Chidsey, 2007; Johnson, & Smith, 2008; VanDerHeyden, Witt, & Gilbertson, 2007). This may 3

include English Language Learners who only need language help and not special education services or students who only need help in one area and would benefit from a tier two intervention. Despite these positive outcomes, it is important to note that RTI research is limited in quantity. In addition, other criticisms of RTI have been posed. For example, the RTI framework often focuses on tasks that are easy to measure and not on addressing more complex cognitive tasks (Artiles & Kozleski, 2010 as cited in Noltemeyer, 2012). Specifically in the area of reading, RTI efforts often focus on mechanical skills such as phonics, and fluency rather than vocabulary and comprehension. Some researchers (e.g., Teale, Paciga, & Hoffman, 2007) have proposed that such a limited focus may hinder the development of critical thinking skills or narrow the curriculum. These and other criticisms of RTI, such as difficulty implementing and sustaining such a systemic change, should be kept in mind in addition to the positive outcomes associated with RTI when evaluating reading ability. High Quality Reading Instruction in an RTI Model Using an RTI model can allow schools to identify the children who need reading interventions early on and provide this intervention accordingly. It can also allow for progress monitoring which, if used correctly, can assist in determining whether or not specific interventions used are working for that student. Using RTI to improve literacy skills has been found to be effective for ELL students (Healy, Vanderwood, and Edelston, 2005), children with disabilities (Lemons and Fuchs, 2010), and for low-income children (Gettinger and Stoiber, 2007). Within an RTI model, it is important to identify what features characterize high quality, research-based reading instruction and intervention. Although their relative importance and emphasis in the curriculum has been debated, and they have been criticized for their over-emphasis of code-based skills and exclusion of writing, five big ideas of beginning reading are important to consider within an RTI model: 1) Phonemic awareness: This is the process of being able to identify and manipulate different sounds within a word. It has been found that children 4

who are taught phonemic awareness early on have better reading achievement in the future (Pullen and Justice, 2003). 2) Phonics: Having phonics skills involves understanding the relationship between the letters and what sound they make. If a child acquired phonics skills he or she would be able to recognize words more quickly and accurately. 3) Fluency: Fluency can be described as the ability to rapidly and correctly recognize words with expression (NICHD, 2000). Having reading fluency skills is important because it allows students to better understand and remember the material they have read. If a student reads with difficulty it is harder for them to comprehend and remember the material. 4) Vocabulary: Vocabulary is an important part of reading because having a wide vocabulary not only allows one to identify words in a passage quicker, but also allows for higher comprehension. If a child understands the word he or she is reading, it will be easier to comprehend what the passage means. 5) Comprehension: Comprehension involves connecting what has been read to the knowledge one already obtains. Students with good comprehension skills use their thought processes to analyze what they have read until they understand. Burns and Gibbons (2008) mention that good reading instruction helps children learn how to manipulate the sounds of each letter of the alphabet in order to make recognition of words much easier. Although supporting holistic reading development is important at all ages, they also note that first graders should be specifically focusing on phonics and phonemic awareness; second graders should be focusing more intently on fluency and phonics; and third graders should be refining their comprehension and fluency skills. It is also recommended that scientifically supported methods are used for tier one instruction (Kovaleski and Pederson, 2008). It is considered best practice for institutions to use assessment data to make decisions about children moving from tier to tier. RTI Implementation Quality 5

One factor that can hinder the effectiveness of an RTI program is the quality of the implementation. If RTI best practices are effective, then implementation quality is important because it allows children to receive the best quality and the largest amount of implementation possible. If children are consistently provided with these interventions and curricula they are more likely to improve their academic achievement in comparison to children who are not receiving high quality, consistently implemented educational practices. Investigations in other comprehensive school reform initiatives (e.g, Datnow, Borman, & Stringfield, 2000; Urban Institute, 2007) have documented that higher implementation fidelity is often associated with better student outcomes. One study focusing specifically on RTI found that high quality implementation of RTI can reduce the number of students qualifying for special education (Brown-Chidsey, 2007). Hamre et. al (2010), found the quality of a teacher s implementation of literacy curricula to be correlated with child academic outcomes. This study measured the degree of variability of three different aspects of implementation. The researchers wanted to see if these factors of fidelity have an effect on literacy and language of children. The results indicated that children in classrooms that implement the curriculum often and with high quality made more significant gains in literacy skills. These results imply that in order for the instruction within an RTI model to be effective, the implementation needs to be of high quality. Noltemeyer & Sansosti (2012) desired to examine the relationship between implementation quality and the results on students academic outcomes. The study involved using the Implementation Evaluation Tool to determine the features of the Integrated Systems Model used in the schools, where improvement is needed, and to compare implementation with previous school years. DIBELS oral reading fluency data was used to determine the number of correct words read per minute. A hierarchical linear analysis was used to determine the relationship between the predictor variables and the criterion variable. The criterion variable was the DIBELS ORF score. The predictor variables were organized into two sets. The first was a control block that included the variables to be controlled for in the analysis. The second block was composed of the implementation variables. A relationship was found between the level of implementation in the schools and the students academic outcomes. This study suggests that the level of 6

implementation is crucial to children s academic success. However, the measure used was not thoroughly validated so we do not know that it accurately assessed implementation quality. The study also used a small sample of schools which affects generalizability. Rationale for and Purpose of Study RTI is an innovative model that has the potential to aid schools in helping children with reading skill deficits from going unnoticed. The literature supports this framework for increasing literacy skills in children. There is, however, little research available on whether or not the self-reported quality of RTI reading implementation affects the literacy scores children receive. This is important because these reports not only increase the participant s awareness of the quality of their RTI implementation in reading but also demonstrates whether there is a relationship between the reported quality and the scores children receive. Although some research has explored this issue (e.g., Noltemeyer & Sansosti, 2012), it did not use a validated measure of implementation quality, was not focused specifically on reading implementation, and used a very limited sample. This study is designed to address these limitations. The purpose of this study is to examine the relationship between quality of RTI reading implementation and students reading scores on a state-wide achievement assessment. Specifically, the research question is: Is there a relationship between selfreported quality of implementation of RTI by tier and student performance on a statewide reading achievement assessment when controlling for demographic school variables? Methodologies Instrumentation The RTIS-R (Noltemeyer et al., under review) (instrument not included due to its publication status; contact first author for instrument, anoltemeyer@miamioh.edu) will be used to assess self-reported quality of RTI implementation. This scale was created to assess the quality of schools RTI implementation in the area of reading. Rasch psychometric theory (Wright & Masters, 1982) was used to guide the development of the 7

scale. Six central components were found necessary for RTI implementation. These include assessment, data based decision making, high quality research-based instruction and intervention, treatment integrity, professional development, and administrative leadership/schoolwide supports. The questions in the RTIS-R were developed based on these six components. The initial development of the RTIS-R consisted of a three-phase process (see Noltemeyer et al., under review). The first phase involved the original drafting of the RTIS-R. This was completed collaboratively by Noltemeyer et al. (under review) based on an exhaustive review of the best practices RTI literature. The second phase involved obtaining feedback about the scale from four expert reviewers. These reviewers thoroughly evaluated the scale to ensure that it included all the necessary components. The third phase involved conducting cognitive interviews of educational practitioners currently in the field. The cognitive interviews were conducted with four participants in order to assess whether participants understood what the items were asking and to analyze the thought processes that occurred when participants were answering the items. Based on the feedback obtained during each of these phases of development, revisions to the scale were made to improve its quality and validity. After these revisions, the final scale contains 30 items. For each item on the scale the participants selected which of 5 descriptors best matches their school s current level of implementation. There are also three demographic fill-in items at the end of the RTIS- R to gather more information about the respondents and their schools. As mentioned above, the instrument was reviewed by experts in the field and aligned with research in the field to enhance its validity. Instrument reliability data were obtained through a validation study of 53 school psychologists and building principals, using Rasch analysis (Noltemeyer et al., under review). Results of the Rasch analysis suggested that the RTIS-R is a rigorous instrument. Neither items nor people are misfitting much, suggesting that the overall development of the instrument was good and that respondents acted in ways which we would expect. Despite the overall high levels of reliability, the analysis did reveal three items that were not functioning as would be expected. Consequently, these three items (which assessed the amount of time schools spent implementing instruction at each of the three tiers) were removed from further 8

iterations of the analyses and are not included in this study. Finally, the analyses conducted by Noltemeyer et al. (under review) also revealed that the items comprising each Tier of the RTI model made empirical sense and that the items related to Tier 1 were easier to endorse than those in Tier 2 and that those in Tier 2 were easier to endorse than those in Tier 3. Participants The participants in this study were 28 principals and 25 school psychologists from 39 schools in the state of Ohio. These participants were the same as those used in the validation study completed by Noltemeyer et al. (under review), as the current study utilized the same data in a different manner than the validation study. The percentage of participants that were school principals was 52.8% and the percentage of participants that were school psychologists was 47.2%. The average number of years the participating school has implemented RTI was 3.9 (See table 2). Four of the schools that returned surveys do not serve students that are of age to be completing the OAA and these schools were not included in the final sample of 35 participating schools. A diverse, purposive convenience sample that targeted schools implementing RTI was used to select all participants. The schools were chosen based on information obtained from faculty and school-based professionals who identify themselves as implementing RTI at some level. Publically available demographic information (e.g., percentage of economically disadvantaged students and percentage of minority students) on each of the participating schools was obtained from the Ohio Department of Education Data Warehouse Reports on http://www.education.ohio.gov/. The average percentage of students in the participating schools that reached the proficiency level on the Ohio Achievement Assessment in the area of reading was 89.2%. This number was higher than the statewide average. The average enrollment of the participating schools was 519.2 students. The percentage of non-white students in the schools, on average, was 17.2%, the average percentage of economically disadvantaged students per school was 37.3%, and the average percentage of students with a disability in the schools was 16.1%. These three demographic percentages all fell below statewide averages, although to varying degrees (See table 1). 9

Variables There were two variables of primary interest in this study: 1) Ohio Achievement Assessment results (outcome variable): The Ohio Achievement Assessment is a measure for children in grades 3-8. It is intended to measure what children know and should be able to do in a variety of subjects based on their grade level. The results of this measure were taken from the Ohio Department of Education website (http://www.education.ohio.gov). These results are reported in four different levels of proficiency: limited, basic, proficient, accelerated, and advanced. For this study, the OAA variable was comprised of the total percentage of students in the school scoring in the proficient, accelerated, and advanced ranges on the OAA reading test (indicating they met or exceeded grade-level expectations). 2) Self-reported scores on the RTIS-R by Tier (predictor variable): The self-reported scores are those that were reported by the principal or school psychologist. The results from the measure reveal the reporter s perceptions on the quality of RTI implementation in his or her school. There are items for each tier of RTI which allows the researcher to examine the quality of implementation per tier. For this study, each response on the RTIS-R was assigned a different numeric value, with 0 indicating the lowest level of implementation and 5 indicating the most advanced level of implementation. The RTIS-R variables will be the total score by tier after these values are added together. In addition, three variables were controlled for in the analyses. The first variable was the percentage of non-white students in the school. The second included the percentage of students with a disability. The final variable was the percentage of economically disadvantaged students. Controlling for these variables allowed the researchers to effectively see the effects of RTI implementation on Ohio Achievement Assessment beyond what effects may have been a result of a control variable. Procedures An initial phone call or email was sent to the participants to determine whether or not they were implementing RTI and would be willing to complete the instrument. The RTIS-R, the consent form and a return envelope were sent to all participants contacted in 10

those schools implementing RTI. A reminder phone call or email was made to participants two weeks after the original distribution date. Participants who completed the RTIS-R and the consent form received a $10.00 Amazon.com gift card. Participants who partially completed RTIS-R and consent form received a $5.00 Amazon.com gift card. Design This study used a correlational research design. This design was appropriate because it allowed the researcher to examine the relationship between self-reported RTI reading implementation and the results on the Ohio Achievement Assessment. Correlational designs do not allow the researcher to determine whether implementation quality causes better OAA performance. However, it can reveal whether an association exists between these two variables. Data Analysis The researcher entered the results into SPSS and the thesis advisor spot checked for accuracy. The spot check involved the advisor reviewing the data entered into SPSS and making sure it correlated with the results on the survey. The results were analyzed using a hierarchical linear regression. A hierarchical linear regression is a measure that allows variance between levels to be assessed at multiple levels. Relationships between the predictor variables and the criterion variable were analyzed. The criterion variable consisted of the results of the school s OAA data. There was a control block that included predictor variables of (a) percentage of non-white students, (b) percentage of economically disadvantaged students, and (c) percentage of students with a disability. The second block of predictor variables included (a) Tier 1 RTI implementation, (b) Tier 2 RTI implementation, and (c) Tier 3 RTI implementation. The analysis method used was similar to that used in the previous study by Noltemeyer and Sansosti (2012). As described by these researchers, by entering the data in two different sets, we were able to decide if analyzing the three RTI implementation variables in a second block increased the proportion of variance in the criterion variable significantly. This means that the significance would be above and beyond what can be attributed to the demographic variables in the first block. Each block was analyzed in a 11

stepwise order. In this type on analysis, each variable in the block is entered in a specific order and it only remains in the equation if it significantly contributes to the model. The other variables that remain in the model are then re-tested to determine whether they still provide contribution to the model. If these variables do not contribute, they are removed (Brace, Kemp, & Snelgar, 2006). Using this procedure increases the likelihood that the variables that remain did significantly contribute to the model (Brace et al., 2006, as cited in Noltemeyer & Sansosti, 2012). Results Descriptive Statistics The average Ohio Achievement Assessment pass rate for the participating schools was 89.2%. The average score on the Tier 1 section of the RTIS-R scale was 28.4. The average score on the tier 2 questions was a 25.8. The average score on the tier 3 section was 23.8. The variance of participant answers in the tier 1 section of the RTIS-R was 30.9%. In tier 2 there was a 44.4% variance in participant s answers and in tier 3 there was 68.6% variance. Regression Analysis The hierarchical linear regression analysis revealed two significant models. The first model revealed that the percentage of economically disadvantaged students was a significant predictor of the percentage of students who passed the reading section of the Ohio Achievement Assessment (see Table 3). The second model revealed that implementation of Tier 2 supports significantly contributed to performance on the Ohio Achievement Assessment above and beyond the effect explained by the percentage of economically disadvantaged students (see Table 3). Tier 1 implementation and Tier 3 implementation were not found to contribute significantly to the Ohio Achievement Assessment results. Discussion The purpose of this study was to examine if there is a relationship between RTI implementation and Ohio Achievement Assessment results. This type of information 12

would be helpful in assisting educators plan for the best education possible. A hierarchical linear regression was used to examine the effect of each tier within the RTI framework on the OAA data. Control variables of economic disadvantage, non-white students, and students with a disability were used in the analysis in order to more clearly assess the relationship of each RTI tier and OAA data. The results of this study displayed a significant effect of Tier 2 implementation on the results of the Ohio Achievement Assessment. After controlling for the percentage of economic disadvantaged students, students with a disability, and non-white students, Tier 2 implementation still had a significant effect on OAA results. This is consistent with findings from Noltemeyer & Sansosti (2012) suggesting that higher quality RTI implementation is associated with better student reading outcomes. Although the direction of the causal relationship documented in this investigation remains unknown, these results suggest that higher fidelity of Tier 2 reading supports may increase school performance on the OAA. This is consistent with previous research suggesting that early intervention with at-risk populations can have positive effects (Flaum, 2009; Nunnery et.al, 2006; Vaughn et.al, 2010;). The availability of these data will assist schools in creating programs for students who qualify for Tier 2 supports. These results also raise the question of why Tier 1 and Tier 3 implementation quality were not significant predictors of reading outcomes. These questions are particularly relevant given previous research supporting the effectiveness of these Tiers at improving outcomes (e.g., Ardoin et. al, 2005; Burns, Appleton, and Stehouwer, 2005). The implementation of Tier 1 supports across schools was quite similar, resulting in the smallest variance in the study of 30.9%. The Tier 1 implementation score (M=28.4) was the highest among tiers, suggesting that, on average, most schools implement Tier 1 services better than the other two tiers. This similarity among schools may explain why there was not a significant effect of Tier 1 on the OAA. In contrast, the Tier 3 average score was the lowest among tiers (M=23.8) and had the highest variance across schools, 68.5%. Since the scores of Tier 3 were the lowest, this may be the reason that there was no significant effect on the OAA data. In addition, Tier 3 supports only impact a very small percentage of the student population, those students with the most significant and non-responsive needs. Consequently, the impact on the overall student population may 13

be more subtle although those individual students affected may demonstrate pronounced gains. These results also suggest that schools could gain the most improvement in RTI implementation in Tier 3 supports. Limitations and Future Directions There are multiple limitations involved with the current study. First, the participant sample was exclusively from one state. The study could have been expanded to include different states across the country to have a larger and more diverse sample to work with. This would allow for greater confidence when generalizing the results to broader populations. Also, the type of residential area could have been controlled for in the sampling to include an amount of rural, suburban, and urban settings that approximates their proportions nationwide. The study could also have looked more in depth at the six components that are central to RTI implementation. These components include assessment, data-based decision making, high quality research-based instruction and intervention, treatment integrity, professional development, and administrative leadership/school-wide supports. The comparison of implementation fidelity of RTI and the results on the Ohio Achievement Assessment could have been organized by the six components in addition to the tier design that was used for this study. It would be beneficial to see if the implementation fidelity of certain components had more of an effect on the outcome of Ohio Achievement Assessment data than other components. Another limitation of this research includes the reliance on self-reported data. The data collected from the surveys was reported by the participants themselves and was not followed with an observation by the researchers due to time limitations. A future observational study could use the RTIS-R to assess each school s RTI implementation level. This would allow the researchers to compare and contrast the data found in observation and the self-reported data from the participants. Also, the participants who completed the RTIS-R may be more invested and interested in the RTI process. This limitation may have had an effect on the results that were found. Reading is just one content area that can be assessed within an RTI framework. Frequently, reading is the content area that schools start implementing and focusing on 14

most intently. There needs to be a larger focus on multiple content areas as opposed to a focus on one specific content area within an RTI framework (Allsopp, 2010). Future studies could focus on other areas such as behavior or mathematics. A revised scale would have to have to be made to correspond with the content area in question. Also, different OAA data would need to be examined to look at different academic content areas. If behavior is chosen as the area of focus, office referrals or behavioral progress monitoring data could be used to assess change. Increasing numbers of states are adopting the common core standards. It is unknown as to how implementation of these common core standards will affect the implementation of the RTI tiers in schools. In future studies, researchers could look at how RTI is being implemented in the schools when the common core standards are being rigorously applied in each classroom (www.corestandards.org). Finally, due to the research design, it is possible that the relationships found in this study are not causal. For example, rather than implementation causing the improved reading outcomes, an alternative explanation for the findings could be a factor in the improvement. One possibility is that schools with higher performing students in the area of reading have more time or resources to devote to RTI implementation. This study attempted to minimize this concern by controlling for school demographic factors, but it remains a limitation that should be addressed with future research utilizing quasiexperimental or experimental designs. Implications for Practice Despite the limitations of this study, there are some helpful results that can be taken into consideration when schools are planning educational curricula and programs. The implications provided below were derived from the highest indicators of implementation on the RTIS-R which were developed from what the literature says about best practices. Schools should aspire to implement these suggestions in the future. The significant effect of Tier 2 supports on OAA data imply that these supports will be beneficial to have in place for all schools. After verifying that an effective, high quality Tier 1 curriculum is in place, one way to start the process of improving Tier 2 would be to progress monitor all students that are receiving Tier 2 supports for reading. This 15

progress monitoring should be done at least twice monthly in at least one identified area of need. To complete the progress monitoring, measures should be used that meet psychometric standards. These measures were selected by comparing the psychometric properties of various progress monitoring measures using reputable sources (e.g., National Center for Response to Intervention Progress Monitoring Tools Chart). Another important aspect of Tier 2 supports is ensuring that all data collectors are accurately trained on administering and scoring the assessments used for progress monitoring. These staff members should undergo administration integrity checks annually to ensure reliable administration and scoring. They should also receive training to refresh their knowledge on the assessment administration at least annually. Several high quality professional development opportunities should be made available to all staff for strengthening Tier 2 supports. These opportunities may include mentoring, modeling, coaching, and guided practice. The staff members should also be given resources to pursue professional development options concerning the RTI interventions. The data collected should be analyzed by teachers and/or grade-level teams at least monthly and should be used to provide children with the appropriate Tier 2 interventions. These decisions should be made with an adequate amount of data. The teams should aim for collecting six to eight weekly progress monitoring data points when making a decision. During this time, direct and indirect treatment integrity data should be collected and used in all grade levels. These data can be presented in the form of observations or teacher report data. The schools should have cut-off scores or decision rules that are documented and well known by the staff. These decision rules should be used to guide placement decisions regarding Tier 3 services as well as decisions about moving students back to Tier 1. These decisions rules should be empirically supported and reviewed after each screening to determine if they are still appropriate and valid. The schools should conduct a review of the Tier 2 curriculum and only continue using it if they find strong, consistent, and overwhelming research to support its effectiveness. To do this, multiple reputable sources should be used, including but not limited to, What Works Clearinghouse, National Center for Response to Intervention, and the Florida Center for Reading Research. 16

Tier 2 intervention time should be provided in addition to the Tier 1 instruction. The intervention groups in Tier 2 should be homogeneous skill-based groups that are made up of less than 4 students. The instruction in Tier 2 should be highly explicit and the staff should use scaffolding to enhance the students learning. The Tier 2 curriculum should focus on a targeted set of reading skills, and should provide frequent opportunities for responding. Above all, the content should be matched to the students needs for best results. Finally, this study suggests that schools may want to consider monitoring RTI implementation regularly. If RTI implementation quality is linked to student outcomes, schools may benefit from analyzing their implementation using the RTIS-R or a similar measure at least annually. Results from this assessment can be used to execute planning for RTI implementation and to monitor progress. In addition, schools can examine the relationships between implementation and outcomes using both standardized and curriculum-based achievement measures. 17

References Allsopp, D. H. (2010). Realizing the potential: How comprehensively are schools implementing RTI? RTI Leadership Forum. Ardoin, S. P., Witt, J. C., Connell, J. E., & Koenig, J. L. (2005). Application of a threetiered response to intervention model for instructional planning, decision making, and the identification of children in need of services. Journal of Psychoeducational Assessment, 23, 362-380. Artiles, A.J., & Kozleski, E.B. (2010). What counts as response and intervention in RTI? A sociocultural analysis. Psichothema, 22, 949-954. Barnett, D. W., Daly, E. J., III., Jones, K. M., & Lentz, F. E., Jr. (2004). Response to intervention: Empirically based special service decisions from single-case designs of increasing and decreasing intensity. The Journal of Special Education, 38(2), 66-79. Batsche, G., Elliott, J., Graden, J.L., Grimes, J., Kovaleski, J.F., Prasse, D., et al. (2006). Response to Intervention: Policy Considerations and Implementation. National Association of State Directors of Special Education Inc: Alexandria, VA. Brace, N., Kemp, R., Snelgar, R. (2006). SPSS for Psychologist (Chapter 7, pp. 206-220). Mahwah, NJ: Lawrence Earlbaum Associates, Inc. Brown-Chidsey, R. (2007, October). No more waiting to fail. Educational Leadership, 65(2), 40-46. Brown-Chidsey, R., Bronaugh, L., & McGraw, K. (2009). RTI in the classroom: guidelines and recipes for success. New York, NY: Guilford Press. Brown-Chidsey, R. & Steege, M.W. (2005). Response to intervention: Principles and strategies for effective practice. New York: Guilford Press. Buffum, A., Mattos, M., & Weber, C. (2010, October). The why behind RTI. Educational Leadership, 68 (2), 10-16. Burns, M. K., Appleton, J. J., & Stehouwer, J. D. (2005). Meta-analytic review of responsiveness-to-intervention research: Examining field-based and researchimplemented models. Journal of Psychoeducational Assessment, 23, 381-394. Burns, M. K., & Gibbons, K. A. (2008). Implementing response-to-intervention in York, 18

elementary and secondary schools: Procedures to assure scientific-based practices. New York, NY: Routledge. Bursuck, B., & Blanks, B. (2010). Evidence based early reading practices within a response to intervention system. Psychology in the Schools, 47(5), 421-431. Byrd, E. S. (2011). Educating and involving parents in the response to intervention process: The school's important role. Teaching Exceptional Children, 43(3), 32-39. Cummings, K. D., Adkins, T., Allison, R., & Cole, C. (2008). Response to intervention: Investigating the new role of special educators. Teaching Exceptional Children, 40(4), 24-31. Delaware Department of Education. Delaware student testing program on-line reports: Statewide summary Math. 2007. Retrieved September 12, 2007, from http://dstp.doe.k12.de.us/dstpmart9. Datnow, A., Borman, G., & Stringfield, S. (2000). School reform through a highly specified curriculum: Implementation and effects of the core knowledge sequence. The Elementary School Journal, 101 (2), 167-191. Dunn, M. (2010). Response to intervention and reading difficulties: A conceptual model that includes reading recovery. Learning Disabilities: A Contemporary Journal, 8(1), 21-40. Flaum, S. K. (2009). Application of the regression discontinuity to the response to intervention (RTI) model of service delivery for determining the effects of early intervention in reading (Unpublished doctoral dissertation). The Ohio State University, Columbus, OH. Fletcher, J. M., & Vaughn, S. (2009). Response to intervention: Preventing and remediating academic difficulties. Child Development Perspectives, 3(1), 30-47. Foorman, B. R., & Torgesen, J. (2001). Critical elements of classroom and small-group instruction promote reading success in all children. Learning Disabilities Research and Practice, 16(4), 203-212. Fuchs, D., Mock, D., Morgan, P. L., & Young, C. L. (2003). Responsiveness-tointervention: Definitions, evidence and implications for the learning disabilities construct. Learning Disabilities Research & Practice, 18(3), 157-171. 19

Gersten, R., Compton, D., Connor, C. M., Dimino, J., Santoro, L., Linan-Thompson, S., et al. (2009). Assisting students struggling with reading: Response to intervention and multi-tier intervention in primary grades. Washington, DC: U.S. Department of Education Institute of Educational Sciences. Gettinger, M., & Stoiber, K. (2007). Applying a response-to-intervention model for early literacy development in low-income children. Topics in Early Childhood Special Education, 27(4), 198-213. Glover, T.A. (2010). Supporting all students: The promise of response to intervention. In T.A. Glover & S. Vaughn (Eds.), The Promise of Response to Intervention: Evaluating Current Science and Practice. New York: The Guilford Press. Goss, S., Noltemeyer, A., & Devore, H. (2007). Treatment integrity: A necessary component of response-to-intervention. The School Psychologist, 61 (2), 34-38. Hamre, B. K., Justice, L. M., Pianta, R. C., Kilday, C., Sweeny, B., Downer, J. T., & Leach, A. (2010). Implementation fidelity of MyTeachingPartner literacy and language activities: Association with preschoolers language and literacy growth. Early Childhood Research Quarterly, 25, 329-347. Hall, S. J. (2008). Implementing Response to Intervention: A Principal s Guide. Thousand Oaks, CA: Corwin Press. Healy, K., Vanderwood, M., & Edelston, D. (2005) Early Literacy Interventions for English Language Learners: Support for an RTI model. The California School Psychologist, 10, 55-63. Implementation Evaluation Tool Manual. (2007). Retrieved on October 18, 2008 from http://www.hcesc.org/resources/ohio Integrated Systems/IETConsumables.pdf. Individuals with Disabilities Education Act of 2004 (IDEA). (2004). Public Law 108-446. Individuals with Disabilities Education Act of 2004 (IDEA). (2004). Public Law 108-446. Johnson, E. S., & Smith, L. (2008). Implementation of response to intervention at middle school: Challenges and potential benefits. Teaching Exceptional Children, 40(3), 46-52. Kovaleski, J.F., & Pederson, J.A. (2008) Best practices in data-analysis teaming. Best 20

Practices in School Psychology V, 2, 115-129. Lemons, C. J., & Fuchs, D. (2010). Modeling response to reading intervention in children with down syndrome: An examination of predictors of differential growth. Reading Research Quarterly, 45(2), 134-168. Linan-Thompson, S., & Ortiz, A. A. (2009). Response to intervention and Englishlanguage-learners: Instructional and assessment considerations. Seminars in Speech and Language, 30(2), 105-119. Mellard, D.F., & Johnson, G. (2008). RTI: A Practitioner s Guide to Implementing Response to Intervention. Thousand Oaks, CA: Corwin Press. National Association of State Directors of Special Education. (2005). Response to Intervention: Policy considerations and implementation. Alexandria, VA: National Association of State Directors of Special Education. National Institute of Child Health and Human Development.(2000). Report of the National Reading Panel. Teaching children to read: An evidence-based assessment of the scientific research literature on reading and its implications for reading instruction (NIH Publication No. 00-4769). Washington, DC: U.S. Government Printing Office. No Child Left Behind Act (NCLB). (2001). Public Law 107-110. Noltemeyer, A. (2012). Addressing the achievement gap with response to intervention. In A. Noltemeyer & C. S. Mcloughlin (Eds.) Disproportionality in education and special education: A guide to creating more equitable learning environments. Springfield, IL: Charles C. Thomas Publisher, Ltd. Noltemeyer, A., & Sansosti, F.J. (2012). Tiered models of integrated academic and behavioral support: Effect of implementation level on academic outcomes. Contemporary School Psychology, 16, 117-127. Noltemeyer, A., Boone, W.J., & Sansosti, F.J. (under review). Assessing systems level RTI implementation for reading: Development and piloting of the RTIS-R.. Nunnery, J. A., Ross, S. M., & McDonald, A. (2006). A randomized experimental evaluation of accelerated reader/reading renaissance implementation on reading achievement in grades 3 to 6. Journal of Education for Students Placed at Risk, 11(1), 1-18. 21

Pullen, P.C., & Justice, L.M. (2003). Enhancing phonological awareness, print awareness, and oral language skills in preschool children. Intervention in School and Clinic, 39(2), 87-98. Rinaldi, C., & Samson, J. (2008). English language learners and response to intervention: Referral Considerations. Teaching Exceptional Children, 40(5), 6-14. Shinn, M. R. (2007). Identifying students at risk, monitoring performance, and determining eligibility within response to intervention: Research on educational need and benefit from academic intervention. School Psychology Review, 36(4), 601-617. Teale, W. H., Paciga, K. A., & Hoffman, J. L. (2007). Beginning reading instruction in urban schools: The curriculum gap insures a continuing achievement gap. Reading Teacher, 61, 344 348 Thomas, A., & Grimes, J. (2008). Best practices in school psychology V (Vol. 4). Bethesda, MD: National Association of School Psychologists. Torgesen, J. K. (2009). The response to intervention instructional model: Some outcomes from a large-scale implementation in reading first schools. Child Development Perspectives, 3(1), 38-40. United States Department of Education. (2004). Comprehensive School Reform Program. Retrieved September 8, 2008 from http://www.ed.gov/programs/compreform/2pager.html. Urban Institute. (2007). Baltimore city s high school reform initiative: Schools, students, and outcomes. Washington, DC: Urban Institute. (ERIC Document Reproduction Service No. ED499437). VanDerHeyden, A. M., Witt, J. C., & Gilbertson, D. (2007). A multi-year evaluation of the effects of a response to intervention (RTI) model on identification of children for special education. Journal of School Psychology, 45, 225-256. Wickstrom, K. F., Jones, K. M., Lafleur, L. H., & Witt, J. C. (1998). An analysis of treatment integrity in school-based consultation. School Psychology Quarterly, 13, 141-154. Wright, B. D., & Masters, G. N. (1982). Rating Scale Analysis Rasch Measurement. Chicago, IL: MESA Press. 22

Table 1. Characteristics of participating schools Characteristic n M SD State Mean % OAA pass* 53 89.22 5.75 80.73** Enrollment 53 519.19 207.59 n/a % Non-White students 53 17.19 16.41 24 % Economically disadvantaged students 53 37.32 25.55 37.7 % Students with disabilities 53 13.12 4.25 14.6 Number of years implementing RTI 49 3.92 2.39 n/a *percentage of students who scored at or above the Proficient range on the Ohio Achievement Assessment reading test **obtained by averaging the pass rates for each of the participating grade levels 23