An Investigation of Gender, Income, and Special Education Status Bias on Curriculum-Based Measurement Slope in Reading

Transcription

1 School Psychology Quarterly 2011 American Psychological Association 2011, Vol. 26, No. 2, /11/$12.00 DOI: /a An Investigation of Gender, Income, and Special Education Status Bias on Curriculum-Based Measurement Slope in Reading Seungsoo Yeo Inje University Jamie Fearrington Appalachian State University Theodore J. Christ University of Minnesota This study investigated slope bias on student background variables for both Curriculum Based Measurement of Oral Reading (CBM-R) and Curriculum Based Measurement Maze Reading (Maze). Benchmark scores from 1,738 students in Grades 3 through 8 were used to examine potential slope bias in CBM-R and Maze. Latent growth modeling was used to both estimate growth rates and examine the extent to which demographic variables affected the estimated growth rates. Results indicate a significant CBM-R slope bias on special education status at Grade 3 and on gender at Grade 7. For Maze, slope bias on gender was associated with Maze slope estimates at Grades 5 and 7. Slope bias on various demographic variables was not consistent across CBM measures and grades. Results and implications are discussed. Keywords: curriculum based measurement, oral reading, maze, slope bias This article was published Online First April 18, Seungsoo Yeo, Department of Special Education, Inje University; Jamie Fearrington, Department of Psychology, Appalachian State University; and Theodore J. Christ, Department of Educational Psychology, University of Minnesota. Correspondence concerning this article should be addressed to Seungsoo Yeo, Department of Special Education, Inje University, 607 Obang-dong, Kimhae, Kyungnam , South Korea. yeoxx008@inje.ac.kr Curriculum Based Measurement (CBM) is a set of standardized procedures that were initially designed to index the level and rate of student achievement within the basic skill areas of reading, mathematics, written expression, and spelling (Deno, 1985, Deno, 2003). CBM has now expanded to include measures of early literacy (Good & Kaminski, 2002) and early numeracy (Clarke & Shinn, 2004). Extensive research and substantial evidence provide support for the reliability and validity of CBM for a variety of applications, which include screening, benchmarking, and progress monitoring (Jiban & Deno, 2007; Fuchs, 2004; Marston, 1989; Wayman, Wallace, Wiley, Ticha, & Espin, 2007; Yeo, 2010). One distinctive feature of CBM that distinguishes it from other measures of academic achievement is its utility. That is, CBM is quick, simple, and repeatable. These characteristics allow teachers to measure student skills on a frequent basis, which might be weekly, biweekly, monthly, or triannually. Frequent and repeated administrations over time are useful to estimate the rate of student achievement over relatively brief periods of instruction or intervention. In the area of reading, two types of CBM measures have been used in research and practice: reading aloud and maze selection. On the CBM-R (reading aloud) measure, student performance is measured by requiring students to read aloud passages of meaningful text for 1 min. The number of words read correctly is scored as the reading rate (Deno, 1985). On CBM-Maze tasks, student performance is measured by having students read silently from a passage where every seventh word has been deleted and replaced with three response choices inside parentheses: one is the correct word and the other two are distracters. The number of correct selections is scored. Recently, CBM-Maze has been receiving more attention due to the fact that it can be administered to a group of students at one time, whereas CBM-R is individually administered. 119

2 120 YEO, FEARRINGTON, AND CHRIST Importance and Relevance of CBM in Relation to Response to Intervention Recent educational reform policies have placed an increased importance on accountability and standards (No Child Left Behind Act, 2002), as well as use of response to intervention (RTI) as a means to identify children with specific learning disabilities (SLDs). The recently reauthorized Individuals with Disabilities Education Improvement Act (2004) stipulates that schools may choose to use RTI to guide special education eligibility decisions. This shift in federal policy highlights the need for assessments that are useful to index both the level and rate of student achievement through a continuous process of screening/benchmarking and progress monitoring. In addition, RTI requires that educators track student progress over time as they are exposed to varying degrees of instructional intervention (Batsche et al., 2005; Vaughn & Fuchs, 2003). In most RTI models, the majority of students receive instruction in Tier 1, which consists of classroom instruction within the general education curriculum. Tier-2 instruction is delivered to students who are identified as at-risk through universal screenings of academic skills with CBM. Tier 2 relies on small-group instruction with scientifically based programs coupled with frequent progress monitoring. Students that do not respond to interventions delivered at Tier 2 may be moved to Tier 3, which is the most intensive and typically consists of individual instruction. CBM is the assessment tool of choice for many RTI models, and many school districts use CBM as part of their RTI model. CBM is used as a universal screening device to identify those children who need more intensive instruction (Tier 1), to evaluate instructional effects (Tiers 2 and 3), and to make special education eligibility decisions, which include the diagnosis of SLD. There are various commercial forms of CBM that are available for purchase by school districts who are implementing RTI. AIMSweb is one such commercial benchmarking- and progress-monitoring system (retrieved from AIMSweb provides its users with CBM measures in the academic areas of early literacy, early numeracy, reading, reading comprehension, math computation, math reasoning, and writing. AIMSweb users also have access to web-based data management that enables educators to organize and report outcomes for students at all three RTI tiers. Slope on CBM Measures Within a RTI Model Within an RTI framework, estimates of slope from frequent assessment are more crucial than estimates of level because slope is a key element for examining whether students are properly responding to a particular set of instructional conditions or interventions. Moreover, CBM is described in the literature as uniquely suited for repeated, ongoing assessment, which functions to establish time-series progressmonitoring data that are useful to evaluate growth (Deno, 1985, 1986, 2003). For example, progress monitoring at Tier 1 estimates the growth rate for all students in the general education classroom. If a student s slope or rate of growth is sufficiently below that of peers in Tier 1, the student moves to Tier 2 for an intervention that is more intensive. Those students who respond to Tier 2 intervention are soon served again within Tier 1, but those students who do not respond are later served within Tier 3 (Vaughn & Fuchs, 2003). The slope estimates derived from CBM data provide the evidence of sufficient or insufficient response that determines continuation or discontinuation of services at Tiers 1, 2, or 3. Slope estimates within and across tiers also provide the evidence to support special education eligibility decisions and SLD identification. As a result, the trustworthiness of the growth data used in a RTI decision-making framework is an important consideration. Group Differences on CBM Slope Given the importance of assessing growth rate within a RTI model, it is important to examine whether CBM slopes are consistent among a variety of groups, including gender, ethnicity, special education status, and socioeconomic status (SES). In other words, estimates of growth derived from CBM data might systematically increase or decrease as a function of group membership or demographic characteristics (Betts et al., 2008). Systematic over- or

3 AN INVESTIGATION OF BIAS AND CBM SLOPE 121 underestimation might function as a source of bias and thereby bias school-based decision making. This is an important concern and a subject worthy of examination. Until recently, many of the early CBM studies examined the extent to which scores at a single time point equally predict concurrent or future performance on criterion measures, which include statewide tests and assessment batteries, across groups and demographic characteristics. For example, a study by Hintze, Callahan, Matthews, Williams, and Tobin (2002) investigated ethnic bias on CBM-R across African American and Caucasian students by controlling for age, gender, and SES. This study found no predictive bias in the use of CBM-R to predict reading comprehension scores on the Woodcock-Johnson Psychoeducational Battery, Revised, meaning that the CBM score is not a biased predictor of future reading performance for either group of students. In fact, conducting research on predictive bias for CBM measures is meaningful to provide evidence that CBM can be broadly used for various groups. The RTI context establishes that it is important to examine the extent to which the estimates of growth based on CBM data are consistent and nonbiased across demographics and group memberships; however, relatively few studies have examined the issue. A review of the research yielded only one study that examined potential slope bias in CBM-R and Maze. Using hierarchical linear modeling (HLM), Shin, Deno, and Espin (2000) evaluated the technical features of the CBM maze task, including reliability, sensitivity, and validity, for measuring growth rates. In this study, group differences in slopes were examined between general and special education students. Results indicated that there was no significant difference in slope between the two groups; however, that study did not include other characteristics, such as gender, SES, and ethnicity. In addition, the analysis in that study did not control for the initial status (i.e., achievement level). Given the possibility that slope estimates might be either positively or negatively related to initial status, it seems necessary to control the initial level of performance. With regard to research examining CBM-R slope bias on group differences, recently, a longitudinal study by Chard et al. (2008) examined the effect of demographic variables, such as ethnicity, gender, and language, on a standardized achievement test in reading. Participants were 668 kindergarten and 1st graders who were identified as in need of intensive intervention in reading through Grade 3. Results revealed that none of the demographic variables included in the analyses were significant predictors of CBM slope. Although CBM growth rate in this study was evaluated in the context of initial status, this study did not examine whether or not demographic variables continued to be insignificant predictors across increasing grade levels. Given that CBM measures are often viewed as a seamless system of measuring students growth rate (Wayman et al., 2007), it would be necessary to investigate whether these demographic variables were significant predictors of CBM growth rate for students from elementary to middle school levels. Aims of the Current Study There is a paucity of research and evidence to support conclusions about potential slope bias in CBM-R and Maze; moreover, there are limitations inherent in the analyses of the research published in the literature. The first aim of this study was to examine whether estimates of slope from CBM data are likely to be an unbiased index of growth across groups and demographic characteristics such as gender, ethnicity, SES, and special education status. As an improvement upon prior analyses, the analytic design for this study controlled for the initial status of achievement by using latent growth modeling (LGM). The second aim of this study was to examine whether estimates of slope from CBM data are differentially biased across grades. The third, and final, aim of this study was to examine whether slope bias on various subgroups is consistent between CBM-R and Maze. Although both CBM-R and Maze are common progress-monitoring measures in reading, research has yet to examine whether slope bias on the demographic variables changes as a function of the differences between CBM-R and CBM-Maze. Therefore, it is important to investigate whether slope bias on the various variables is consistent between the two measures.

4 122 YEO, FEARRINGTON, AND CHRIST Method Participants and Setting The study took place in a rural school district located in the Southeastern United States. Students (N 1,738) in Grades 3 through 8 attending two elementary schools, and three middle schools participated in this study. The sample included 850 females (49.1%) and 880 males (50.9%) enrolled in both general and special education. The percentage of students that received special education services was 14.6%. The sample was predominantly Caucasian (94.5%). The ethnic breakdown for the remainder of the sample was 3.2% Hispanic, 1.4% African American, 0.5% Asian, and 0.2% Native American. Further, 1.33% of the students were classified as English as Second Language (ESL). Approximately half of the students (55.4%) were eligible to receive free or reduced-price lunches. Procedures All assessments used in this study were part of systematic universal screenings that were conducted three times a year by the school district. CBM-R and Maze data that were collected during the 2006 to 2007 school year were used in this study. CBM-R and Maze probes from AIMSweb were administered by a benchmark team that consisted of school psychologists, teaching assistants, and graduate students from a nearby school psychology doctoral training program. All persons on the benchmark team attended training sessions prior to testing. Assessments were conducted in a quiet area of the classroom or in the hallway outside the classroom. Each CBM-R benchmark score represented the median score from a series of three grade-appropriate passages. Selecting the median score from three probes is the commonly used method in practice and research on CBM (Yeo, 2008). The CBM-R passages were administered on an individual basis. Maze passages were administered to the entire classroom simultaneously and took approximately 5 min to administer. Each Maze passage was scored by the administration team directly after administration. Both scores were then entered into a database. Measurements CBM-R scores reflect the number of words correctly read out loud in 1 min (Shinn & Shinn, 2002). Over two decades of research have established the technical adequacy of CBM-R (Wayman et al., 2007). CBM-R is typically considered to be the AIMSweb reading measure with the most psychometric support (see Marston, 1989; Shinn, Good, Knutson, Tilly, & Collins, 1992). Test-retest and interrater reliability coefficients range from.82 to.99, respectively (Marston, 1989). Concurrent validity with other standardized reading achievement tests is also well established, with coefficients ranging from.58 to.86 (Jenkins & Jewell, 1993). To specifically measure reading comprehension, AIMSweb also provides Maze-reading probes. Maze requires students to complete a multiple-choice cloze task while they read a 150- to 400-word passage silently. The first sentence of each passage is complete. After, every seventh word is replaced by three plausible selections, and the student must circle the word that correctly completes the sentence. The Maze score represents the number of words circled correctly during a 3-min time period (Shinn & Shinn, 2002). Maze also has satisfactory evidence of technical adequacy. Criterionrelated validity with CBM-R is strong, with coefficients ranging from.77 to.86 (Espin, Deno, Maruyama, & Cohen, 1989). The concurrent validity of Maze is also well established with group-administered standardized tests of reading achievement (Jenkins & Jewell, 1993). Results Descriptive statistics for all study variables are presented in Table 1. Data are depicted as a proportion for all demographic characteristics and as a mean of CBM scores for each point in time. It seems that though the mean level of CBM-R scores increased over time for all grade levels, the mean level of Maze scores increased for only Grades 3, 5, and 7. Most students in all grade levels belonged to the Caucasian ethnic group. For this reason, the ethnic group was not used as a potential covariate in this study. Before proceeding with the substantive analyses, all study variables were screened for the potential problems of skewness and

5 AN INVESTIGATION OF BIAS AND CBM SLOPE 123 Table 1 Descriptive Statistics for All Variables Variables Grade 3rd 4th 5th 6th 7th 8th Mean or proportion (SD) Mean or proportion (SD) Mean or proportion (SD) Mean or proportion (SD) Mean or proportion (SD) Mean or proportion (SD) CBM-R a Fall (35.51) (34.43) (45.99) (45.57) (42.44) (42.13) Winter (40.54) (36.01) (48.31) (43.51) (42.62) (40.69) Spring (41.50) (38.68) (47.07) (43.62) (40.44) (40.51) Maze b Fall (5.67) (4.83) (7.05) (8.13) (7.85) (7.13) Winter (5.42) (5.92) (8.37) (8.38) (9.75) (7.13) Spring (6.01) (6.24) (9.23) (9.52) (8.97) (9.14) Caucasian.95 (.23).95 (.21).94 (.23).93 (.26).94 (.23).95 (.21) Female.48 (.50).51 (.50).50 (.50).49 (.50).48 (.50).48 (.50) Free lunch.43 (.50).46 (.50).41 (.49).44 (.50).44 (.50).49 (.50) Special education.13 (.34).13 (.34).13 (34).19 (.39).12 (.23).15 (.36) Note. Missing data for each grade ranged from 10.2% to 13.2%. a Results reported as words read correctly per minute. b Results reported as words selected correctly per 3 min. kurtosis. The skewness and kurtosis coefficients of CBM measures varied (.19 for CBM-R scores to 1.29 for Maze scores), indicating that all CBM scores approximated a normal distribution. Testing Shape of Growth In the first step, latent growth modeling (LGM) techniques were conducted with the AMOS 4.0 program, using full information maximum likelihood (FIML), which provides efficient estimates from data with missing values (Ming, Conger, & Lorenz, 2005). An alpha level of.05 was set for all statistical tests. To determine the developmental growth trajectories, the following two models were initially fit to CBM measures. The first model, a no growth model, provided estimates that included initial status but not growth rate. Second, a linear growth model with both initial status and linear growth trajectory was fitted for change in CBM measures over time. The loadings for the slope factors were specified as 0, 5, and 9 (August 0, January 5, and May 9) over three time points. This set of factor loading allows for representing a liner growth rate. Especially, the first factor loading on the slope factor (August) was coded as zero so that the intercept is directly interpreted as the starting point of the growth trajectory (Bollen & Curran, 2006). Although it would be appropriate to model quadratic or cubic parameters before selecting a linear model, with only three data points per grade, nonlinearity of slopes could not be evaluated adequately as a potential growth model (Fitzmaurice, Laird, & Ware, 2004). As shown in Tables 2 and 3, the chi-square difference tests ( 2 ) show that the linear growth model fit the data best for CBM-R and Maze scores. In addition, the linear growth models were evaluated by using RMSEA (root mean square error of approximation; good model.08) and CFI (comparative fit index; good model.95; Hu & Bentler, 1999). Our findings indicated that, overall, the linear growth models are considered acceptable for the data from the CBM-R and Maze scores for all grade levels. Testing Variance in Slope and Intercept Tables 2 and 3 also provide information on whether there is significant variability in the linear-slope and intercept estimates for the CBM-R and Maze scores. If significant variance in the slope exists, it is necessary to add predictors that explain the significant variance. In contrast, if there is no significant variance in the slope, no further test is required, indicating that the CBM

6 Table 2 Fit Indices and Unstandardized Parameter Estimates for CBM-R RMSEA a Slope Intercept Model label 2 df (90%CI) CFI b 2 df p (d) M (SE) Var M (SE) Var Grade 3 No growth (.63.74) (2.90) 1, Linear growth (.00.11) (.12) (2.47) 1, Conditional model (.03.14) (.58) (11.01) Grade 4 No growth (.58.68) (2.39) Linear growth (.00.15) (.11) (2.21) Conditional model Grade 5 No growth (.51.60) (2.72) 2, Linear growth (.00.02) (.09) (2.64) 2, Conditional model 6 Grade 6 No growth (.38.48) (2.68) 1, Linear growth (.00.13) (.11) (2.73) 1, Conditional model Grade 7 No growth (.40.50) (2.44) 1, Linear growth (.00.15) (.10) (2.34) 1, Conditional model (.00.09) (.40) (8.23) Grade 8 No growth (.40.49) (2.35) 1, Linear growth (.00.02) (.10) (2.35) 1, Conditional model Note. CBM-R results reported as words read correctly per minute. 2 chi-square; df degree of freedom; RMSEA root mean square error of approximation; CFI comparative fit index; 2 difference in chi-square tests; df difference in df; p (d) probability of the difference tests. The best fitting models are in italics. a RMSEA.08 indicates good fit. b CFI.95 indicates good fit. p YEO, FEARRINGTON, AND CHRIST

7 Table 3 Fit Indices and Unstandardized Parameter Estimates for Maze Model label 2 df RMSEA CFI 2 df p (d) M (SE) Var M (SE) Var Grade 3 No growth (.16.27) (.35) Linear growth (.00.18) (.03) (.38) Conditional model Grade 4 No growth (.46.56) (.37) Linear growth (.00.19) (.03) (.31) 7.81 Conditional model Grade 5 No growth (.41.50) (.45) Linear growth (.02.16) (.03) (.51) Conditional model (.05.14) (.16) (1.62) Grade 6 No growth (.25.35) (.49) Linear growth (.00.19) (.04) (.54) Conditional model (.09.18) (.17) (1.50) Grade 7 No growth (.25.34) (.46) Linear growth (.00.13) (.04) (.43) Conditional model (.00.07) (.17) (1.60) Grade 8 No growth (.26.36) (.46) Linear growth (.00.19) (.05) (.48) Conditional model (.24.32) (.25) (2.11) Note. Maze results are reported as words selected correctly per 3 min. 2 chi-square; df degree of freedom; RMSEA root mean square error of approximation; CFI comparative fit index; 2 difference in chi-square tests; df difference in df; p (d) probability of the difference tests. The best fitting models are italicized. a RMSEA.08 indicates good fit. b CFI.95 indicates good fit. p.05. Slope Intercept AN INVESTIGATION OF BIAS AND CBM SLOPE 125

8 126 YEO, FEARRINGTON, AND CHRIST slope is consistent among students, regardless of the characteristics of the population (Ming et al., 2005). In other words, significant variance in the slope would suggest that there are significant individual variances in development trajectories for CBM measures that should be explained by additional variables. The results in Tables 2 and 3 show that there are significant variances in the linear slope for CBM-R and Maze scores in the 3rd and 7th grades and in the 5th, 6th, 7th, and 8th grades, respectively. Interestingly, the CBM-R slope is consistent for all grade levels, except for the 3rd and 7th grade levels. This suggests that the CBM-R slope can be used as an unbiased predictor of improved performance in reading in Grades 4, 5, 6, and 8. For only upper grade level students, it appears that the estimated Maze slope is inconsistent among various subgroups of students. Conditional LGM CBM Fall 1 Free Lunch 1 1 CBM Intercept CBM Winter 5 9 Given the substantial variance in the slope coefficient of the CBM-R and Maze scores in the 3rd and 7th grades and in the 5th, 6th, 7th, and 8th grades, respectively, the next step involved conducting conditional linear growth modeling with predictors, such as demographic characteristics, to account for the significant variance in the slopes and intercepts (see Figure 1). Effects of the predictors on the slopes and intercepts are presented in Table 4. For Grade 3, special education status was negatively related to the slope, controlling for the initial status and other demographic variables, meaning that students receiving special education services had lower growth rates than did general education students. In addition, for Grade 7, the CBM slope was affected by gender, controlling for the initial status and other demographic characteristics, meaning that female students are more likely to have a higher slope than male students. For Grade 3, special education status influenced the initial performance on the CBM-R measure, and for Grade 7, significant variance in the intercept estimates was accounted for by all demographic characteristics. With regard to Maze scores in Grades 5 and 7, there was a significant difference in the slope estimates of boys and girls, meaning that female students showed a faster rate of increase in Maze scores than did male students, controlling for the rate of change and other demographic variables. Gender was a significant predictor of the Maze slopes at all grade levels, meaning that female students were more likely to have higher initial Maze scores than male students, controlling for the rate of change and other demographic variables. At all grade levels (except for Grade 6), SES was related to higher initial Maze scores, controlling for the rate of change and other demographic variables, indicating that students with high SES showed higher initial Maze scores than those with lower SES. Finally, students receiving special education services had lower initial Maze scores than those not receiving special education services, controlling for other demographic variables. Comparison of CBM-R and Maze Scores Interestingly, slope bias on various demographic variables is not consistent between CBM-R and Maze scores. For example, although the slope estimate for CBM-R scores in Grade 3 is homogeneous among participants, the slope estimate for Maze scores among Grade 3 participants is heterogeneous. In addition, for 7th grade students, gender was a significant predictor of increased growth rates for CBM-R scores, whereas the gender variable 0 Gender CBM Slope CBM Spring Special Education Figure 1. Estimated latent growth model (LGM) for demographic characteristics.

9 AN INVESTIGATION OF BIAS AND CBM SLOPE 127 Table 4 Unstandardized Parameter Estimates and Standard Errors for Conditional Latent Growth Models CBM-R a Maze b Grade 3 Grade 7 Grade 5 Grade 6 Grade 7 Grade 8 Est SE Est SE Est SE Est SE Est SE Est SE Regression effects on intercept Female3I Free lunch3i Special education3i Regression effects on slope Female3S Free lunch3s Special education3s Note. I intercept; S slope. a Results reported as words read correctly per minute. b Results reported as words selected correctly per 3 min. p.05. was a significant predictor of decreased growth rates for Maze scores. Discussion The focus of the current study was to examine whether the CBM slope is an unbiased indicator of improved performance in reading. The present study extends previous work on bias in CBM measures. As mentioned above, because most previous studies have only highlighted bias in CBM scores at one point in time among subgroups of participants, it was necessary to examine whether the CBM slope estimates are consistent among various characteristics of the population. To our knowledge, this is the first study to examine the potential slope bias of CBM-R and Maze data as a function of demographic variables. This is an important research question, because both CBM-R and Maze are commonly employed to monitor student progress and evaluate instructional effects within RTI models of service delivery. The results of the current study show that there were significant variances in CBM-R slopes in Grades 3 and 7 only, meaning that the growth rate of CBM-R scores from fall to spring is likely to be homogenous among subgroups, except for Grades 3 and 7. Unlike the CBM-R measurement conditions, there were significant variances in the growth rates derived from Maze scores for only the higher grade levels (Grades 5, 6, 7, and 8). This indicates that students from Grades 5 to 8 did not appear to hold a similar growth rate from fall to spring, but that the 3rd and 4th grade readers growth rates were identical from fall to spring. Given that significant variance in CBM-R slopes only existed for Grades 3 and 7, it is evident that the use of the different CBM measures affected the different shape of the growth trajectories of the participants. With regard to student background variables leading to slope bias, those within the CBM-R measurement condition were influenced by group membership with regard to special education and gender. In the case of Maze scores, gender was the only variable that affected slope estimates. Interestingly, for all grade levels (with the exception of Grade 3), special education status was not a significant predictor of slope within the CBM-R measurement conditions. This means that the CBM-R slopes for students receiving special education services are similar to those of general education students. Given that prior published studies (e.g., Deno, Fuchs, Marston, & Shin, 2001) have shown that general education students had larger growth rates than did students with disabilities, it is somewhat unexpected that special education status had no negative impact on the slopes of CBM-R scores in the current sample. One possible explanation for the discrepancy in findings is methodological differences. That is, unlike the previous studies, other demographic variables as well as initial status were controlled for when examining the relationship between special education status and slope. Another reason is that the number of students with disabil-

10 128 YEO, FEARRINGTON, AND CHRIST ities is very small, comparing the number of students without disabilities. Finally, it is possible that the only absolute difference in CBM-R slope estimates between students with and without disabilities existed during the early grades (e.g., Grade 3). As mentioned above, gender was the only significant variable associated with Maze slope bias. Interestingly, although female students in Grade 7 had higher levels of Maze score growth than male students, CBM-R slope estimates were higher for male than female students. That is, depending on the use of different CBM measures, female students in Grade 7 were either positively or negatively associated with growth rate on CBM measures. Why does gender in Grade 7 lead to inconsistent results? One possible explanation for this result involves the AIMSweb reading passages used in this study. That is, it is quite plausible the AIMSweb passages for the Maze test are biased in favor of male students at the Grade 7 level. Such a result implies that slope bias on various demographic variables may be attributable to effects of progress-monitoring measurement materials used for the two measures. Although both CBM-R and Maze reading measures have been viewed as valid, reliable indices of overall reading competence, the two measures pose different technical adequacies in terms of effects of text materials. Thus, future research should examine the possibility that slope bias on student background variables is due, in part, to differences in the use of CBM measures. We were also interested in the extent to which grade level is associated with significant background variables affecting the magnitude of growth rate. Consistent with prior findings by Chard et al. (2008), we found that regardless of grade level, growth rate of CBM-R scores did not vary as a function of SES background variables. This result suggests that the CBM-R slope may be not biased in terms of SES. However, as previously mentioned, gender and special education status varied with grade level. Limitations and Directions for Future Research Although this study provides meaningful insights into the slope bias derived from CBM measures in reading, some key limitations remain. First, in this study, slope bias of demographic variables was examined with universal benchmark screenings (i.e., fall, winter, and spring) rather than frequent CBM progress monitoring that would be administered weekly or biweekly. As such, it is impossible to generalize outcomes from the current study to other CBM research in which it has been used as a means of frequently measuring the progress of students in the problem-solving model (Hintze et al., 2002). The schedule of data collection also precluded evaluation of alternate nonlinear models for which research provides support (Christ, Silberglitt, Yeo, & Cormier, 2010). Replication and extension of this work, especially using more frequent progress monitoring data, is needed. Second, as mentioned above, the current sample was primarily Caucasian (more than 90%). Thus, future research should consider utilizing similar sample sizes for each ethnic/ racial group in order to evaluate the generalizability of the findings from this study. Third, although this study shows that special education status was not a significant predictor of slope within the CBM measures, except for CBM-R measurement at the Grade 3 level, we did not provide information on the special education sample in this study. It is possible that the percentages of students associated with severe or mild disabilities could be one of potential factors affecting the magnitude of growth rate. Fourth, it should be also noted that the nature of the CBM reading passages used in this study might result in very different outcomes. In fact, some studies have reported evidence that rates of growth change as a function of the effects of text material (e.g., Fuchs & Deno, 1992; Hintze & Shapiro, 1997). In this study, we only used the CBM measures produced by AIMSweb. To advance understanding of the effects of curriculum on CBM reading measures, additional research is needed. Such research should directly examine whether the CBM slope bias of demographic variables is dependent upon the effects of text materials, indicating that the generalizability of our findings should be limited to studies with AIMSweb passage sets. Finally, given the fact that CBM has been used as a valid, reliable indicator of global proficiency in basic academic skills, such as math and written expression, it would be meaningful to conduct similar studies focusing on CBM in other academic areas.

11 AN INVESTIGATION OF BIAS AND CBM SLOPE 129 Implications for Practice and Research Despite the limitations described above, the present study has significant implications for practice and research. The first implication is that a student s background variables may be associated with a significant amount of variance in the student s rate of growth. Although a previous study by Chard et al. (2008) showed that student demographic variables were not significant predictors of CBM-R slope, this study provides evidence that special education status and gender lead to biased slope estimates at some grade levels. Given that CBM slope estimates serve as a means of identifying students who are struggling to read with researchbased interventions as part of RTI (Fuchs, Fuchs, & Compton, 2004), practitioners who administer CBM measures within an RTI model should realize that average rates of growth might be significantly influenced by students background variables, regardless of the quality of intervention delivered to students at risk. However, more research is needed to provide firm support for the conclusions of this study. The second implication drawn from our study is that schools and teachers should use multiple progress monitoring measures within an RTI framework. Using a single progress monitoring measure is not appropriate for estimating the mean rate of growth to make high-stakes decisions about which students are eligible for special education services providing more intensive interventions (Ardoin & Christ, 2008). This study suggests that choosing which CBM measures to administer may result in substantial slope bias on demographic variables. For instance, it is possible that using CBM-R slopes as indicators of overall reading proficiency leads to overestimations of growth rates for a specific background variable, such as gender, whereas the use of Maze slopes is associated with underestimations of growth rates. Therefore, we encourage schools and practitioners who use CBM measures in reading to administer multiple progress-monitoring measures. The final implication involves previous research in which the rate of progress was estimated without considering student background factors affecting variance in the estimated slope or controlling for initial level of performance. Surprisingly, most CBM studies have ignored the effects of demographic variables and initial levels of performance. As Baker and his colleagues (2008) mentioned, to correctly evaluate CBM slope estimates, multiple factors resulting in their variance should be controlled before interpreting growth rate. References Ardoin, S. P., & Christ, T. J. (2008). Evaluating curriculum-based measurement slope estimates using data from triannual universal screenings. School Psychology Review, 37, Baker, S. K., Smolkowski, K., Katz, R., Fien, H., Seeley, J. R., Kame enui, E. J., & Beck, C. T. (2008). Reading fluency as a predictor of reading proficiency in low-performing, high-poverty schools. School Psychology Review, 37, Batsche, G., Elliot, J., Graden, J. L., Grimes, J., Kovaleski, J. F., Prasse, D.,... Tilly, W. D., III. (2005). Response to intervention: Policy considerations and implementation. Alexandria, VA: National Association of State Directors of Special Education. Betts, J., Reschley, A., Pickart, M., Heistad, D., Sheran, C., & Marston, D. (2008). An examination of the predictive bias for second grade reading outcomes from measures of early literacy skills in kindergarten with respect to English language learners and ethnic subgroups. School Psychology Quarterly, 23, doi: / Bollen, K. A., & Curran, P. J. (2006). Latent curve models: A structural equation perspective. Hoboken, NJ: Wiley-Interscience. Chard, D. J., Stoolmiller, M., Harn, B. A., Wanzek, J., Vaughn, S., Linan-Thompson, S., & Kame enui, E. J. (2008). Predicting reading success in a multilevel schoolwide reading model: A retrospective analysis. Journal of Learning Disabilities, 41, doi: / Christ, T. J., Silberglitt, B., Yeo, S., & Cormier, D. (2010). Curriculum-Based Measurement of Oral Reading (CBM-R): An evaluation of growth rates and seasonal effects among students served in general and special education. School Psychology Review, 39, Clarke, B., & Shinn, M. R. (2004). A preliminary investigation into the identification and development of early mathematics curriculum-based measurement. School Psychology Review, 33, Deno, S. L. (1985). Curriculum-based measurement: The emerging alternative. Exceptional Children, 52, Deno, S. L. (1986). Formative evaluation of individual student programs: A new role for school

12 130 YEO, FEARRINGTON, AND CHRIST psychologists. School Psychology Review, 15, Deno, S. L. (2003). Developments in curriculumbased measurement. Journal of Special Education, 37, Deno, S. L., Fuchs, L. S., Marston, D., & Shin, J. (2001). Using curriculum-based measurement to establish growth standards for students with learning disabilities. School Psychology Review, 30, Espin, C. A., Deno, S. L., Maruyama, G., & Cohen, C. (1989, April). The basic academic skills samples: An instrument for screening and identifying children at risk for failure in the regular education classroom. Paper presented at the American Educational Research Association Meeting, San Francisco. Fitzmaurice, G. M., Laird, N. M., & Ware, J. H. (2004). Applied longitudinal analysis. Hoboken, NJ: Wiley-Interscience. Fuchs, L. S. (2004). The past, present, and future of curriculum-based measurement research. School Psychology Review, 33, Fuchs, L. S., & Deno, S. L. (1992). Effects of curriculum within curriculum-based measurement. Exceptional Children, 58, Fuchs, L. S., Fuchs, D., & Compton, D. L. (2004). Monitoring early reading development in first grade: Word identification fluency versus nonsense word fluency. Exceptional Children, 71, Good, R. H., & Kaminski, R. A. (Eds.). (2002). Dynamic indicators of basic early literacy skills (6th ed.). Eugene, OR: Institute for the Development of Educational Achievement. Retrieved from Hintze, J. M., Callahan, J. E., Matthews, W. J., Williams, S. A., & Tobin, K. G. (2002). Oral reading fluency and prediction of reading comprehension in African American and Caucasian elementary school children. School Psychology Review, 31, Hintze, J. M., & Shapiro, E. S. (1997). Curriculumbased measurement and literature-based reading: Is curriculum-based measurement meeting the needs of changing reading curricula? Journal of School Psychology, 35, Hu, L.-T., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus. Structural Equation Modeling, 6, Individuals with Disabilities Education Improvement Act, Pub. L. No , 118 Stat (2004). Jenkins, J. R., & Jewell, M. (1993). Examining the validity of two measures for formative teaching: Reading aloud and maze. Exceptional Children, 59, Jiban, C., & Deno, S. L. (2007). Using math and reading curriculum-based measurements to predict state mathematics test performance: Are simple one-minute measures technically adequate? Assessment for Effective Intervention, 32, Marston, D. B. (1989). A curriculum-based measurement approach to assessing academic performance: What it is and why do it. In M. R. Shinn (Ed.), Curriculum-based measurement: Assessing special children (pp ). New York: Guilford Press. Ming, C., Conger, R. D., & Lorenz, F. O. (2005). Predicting change in adolescent adjustment from change in marital problems. Developmental Psychology, 41, No Child Left Behind Act, Pub. L. No , 115 Stat (2002). Shin, J., Deno, S. L., & Espin, C. (2000). Technical adequacy of the maze task for curriculum-based measurement of reading growth. Journal of Special Education, 34, Shinn, M. R., Good, R. H., Knutson, N., Tilly, W. D., & Collins, V. L. (1992). Curriculum-based measurement of oral reading fluency: A confirmatory analysis of its relation to reading. School Psychology Review, 21, Shinn, M. R., & Shinn, M. M. (2002). AIMSweb training workbook. Eden Prairie, MN: EdInformation, Inc. Vaughn, S., & Fuchs, L. S. (2003). Redefining learning disabilities as inadequate response to instruction: The promise and potential problems: Learning Disabilities Research & Practice, 18, Wayman, M. M., Wallace, T., Wiley, H. I., Ticha, R., & Espin, C. A. (2007). Literature synthesis on curriculum-based measurement in reading. Journal of Special Education, 41, Yeo, S. (2008). Relation between 1-minute CBM reading aloud measure and reading comprehension tests: A multilevel meta-analysis (Unpublished doctoral dissertation). University of Minnesota, Minneapolis and St. Paul. Yeo, S. (2010). Predicting performance on state achievement tests using curriculum-based measurement in reading: A multilevel meta-analysis. Remedial and Special Education, 31, doi: /