Laurie E. Cutting Kennedy Krieger Institute, Johns Hopkins School of Medicine, Johns Hopkins University, and Haskins Laboratories

Transcription

1 SCIENTIFIC STUDIES OF READING, 10(3), Copyright 2006, Lawrence Erlbaum Associates, Inc. Prediction of Reading Comprehension: Relative Contributions of Word Recognition, Language Proficiency, and Other Cognitive Skills Can Depend on How Comprehension Is Measured Laurie E. Cutting Kennedy Krieger Institute, Johns Hopkins School of Medicine, Johns Hopkins University, and Haskins Laboratories Hollis S. Scarborough Kennedy Krieger Institute and Haskins Laboratories Reading comprehension scores from the Wechsler Individual Achievement Tests, the Gates MacGinitie Reading Test, and the Gray Oral Reading Test were examined in relation to measures of reading, language, and other cognitive skills that have been hypothesized to contribute to comprehension and account for comprehension differences. In a sample of 97 first through tenth graders, the relative contributions of word recognition/decoding and oral language skills to comprehension varied from test to test. The inclusion of reading speed accounted for additional variance, but prediction of comprehension scores was minimally improved by including measures of rapid serial naming, verbal memory, IQ, or attention. The findings suggest that commonly used tests of reading comprehension, such as the three we compared, may not tap the same array of cognitive processes. Implications for research and practice are discussed. Children s difficulties in reading comprehension are increasingly a focus of interest in both research and practice. In this study, we examined the contributions of Correspondence should be sent to Laurie E. Cutting, Kennedy Krieger Institute, 707 North Broadway, Suite 232, Baltimore, MD cutting@kennedykrieger.org

2 278 CUTTING AND SCARBOROUGH cognitive and linguistic skills to the prediction of reading comprehension. Because we were concerned that the relative importance of these predictors might depend on how one chooses to measure comprehension, we simultaneously conducted our analyses for three different reading comprehension instruments. FOUNDATIONS OF READING COMPREHENSION According to Gough and Tunmer s (1986) influential simple model, reading comprehension is hypothesized to be the product of just two necessary factors: decoding and listening comprehension. That is, successful understanding of text requires accurate bottom-up identification of the printed words and linguistically proficient top-down analyses of the semantic and syntactic relationships among the words to attain an understanding of the text s meaning. When bottom-up skills are weak and effortful, comprehension will likely be impeded because words are misidentified and because fewer cognitive resources can be devoted to the processing of meaning (Adams, 1990; LaBerge & Samuels, 1974; Lyon, 1995; Perfetti, 1985; Perfetti & Hogaboam, 1975; Perfetti, Marron, & Foltz, 1996; Shankweiler, 1999; Torgesen, 2000). When top-down skill is lacking, even if all words can be correctly decoded, an understanding of the text may not be attained because the meanings of the words are unknown or the logical and structural relationships among them are not appreciated, as would presumably occur even if the text were to be read aloud for oral comprehension (e.g., Catts, Fey, Zhang, & Tomblin, 1999; Catts & Hogan, 2002; Gough & Tunmer, 1986; McCardle, Scarborough, & Catts, 2001; Nation, 2005; Nation & Snowling, 1998, 2000; Scarborough, 1990). Evidence that each of these components is necessary, and that neither is sufficient, for reading comprehension derives from studies of (a) the effectiveness of bottom-up interventions, (b) the prediction of comprehension differences, and (c) characteristics of students with specific comprehension deficits. Intervention Studies Many interventions have been designed to strengthen children s bottom-up processing, including phonological awareness and decoding, and their effectiveness has been investigated in rigorous research efforts (e.g., Foorman et al., 1997; Foorman, Francis, Fletcher, Schatschneider, & Metha, 1998; Olson, Wise, Johnson, & Ring, 1997; Rashotte, MacPhee, & Torgesen, 2001; Shaywitz et al., 2004; Torgesen et al., 2001; Torgesen et al., 1999). 1 Although robust improvements in word recogni- 1 Note that not all intervention studies have used or reported on measures of reading comprehension (e.g., Foorman et al., 1997; Torgesen, Morgan, & Davis, 1992), so it is not known if the children would have showed gains in this area.

3 PREDICTION OF READING COMPREHENSION 279 tion/decoding have been clearly demonstrated in most of these studies (National Reading Panel & National Institute for Literacy, 2001), reading comprehension outcomes have been more mixed, with substantial gains seen in some studies (e.g., Rashotteetal.,2001;Shaywitzetal.,2004;Torgesenetal.,2001)butnotothers(e.g., Lovett et al., 1994). Moreover, gains have not always been greater for students trained in bottom-up skills than for students whose instruction placed less emphasis on decoding and phonological processing, as would be predicted if bottom-up skills were the only, or predominant, factor contributing to comprehension (Foorman et al., 1998; Torgesen et al., 1999). Hence, although demonstrating that instruction to strengthen children s bottom-up skills is extremely valuable, these studies also indicate that mastering bottom-up skills will not automatically yield gains in reading comprehension, presumably because other necessary components of successful comprehension have not been developed through such interventions. Prediction of Comprehension Differences Multiple regression and latent variable modeling methods have been used to examine contributions to reading comprehension beyond those of word recognition/decoding skill (e.g., Catts et al., 1999; Francis, Fletcher, Catts, & Tomblin, 2005; Share & Leikin, 2004). Although bottom-up reading and phonological skills have indeed been shown to predict reading comprehension scores well, additional variance has been accounted for by the inclusion of other skills, most notably oral language proficiencies (e.g., listening comprehension, syntax, vocabulary, etc.; e.g., Catts, Hogan, Adlof, & Barth, 2003; Joshi, Williams, & Wood, 1998). It is not clear, however, whether particular aspects of language proficiency might be more essential than others (e.g., lexical vs. sentence-level processing) or whether the contributions of oral language to reading comprehension might differ depending on which reading comprehension measure is used. Characteristics of Students With Specific Comprehension Deficits Some children are very poor comprehenders of text but nevertheless demonstrate unimpaired word recognition and decoding skills. It is estimated that approximately 10% to 25% of poor readers, or about 3% of the school-age population, exhibit this profile, particularly in the upper elementary school grades and at older ages (e.g., Aaron, Joshi, & Williams, 1999; Catts, Adlof, & Weismer, in press; Catts et al., 2003; Leach, Scarborough, & Rescorla, 2003; Shankweiler et al., 1999). Moreover, in a large, epidemiological sample, Catts et al. (1999) found that only 14% of all poor comprehenders exhibited phonological-processing deficits. Hence, specific comprehension deficits are not adequately explained in terms of bottom-up weaknesses, suggesting that other factors are largely responsible for

4 280 CUTTING AND SCARBOROUGH these children s comprehension difficulties. Furthermore, and consistent with the simple model, specific comprehension deficits have been associated with oral language weaknesses, especially in vocabulary and syntactic skills (Cain, Oakhill, Barnes, & Bryant, 2001; Catts et al., in press; Nation, Adams, Bowyer-Crane, & Snowling, 1999; Nation & Snowling, 1998, 1999, 2000). OTHER POTENTIAL INFLUENCES ON READING COMPREHENSION Reading speed is often mentioned as another factor that could affect reading comprehension, because inefficient word recognition/decoding is thought to create a processing bottleneck, preventing sufficient cognitive resources to be allocated for comprehension (LaBerge & Samuels, 1974; Perfetti, 1985; Perfetti & Hogaboam, 1975; Perfetti et al., 1996). Indeed, it has been shown that reading speed, both of isolated words and words in context, influences reading comprehension (e.g., Jenkins, Fuchs, van den Broek, Espin, & Deno, 2003; Lovett, 1987; Rupley, Willson, & Nichols, 1998; Swanson & Trahan, 1996). Furthermore, inclusion of measures of symbol-naming speed have been shown to increase the prediction of reading comprehension (Joshi & Aaron, 2000). It may be, therefore, that taking into account the speed with which printed words can be identified, rather than just bottom-up accuracy, may lead to even stronger prediction of reading comprehension. A few other factors that may also play a role in fostering reading comprehension include verbal memory (Perfetti et al., 1996; Swanson, Cochran, & Ewers, 1989); inferential and reasoning skills (Cain et al., 2001; Catts et al., in press); and attention (Gehlani, Sidhu, Jain, & Tannock, 2004; McInnes, Humphries, Hogg-Johnson, & Tannock, 2003). In principle, a lack of memory capacity could limit a reader s ability to retain sufficient information about the words in a text to process meaning adequately. To the extent that passage comprehension requires reading between the lines, inferential skills that transcend basic listening comprehension abilities may be needed. Finally, maintaining attention to the task and allocating resources appropriately to bottom-up and top-down requirements may also be essential for successful comprehension, such that individuals with attention deficits could show impaired reading comprehension despite adequate decoding and oral language competencies. MEASUREMENT OF READING COMPREHENSION In research on the relative necessity and importance of various components of reading comprehension, attention to how reading comprehension is measured has not always been a focus. Yet comprehension tests vary markedly in their task de-

5 PREDICTION OF READING COMPREHENSION 281 mands and conceptual underpinnings, and there are indications that the contributions of bottom-up and top-down factors may not be the same across tests. In prediction studies, substantial differences have been seen in the percentage of variance that word recognition/decoding accounts for, with estimates ranging from approximately 25% to 81% (Hoover & Gough, 1990; Juel, 1988; Shankweiler et al., 1999; Torgesen et al., 1999). Although the role of bottom-up skills may diminish over time (e.g., Catts et al., 1999; Catts et al., 2003; Francis et al., 2005; Juel, 1988; Storch & Whitehurst, 2002; Vellutino, Scanlon, & Tanzman, 1994), age does not appear to account fully for this variability, because differences of similar magnitude have been seen between samples of the same age, for example, 23% versus 44% for first graders (Hagtvet, 2003; Juel, 1988), and 46% to 48% versus 64% to 67% for second and third graders (Catts et al., 1999; Hoover & Gough, 1990; Vellutino et al., 1994). Some differences may instead stem from how reading comprehension is measured. For example, some test formats may be more demanding of bottom-up skills than others. This could explain, for instance, why word recognition/decoding has accounted for more variance in comprehension scores when cloze tests versus question-and-answer tests have been used in a single sample, for example, 79% versus 53% in Nation and Snowling s (1997) sample of 7- through 10-year-olds; 49% versus 16% in Bowey s (1986) sample of fourth and fifth graders; and 51% versus 34% in Spear-Swerling s (2004) fourth-grade sample, in which the additional variance accounted for by oral language skills was more similar across formats (14% vs. 20%). Similarly, Francis et al. (2005) found, using latent variable modeling techniques, a stronger relationship between decoding and a cloze test than for comprehension measures that used silent or oral passage reading with multiple-choice questions (both of which had a stronger relationship with language than did the cloze measure). Format may not necessarily be the only, or the most critical, difference among reading comprehension instruments however. The handful of tests that are most commonly used in research were created by different authors, whose conceptions of reading comprehension are often not explicitly stated and may be quite varied. Yet each must have had some guiding construct in mind regarding what kinds of text manipulations will raise comprehension difficulty across test items. Sentence and passage length, word frequencies, syntactic complexity, inclusion of academic versus colloquial language forms, and so forth, could conceivably affect the comprehensibility of passages, and for different reasons. Also, some tests allow readers to see the texts while answering questions about them, but others do not, thus presumably imposing heavier memory demands. If different skill sets are more important to performance on some reading comprehension tests than others, conclusions about the nature of comprehension (and comprehension difficulties) will be specific to the test, leading to the kinds of mixed findings that have been reported about the relative contributions of bot-

6 282 CUTTING AND SCARBOROUGH tom-up skills and various cognitive and linguistic abilities. We felt, therefore, that investigating the differences among several reading comprehension measures would be worthwhile. In this study, to gain a better understanding of these issues, we aimed to address the following questions: 1. Do the contributions of word recognition/decoding and oral language skills to reading comprehension depend on the measure of comprehension that is used? 2. Beyond word recognition/decoding and oral language, do other skills account for additional variance in reading comprehension as measured by different tests? Specifically, is the prediction of reading comprehension enhanced by taking into account reading speed, verbal working memory, serial naming speed, IQ, or attention? 3. Do the relative contributions of various predictors of comprehension differ for readers with differing levels of reading skill? METHOD Participants The sample included 97 children (65 boys and 32 girls) in Grades 1.5 through 10.8 (M = 4.4, SD = 2.2), whose ages ranged from 7.0 to 15.9 years (M = 9.7, SD = 2.1). According to Hollingshead s (1975) five-tiered socioeconomic scale based on parental education and occupation, 81% of the participants were from the higher strata (Levels I and II), and 19% were from the lower three tiers. The sample was predominantly Caucasian (85%) but also included African Americans (8%), Asians (3%), and students of mixed race (4%). All were native speakers of English. This sample was not recruited specifically for this study but rather was drawn from the comparison sample for an ongoing investigation of reading and language deficits associated with Neurofibromatosis Type 1 (NF 1). The eligibility criteria for that project were as follows: an age between 6 and 16 years; no history of seizures, head injury, or other neurological illness; no history of major psychiatric illness; no treatment for any psychiatric disorder with psychotropic medications (other than stimulant medications); no uncorrected hearing or visual impairments; and IQ of 80 (Verbal IQ, Performance IQ, or Full-Scale IQ) or higher. Children with a diagnosis of attention deficit hyperactivity disorder (ADHD) were excluded only if they were being treated with medications other than stimulants. The comparison sample met the foregoing criteria and did not have NF 1. From that group, data were analyzed for all children for whom scores were available on all three reading comprehension instruments. Twenty-five children appeared to

7 PREDICTION OF READING COMPREHENSION 283 meet diagnostic criteria for ADHD on the basis of data collected for the study 2 ; those with a prior diagnosis of ADHD who were being treated with stimulants were on medication at the time of testing. Measures From a larger battery administered in the NF 1 project, a subset of measures was selected for analyses in this study. These included scores on three reading comprehension tests and a variety of measures of word recognition/decoding, oral language, reading speed, IQ, serial naming speed, verbal working memory, and attention. All measures were individually administered in the same order during three sessions lasting approximately 2.5 hr each. Unless noted otherwise, standard scores based on national norms were computed for use in the analyses. Reading comprehension. The (reading) comprehension subtests from three widely used instruments were used: the Gates MacGinitie Reading Test Revised (G M; MacGinitie, MacGinitie, Maria, & Dreyer, 2000); the Gray Oral Reading Test Third Edition (GORT 3; Wiederholt & Bryant, 1992); and the Wechsler Individual Achievement Test (WIAT; Wechsler, 1992); On the G M, expository and narrative passages, each containing 3 to 15 sentences, are read silently. Each is followed by three to six written multiple-choice questions that are answered while the passage is still in view. Items increase in difficulty, and there is a 35-min time limit. According to the manual, internal consistency reliability ranges from.91 to.93 and alternate form reliability from.80 to.87 across levels. On the GORT 3, expository and narrative passages, each containing six or seven sentences, are read aloud as quickly as possible. Five multiple-choice questions are read orally by the examiner after the passages is removed from view. Passages increase in difficulty and testing terminates after the participant incorrectly answers three out of five comprehension questions. Internal consistency reliability is reported as.87 in the test manual. On the WIAT, expository and narrative passages, each containing two or three sentences, are read silently. Two open-ended questions (one literal and one inferential) about each passage are asked orally by the examiner while the text remains in view. Items increase in difficulty, and testing is discontinued after four questions 2 Although no formal clinical diagnostic interview was conducted, children were considered to exhibit signs of ADHD if they met two of the following three criteria: (a) a rating of 2 or higher for six of nine hyperactivity items and six of nine inattention items on the ADHD IV rating scale (DuPaul, Power, Anastopoulos, & Reid, 1998); (b) at least 1.5 standard deviations above the mean (T 65) on the Inattentive and/or Hyperactivity/Impulsive scales on the Conners Rating Scales Revised (Conners, 1997); or (c) at least 1.5 standard deviations above the sample s mean (T > 65) for the Attention Problem Index of the Child Behavioral Checklist (CBCL; Achenbach, 1991).

8 284 CUTTING AND SCARBOROUGH in a row are answered incorrectly. In the test manual, estimates of.88 for split half reliability and.85 for retest reliability are reported. Word recognition/decoding. Two tests were used to examine children s bottom-up skills: the Basic Reading subtest of the WIAT and the Word Attack subtest from the Woodcock Johnson Psychoeducational Battery Revised (Woodcock & Johnson, 1989). A composite score was created by averaging these two standard scores. Oral language. Measures of two aspects of language proficiency were available. Lexical skills were assessed with the Peabody Picture Vocabulary Test Third Edition (Dunn & Dunn, 1997), a receptive vocabulary test on which the child must indicate which of four pictures best represents a word spoken by the examiner, for a series of increasingly difficult items; the Boston Naming Test (Kaplan & Goodglass, 1978), a measure of expressive vocabulary on which the child is asked to name as many items as possible in a series of 60 line drawings of objects, decreasing in familiarity from bed to abacus; and the Word Classes subtest of the Clinical Evaluation of Language Fundamentals, Third Edition (CELF 3; Semel, Wiig, & Secord, 1995), which assesses knowledge of word meanings by requiring the child to indicate which two words, out of a series of three to four words spoken by the examiner, are most closely related to each other. A lexical composite score was created by extracting a principal component from a factor analysis into which all three vocabulary scores were entered. The four measures of sentence processing were used. Three were other subtests of the CELF 3: Concepts and Directions, on which the child listens to directions of increasing complexity and then carries them out by pointing to the items specified in the appropriate sequence; Formulated Sentences, which requires the child to generate sentences that include target words; and Recalling Sentences, which requires recalling and repeating sentences of increasing length and syntactic complexity. In addition, we used a 16-item experimental syntactic comprehension measure (Menyuk & Cohen, n.d.) that was designed to evaluate a child s understanding of complex sentences with embedded clauses (e.g., The lion that the tiger bit jumped over the giraffe ). After presenting each sentence orally, the examiner asked a comprehension question (e.g., Who jumped over the giraffe? ). A sentence-processing composite score was created using principal-components analysis. Reading speed. The Rate subtest from the GORT 3 was used to assess how quickly a child is able to read words in connected text. Rapid serial naming. The Rapid Naming score from the Comprehensive Test of Phonological Processing (CTOPP; Wagner, Torgesen, & Rashotte, 1999)

9 PREDICTION OF READING COMPREHENSION 285 was used. It is based on the child s naming times for separate arrays of letters, numbers, and colors. Full-Scale IQ. The Wechsler Intelligence Scales for Children, Third Edition (WISC III; Wechsler, 1991) was administered. Verbal memory. Four memory measures were collected. On the Immediate Recall subtest of the Wide Range Assessment of Memory and Learning (Sheslow & Adams, 1990), the child listens to two stories and retells each of them with as much detail as possible. The score is based on the number of story elements that are recalled. From the CTOPP, we used the Nonword Repetition test, which requires immediate imitation of each of a series of increasingly longer pseudowords, and the Memory for Digits test, which requires listening to a series of numbers and then recalling them in correct order. A nonstandardized sentence span measure(swanson et al., 1989; based on Daneman & Carpenter, 1980) was also administered. On each trial, the examiner reads aloud a set of two or more sentences and asks a question abouteachofthemthatthechildmustanswer.aftereachset,recallofthelastwordof each sentence was required. A composite verbal memory score was not created, because high correlations among the four measures were not obtained. Attention. The parents of the participants completed three questionnaires, from which five scores were derived: the Inattentive and Hyperactivity/Impulsive scales from the ADHD IV rating scales (DuPaul et al., 1998), the Attention Problem Index from the CBCL (Achenbach, 1991), and the Inattentive and Hyperactivity/Impulsivity scales on the Conners Parent Rating Scales Revised (Conners, 1997). For use in analyses, inattention, hyperactivity, and attention composite scores were created by extracting a principal component from an analysis into which either two scores (for the inattention and hyperactivity composites) or all five scores (for the attention composite) were entered. RESULTS Because raw scores on the three nonstandardized tests were correlated with age (r =.62 for the Boston Naming Test,.45 for syntactic comprehension, and.50 for sentence span), these scores were regressed onto age, and the standardized residuals were used in all analyses of these variables. Standard scores based on national norms were analyzed for all other measures. Distributions of scores on each test were examined for skewness, outliers, and other irregularities that could jeopardize the validity of parametric analyses, and none were found. Missing data (scores for 1 child each on the Wide Range Assessment of Memory and Learning, CTOPP, CELF, syntactic comprehension, sentence span, and CBCL) were im-

10 286 TABLE 1 Performance on Reading, Language, and Cognitive Measures by the Sample and Correlations of Reading Comprehension Scores With Other Variables Test M SD Range G M GORT 3 WIAT Differences a Reading comprehension G M Comprehension GORT 3 Comprehension WIAT Comprehension Word recognition/decoding WIAT Basic Reading WJ Word Attack Word reading composite GORT < WIAT Oral language: Lexical knowledge CELF 3 Word Classification PPVT Boston Naming Test b Lexical Composite none Oral language: Sentence processing CELF 3 Concepts and Directions CELF 3 Formulated Sentences CELF 3 Recalling Sentences Syntactic Comprehension b Sentences Composite GORT < others Reading speed GORT 3 Rate none Rapid serial naming CTOPP Rapid Naming none

11 Verbal memory CTOPP Memory for Digits GORT < G-M CTOPP Nonword Repetition none Sentence span b G-M < WIAT WRAML Immediate Recall none IQ WISC III Full-Scale IQ none Attention ADHD IV Hyperactivity Scale c Conners Hyperactivity Scale Hyperactivity composite none ADHD IV Inattention Scale (%ile) Conners Inattention Scale Inattention composite none CBCL Attention Scale Total attention composite none Note. N = 97. All coefficients differ significantly from zero (p <.05, two-tailed). GORT 3 = Gray Oral Reading Test Third Edition; WIAT = Wechsler Individual Achievement Test; WJ = Woodcock Johnson; CELF 3 = Clincial Evaluation of Language Fundamentals, Third Edition; PPVT 3 = Peabody Picture Vocabulary Test Third Edition; CTOPP = Comprehensive Test of Phonological Processing; WRAML = Wide Range Assessment of Memory and Learning; WISC III = Wechsler Intellligence Scale for Children Third Edition; ADHD IV = Attention Deficit Hyperactivity Disorder Rating Scale Fourth Edition; CBCL = Child Behavorial Checklist. a p <.05, Hotelling Williams z test. b Age adjusted regression residuals were created and used in analyses for these measures. c Percentiles. 287

12 288 CUTTING AND SCARBOROUGH puted via regression when possible and in one instance by entering the mean, following Tabachnick and Fidell s (1996) guidelines. Table 1 provides descriptive statistics for all measures of reading, language, and cognitive skills and lists correlations of each reading comprehension score with other variables. Table 2 shows the correlations among the predictor measures. For all three comprehension tests, performance levels in the sample approximated the national averages. The G M and WIAT correlated very strongly with each other (r =.79) but less well with the GORT 3 (r =.64 with G M, z = 3.11, p =.003; r =.70 with WIAT, z = 1.75, p =.08). As shown in Table 1, the reading comprehension measures differed somewhat in the strength of their associations with word recognition/decoding, sentence processing, and verbal memory skills. Prediction of reading comprehension scores was examined using hierarchical multiple regression analyses (summarized in Table 3). First, to investigate the relative contributions unique and shared of word recognition/decoding and oral language skills to reading comprehension, a pair of hierarchical multiple regression analyses was conducted for each of the three comprehension measures. In one analysis (Model 1A), the word reading composite was entered at the first step, and the lexical and sentence-processing composites at the second. In the other analysis (Model 1B), the order of entry was reversed. Each of the two factors accounted for significant variance in comprehension beyond that accounted for by the other. Their shared and unique contributions are illustrated in the first panel of Figure 1. TABLE 2 Correlations Among Predictors Variable Word reading composite 2. Lexical composite Sentences composite GORT 3 rate CTOPP Rapid Naming IQ CTOPP Memory for Digits CTOPP NW Repetition Sentence span WRAML Immediate Memory Hyperactivity composite Inattention composite Total attention composite Note. N = 97. Coefficients of.20 or larger differ signficantly from zero, p <.05, two-tailed. GORT 3 = Gray Oral Reading Test Third Edition; CTOPP = Comprehensive Test of Phonological Processing; NW = Nonword; WRAML = Wide Range Assessment of Memory and Learning.

13 TABLE 3 Prediction of Reading Comprehension: Multiple Regression Results Step and Predictors Entered G M WIAT GORT 3 R R 2 R 2 + β R R 2 R 2 + β R R 2 R 2 + β Model 1A 1 Word Recognition *.35* (.19*) *.50* (.39*) *.39* (.12) 2 Oral Language * * * Sentence factor.23* (.21*).31* (.29*).03 (.01) Lexical factor.34* (.25*).14 (.09).37* (.22) Model 1B 1 Oral Language * * * 2 Word Recognition * * * Model 1C 2 Lexical factor * * * 3 Sentence factor * * Model 1D 2 Sentence factor * * * 3 Lexical factor * * Model 2 3 Reading Speed *.30* *.18* *.48* 289 (continued)

14 290 TABLE 3 (Continued) Step and Predictors Entered G M WIAT GORT 3 R R 2 R 2 + β R R 2 R 2 + β R R 2 R 2 + β Model 3 3 Rapid Serial Naming Model 4 3 Full Scale IQ Model 5 3 Verbal Memory Memory for Digits NW Repetition Immediate Recall Sentence span Model 6A, B, C 3 Inattention Hyperactivity * Attention total Note. Beta values in parentheses are those for Model 2. G M = Gates MacGinitie Reading Test Revised; WIAT = Wechsler Individual Achievement Test; GORT 3 = Gray Oral Reading Test Third Edition; NW = Nonword. *p <.05.

15 PREDICTION OF READING COMPREHENSION 291 FIGURE 1 Decomposition of variance accounted for by components of the simple model (left panel), and by components within the unique proportion attributed to oral language proficiencies (right panel), in analyses of reading comprehension scores from the Gates MacGinitie (G M), Wechsler Individual Achievement (WIAT), and Gray Oral Reading (GORT 3) tests. Second, the potentially separate contributions of different aspects of oral language proficiency namely, lexical and sentence-processing skills were examined in a second pair of regression analyses, Models 1C and 1D in Table 3. Both aspects of language made unique as well as shared contributions to G M scores. However, only lexical skills accounted for unique variance on the GORT 3, and only sentence processing did so when the WIAT was predicted. These differences are illustrated in the second panel of Figure 1. Third, we investigated whether the prediction of reading comprehension was improved by including reading speed (Model 2), rapid serial naming (Model 3), verbal memory (Model 4), IQ (Model 5), or attention (Model 6) at third step, after the factors in Model 1 had been entered. For each comprehension test, an additional 1% to 6% of the variance was accounted for by reading speed. No other variables contributed significantly to reading comprehension, except for a small (1%) effect when the hyperactivity composite was included in predicting WIAT scores. Last, we examined the hypothesis that how well reading comprehension can be predicted might interact with word recognition/decoding ability. For each comprehension measure, a pair of hierarchical regression analyses was conducted. In all

16 292 CUTTING AND SCARBOROUGH analyses, the predicted score from Model 2 (representing the combined contributions of word recognition/decoding, oral language proficiency, and reading speed) was entered at the first step; then, after centering, cross-products were computed by multiplying predicted scores by a measure of reading ability, creating an interaction term that was entered at the second step of the analysis. In one analysis of each pair, absolute decoding ability was the focus, so raw scores on Word Attack were used to create the cross-product. In the other analysis, bottom-up skills relative to age norms were the focus, so the word reading composite (based on standard scores) was used in computing the interaction term. For all three comprehension scores, adding these interaction terms to the model produced increases of less than 1% in the proportion of variance accounted for and thus did not significantly improve prediction (all ps >.19). DISCUSSION A major aim of this study was to examine the contributions of word recognition/decoding, oral language, and other cognitive skills to children s reading comprehension. By analyzing three comprehension measures in parallel in a sample with a wide age range, we also sought to determine whether the prediction of reading comprehension might depend on the particular dependent measure that was used, or on a child s reading level. Do the Contributions of Word Recognition/Decoding and Oral Language Skills to Reading Comprehension Depend on the Measure of Comprehension That Is Used? Consistent with much previous research (e.g., Catts et al., 1999; Joshi et al., 1998; Share & Leikin, 2004), we found that both word recognition/decoding and oral language skill the two components of Gough and Tunmer s (1986) simple model made unique contributions to prediction, regardless of which comprehension measure was analyzed. The total amount of variance that could be accounted for by the simple model ranged from only 49% for the GORT 3 to 67% to 72% for the WIAT and G M tests, and even when reading speed was included as a predictor, only 56% of the variance in GORT 3 scores could be explained. It is not clear what other skills, aside from the several that we examined, may be responsible for the especially high proportion of unexplained GORT 3 variance. The unique contributions of word recognition/decoding skill varied across comprehension measures, with nearly twice as much variance accounted for in WIAT scores (11.9%) than in G M (6.1%) and GORT (7.5%) scores. The zero-order correlations in Table 1 also indicated that WIAT performance was most strongly influenced by bottom-up skills, accounting for 62% of the variance, in contrast to only

17 PREDICTION OF READING COMPREHENSION % to 49% for the other comprehension measures. The differences we observed were not as extreme, however, as those that have been seen in previous research, especiallywhenaclozemeasurehasbeenusedasoneofthecomprehensiontestsgiven to a single sample, for example, 16% versus 49% (Bowey, 1986), 53% versus 79% (Nation & Snowling, 1997), and 34% versus 51% (Spear-Swerling, 2004). Taken all together, the results indicate that bottom-up skills affect performance on some kinds of reading comprehension tests more than on others. The percentage of variance uniquely explained by oral language proficiency was similar for the WIAT and GORT 3 (each 9%) but substantially higher for the G M (15%). Unique contributions were made by both language composites when predicting G M scores (4.5% by lexical and 1.8% by sentence processing), but only by sentence processing (3.4%) in the analysis of WIAT scores and only by lexical processing (5.3%) when the GORT 3 was the dependent measure, with less than 1% of the variance attributable uniquely to the other language measures in those analyses. These findings suggest that different measures of reading comprehension may make differential demands on vocabulary knowledge and sentence-processing abilities. In prior prediction research, investigators have often lumped oral language measures together(e.g., by using a listening comprehension score or a composite) or have used a single measure of linguistic skill (typically, listening comprehension or vocabulary; e.g., Catts et al., 1999; Joshi et al., 1998; Share & Leikin, 2004). In light of our findings, we think it would be fruitful to measure and analyze separately several facets of oral language proficiency in future research on the nature of reading comprehension and comprehension difficulties. It also bears noting that, as previously pointed out by Catts et al. (2003), there is a very substantial amount of shared variance between word recognition/decoding and oral language measures when comprehension scores are predicted. These components are usually conceptualized as largely separate skill sets one involving print-based skills acquired largely through instruction and the other reflecting the culmination of years of oral language development. The basis for their largely combined, rather than unique, contributions to reading comprehension is not entirely clear and merits further investigation. Beyond Word Recognition/Decoding and Oral Language, Do Other Skills Account for Additional Variance in Reading Comprehension as Measured by Different Tests? The inclusion of reading speed in regression analyses improved prediction significantly, accounting for an additional 1% to 6% of the variance on the three measures of reading comprehension. We therefore concur with Joshi and Aaron s (2000) suggestion that the simple model plus reading speed appears to predict reading comprehension optimally, regardless of the measure of comprehension that is used.

18 294 CUTTING AND SCARBOROUGH In contrast, the prediction of comprehension scores was not enhanced by taking into account any measures of verbal memory, rapid serial naming, IQ, or (with one minor exception) attention. There was ample power to detect potential contributions of meaningful magnitude by these various skills that have been hypothesized to matter for successful comprehension. Given the substantial bivariate correlations of these measures with the reading comprehension measures, it appears that the variance that they account for is almost entirely subsumed within the contributions of word recognition/decoding and oral language proficiency. Do the Relative Contributions of Predictors of Comprehension Differ for Readers With Differing Levels of Reading Skill? Despite the wide range of ages and reading levels in the sample, we found no evidence that the prediction of reading comprehension could be improved by taking into account either the child s absolute level of skill in decoding or the child s word recognition skills relative to peers. These results are thus contrary to the hypothesis that comprehension would be more constrained by bottom-up processing for novice readers and lower achieving students. Some empirical support for age differences has been observed, however, in several prior studies (e.g., Catts et al., 1999; Catts et al., 2003; Francis et al., 2005; Juel, 1988; Storch & Whitehurst, 2002; Vellutino et al., 1994). Hence, although the hypothesized effects were not obtained in our sample, their occurrence under some circumstances is certainly possible and theoretically plausible. Implications and Future Directions Our findings converge with those from the few other prediction studies that have compared two or more reading comprehension tests (Bowey, 1986; Francis et al., 2005; Keenan, Betjemann, & Roth, 2005; Nation & Snowling, 1997, Spear-Swerling, 2004). Taken together, the results raise a concern that commonly used tests of reading comprehension do not necessarily tap the same array of cognitive processes and may be influenced to different degrees by particular skills that can influence comprehension. Given that so many different instruments have been used in previous research, the apparent nonequivalence of such tests may have contributed to disagreements across studies in their conclusions about which components are necessary, sufficient, and most important for successful comprehension. We are concerned, furthermore, that the picture would become even more complex if cloze measures of reading comprehension were to be analyzed alongside the question-and-answer tests of the sort we included. In our view, there needs to

19 PREDICTION OF READING COMPREHENSION 295 be a systematic investigation of similarities and differences among reading comprehension measures, perhaps ultimately leading to the development of new instruments that correspond more closely to particular theoretical models of the construct being measured. In doing so, the effects of variation in test format and passage characteristics, and perhaps other aspects of the assessment situation, need to be examined and disentangled. In addition, more refined measures of reading comprehension may be sensitive to the influence of some of the variables that showed no effects in our analyses. There are also some important practical implications of the findings. First, whether a reading comprehension deficit will be detected in clinical assessment may depend on the choice of measure for that purpose. This was demonstrated in a recent study of a subgroup of children from this sample (Rimrodt, Lightman, Roberts, Denckla, & Cutting, 2005). Of all children identified by any of the three tests as having a comprehension deficit, only about 25% were identified as such by all three tests, and about half were identified by a single test but not the others. Furthermore, different tests may provide discrepant information about which component skills are the basis for a child s comprehension difficulties and need to be targeted for remediation. Given the current state of affairs, special educators and psychologists may need to use multiple reading comprehension measures, therefore, to determine eligibility for special educational services and for planning interventions. Even so, it is reassuring that a common core model essentially, an expansion of the model of Gough and Tunmer (1986) to include reading speed was supported in our analyses. For all three tests we compared, both word recognition/decoding and oral language made unique and shared contributions, even though there was not always agreement from test to test regarding the total amount of variance that could be predicted and the relative contributions of lexical and sentence-level language processes. Additionally, reading speed made significant contributions to the prediction of reading comprehension, beyond word recognition/decoding and oral language measures. We are optimistic that, working from this firm starting point, once measurement issues are better resolved it will be possible to arrive at a fuller understanding of the essential components of reading comprehension and the bases for comprehension deficits. ACKNOWLEDGMENTS This work was supported in part by the Johns Hopkins School of Medicine General Clinical Research Center (NIH M01-RR00052), U.S. Congressionally Directed Materiel and Medical Command(DAMD ) and NIH R01-HD

20 296 CUTTING AND SCARBOROUGH REFERENCES Aaron, P. G., Joshi, R. M., & Williams K. A. (1999). Not all reading disabilities are alike. Journal of Learning Disabilities, 32, Achenbach, T. (1991). Manual for the Child Behavior Checklist (Parent Form). Burlington, VT: University Associates in Psychiatry. Adams, M. J. (1990). Beginning to read: Thinking and learning about print. Cambridge, MA: MIT Press. Bowey, J. (1986). Syntactic awareness in relation to reading skill and ongoing reading comprehension monitoring. Journal of Experimental Child Psychology, 41, Cain, K., Oakhill, J. V., Barnes, M. A., & Bryant, P. E. (2001). Comprehension skill, inference-making ability, and the relation to knowledge. Memory & Cognition, 29, Catts, H. W., Adlof, S. M., & Weismer, S. E. (in press). Language deficits in poor comprehenders: A case for the simple view of reading. Journal of Speech-Language-Hearing Research. Catts, H. W., Fey, M. E., Zhang, X., & Tomblin, J. B. (1999). Language basis of reading and reading disabilities: Evidence from a longitudinal investigation. Scientific Studies of Reading, 3, Catts, H. W., & Hogan, T. P. (2002, June). Late-emerging readers: The fourth grade slump. Poster presented at the annual meeting of the Society for the Scientific Study of Reading, Chicago. Catts, H. W., Hogan, T. P., Adlof, S. M., & Barth, A. E. (2003, June). The simple view of reading changes over time. Paper presented at the annual meeting of the Society for Scientific Study of Reading, Boulder, CO. Conners, C. K. (1997). Conners Rating Scales Revised. North Tonawanda, NY: Multihealth Systems. Daneman, M., & Carpenter, P. (1980). Individual differences in working memory and reading. Journal of Verbal Learning and Verbal Behavior, 19, Dunn, L. M., & Dunn, L. M. (1997). Peabody Picture Vocabulary Test Third edition. Circle Pines, MN: American Guidance Service. DuPaul, G. J., Power, T. J., Anastopoulos, A. D., & Reid, R. (1998). ADHD Rating Scale IV. New York: Guilford. Foorman, B. R., Francis, D. J., Fletcher, J. M., Schatschneider, C., & Metha, P. (1998). The role of instruction in learning to read: Preventing reading failure in at-risk children. Journal of Educational Psychology, 90, Foorman, B. R., Francis, D. J., Winikates, D., Mehta, P., Schatschneider, C., & Fletcher, J. M. (1997). Early interventions for children with reading disabilities. Scientific Studies of Reading, 1, Francis, D. J., Fletcher, J. M., Catts, H. W., & Tomblin, J. B. (2005). Dimensions affecting the assessment of reading comprehension. In S. G. Paris & S. A. Stahl (Eds.), Children s reading comprehension and assessment (pp ). Mahwah, NJ: Lawrence Erlbaum Associates, Inc. Ghelani, K., Sidhu, R., Jain, U., & Tannock, R. (2004). Reading comprehension and reading related abilities in adolescents with reading disabilities and attention-deficit/hyperactivity disorder. Dyslexia, 10, Gough, P. B., & Tunmer, W. E. (1986). Decoding, reading, and reading disability. Remedial and Special Education, 7, Hagtvet, B. E. (2003). Listening comprehension and reading comprehension in poor decoders: Evidence for the importance of syntactic and semantic skills as well as phonological skills. Reading & Writing, 16, Hollingshead, A. (1975). Four factor index of social status. Unpublished manuscript. Hoover, W. A., & Gough, P. B. (1990). The simple view of reading. Reading & Writing, 2, Jenkins, J. R., Fuchs, L. S., van den Broek, P., Espin, C., & Deno, S. L. (2003). Sources of individual differences in reading comprehension and fluency. Journal of Educational Psychology, 95,

21 PREDICTION OF READING COMPREHENSION 297 Joshi, R., & Aaron, P. G. (2000). The component model of reading: Simple view of reading made a little more complex. Reading Psychology, 21, Joshi, R. M., Williams, K. A., & Wood, J. R. (1998). Predicting reading comprehension from listening comprehension: Is this the answer to the IQ debate? In C. Hume & R. M. Joshi (Eds.), Reading and spelling: Developmentanddisorders(pp ). Mahwah, NJ: LawrenceErlbaumAssociates, Inc. Juel, C. (1988). Learning to read and write: A longitudinal study of 54 children from first through fourth grades. Journal of Educational Psychology, 80, Kaplan, E., & Goodglass, H. (1978). Boston Naming Test (experimental ed.). Philadelphia: Lea & Febiger. Keenan, J. M., Betjemann, R. S., & Roth, L. S. (2005, May). A comparison of comprehension tests. Paper presented at the 77th Midwestern Psychological Association Meeting, Chicago. LaBerge, D., & Samuels, S. (1974). Toward a theory of automatic information processing in reading. Cognitive Psychology, 6, Leach, J. M., Scarborough, H. S., & Rescorla, L. (2003). Late-emerging reading disabilities. Journal of Educational Psychology, 95, Lovett, M. W. (1987). A developmental approach to reading disability: Accuracy, and speed criteria of normal and deficient reading skill. Child Development, 58, Lovett, M. W., Borden, S. L., DeLuca, T., Lacerenza, L., Benson, N. J., & Brackstone, D. (1994). Treating the core deficits of developmental dyslexia: Evidence of transfer of learning after phonologically- and strategy-based reading training programs. Developmental Psychology, 30, Lyon, G. R. (1995). Toward a definition of dyslexia. Annals of Dyslexia, 45, MacGinitie, W. H., MacGinitie, R. K., Maria, K., & Dreyer, L. G. (2000). Gates MacGinitie Reading Tests (4th ed.). Itasca, IL: Riverside. McCardle, P., Scarborough, H. S., & Catts, H. W. (2001). Predicting, explaining, and preventing children s reading difficulties. Learning Disabilities Research & Practice, 16, McInnes, A., Humphries, T., Hogg-Johnson, S., & Tannock, R. (2003). Listening comprehension and working memory are impaired in attention-deficit hyperactivity disorder irrespective of language impairment. Journal of Abnormal Child Psychology, 31, Menyuk, P., & Cohen, L. (n.d.). [Unpublished manuscript]. Boston University and Children s Hospital Medical Center, Boston. Nation, K. (2005). Children s reading comprehension difficulties. In M. Snowling & C. Hulme (Eds.) The science of reading: A handbook (pp ). Boston: Blackwell Synergy. Nation, K., Adams, J. W., Bowyer-Crane, C. A., & Snowling, M. J. (1999). Working memory deficits in poor comprehenders reflect underlying language impairments. Journal of Experimental Child Psychology, 73, Nation, K., & Snowling, M. (1997). Assessing reading difficulties: The validity and utility of current measures of reading skill. British Journal of Educational Psychology, 67, Nation, K., & Snowling, M. J. (1998). Semantic processing and the development of word-recognition skills: Evidence from children with reading comprehension difficulties. Journal of Memory and Language, 39, Nation, K., & Snowling, M. J. (1999). Developmental differences in sensitivity to semantic relations among good and poor comprehenders: Evidence from semantic priming. Cognition, 70, B1 B13. Nation, K., & Snowling, M. J. (2000). Factors influencing syntactic awareness skills in normal readers and poor comprehenders. Applied Psycholinguistics, 21, National Reading Panel and National Institute for Literacy. (2001). Reading, know what works. Washington, DC: U.S. Department of Education, National Institute for Literacy. Olson, R. K., Wise, B., Johnson, M. C., & Ring, J. (1997). The etiology and remediation of phonologically based word recognition and spelling disabilities: Are phonological deficits the hole story? In B. A. Blachman (Ed.), Foundations of reading acquisition and dyslexia: Implications for early intervention (pp ). Mahwah, NJ: Lawrence Erlbaum Associates, Inc.