TECHNICAL REPORT TEXAS PRIMARY READING INVENTORY (1999 EDITION) Center for Academic and Reading Skills

Transcription

1 TECHNICAL REPORT TEXAS PRIMARY READING INVENTORY (1999 EDITION) Center for Academic and Reading Skills University of Texas-Houston Health Science Center Texas Institute for Measurement, Evaluation, and Statistics University of Houston

2 2 Introduction The Texas Primary Reading Inventory (TPRI) is a teacher-administered assessment of reading skills for children in kindergarten, Grade 1, and Grade 2. It was designed to comply with the requirements of TEC by providing a research-based assessment of early reading skills, which is required for all children in Grades K-2 attending public school in Texas. The primary purposes of the TPRI are to facilitate a teacher s capacity to a) identify children at-risk for reading difficulties, including dyslexia, in Grades K-2; and b) set learning objectives and develop instructional plans for these at-risk children. Originally developed in 1997 by the English and Language Arts Curriculum Department at the Texas Education Agency (TEA), the Center for Academic and Reading Skills (CARS) at The University of Texas-Houston Health Science Center and the Texas Institute for Measurement, Evaluation, and Statistics (TIMES) at the University of Houston were contracted to revise the TPRI in order to ensure alignment with a) the recently adopted Texas Essential Knowledge and Skills (TEKS); and b) research on reading skills development. In addition, CARS/TIMES were asked to provide evaluations of the reliability, validity, and implementation of the TPRI, which will be an ongoing process. A description, rationale and purposes of the TPRI can be found in the teachers guide. The TPRI is designed for administration at the middle and end of kindergarten, beginning and end of Grade 1, and beginning of Grade 2. Assessments are also possible at the middle of Grade 1 and the middle and end of Grade 2, but the forms for these assessments are essentially the same at the beginning of Grade 1 or 2. At each of the primary timepoints, the TPRI consists of a screen and an inventory. The screen permits the rapid assessment of individual children. Designations of risk status are yielded, which identify children who most likely do not need additional assessment. This saves teachers considerable time because they do not need to administer the entire inventory to these low-risk children. The inventory is a detailed assessment of reading and reading-related skills that allows the teacher to gain more in-depth information

3 3 that can be used to determine the child s level of risk for reading problems and to help the teacher set learning objectives for the child. Both the screen and the inventory are individually administered and are designed to be given by a trained teacher. In 1998, CARS/TIMES completed an initial assessment of the reliability, validity, and teacher responses to the 1998 edition of the TPRI. Overall, the reliability and validity of the TPRI were satisfactory and teacher response was positive. However, there were components of the 1998 version of the TPRI where the reliability was not adequate, particularly tasks involving book and print awareness, and some comprehension tasks. Teachers provided many comments that suggested ways in which the TPRI could be improved, especially in formatting, directions, and scoring. The 1998 study was conducted on a relatively small sample in Houston. Thus, it was necessary to evaluate the TPRI in a larger, more diverse sample, where data was collected in school districts across the state. To address these issues, the 1998 TPRI was revised to create what we will refer to as the 1999 edition of the TPRI. The most significant revisions involved the development of standardized scoring rubrics for the passages used to assess listening and reading comprehension skills, a major concern of teachers. Many items were re-written to improve the reliability of specific tasks of the TPRI. Directions and formats were changed to make the TPRI easier to administer. An instructional activities guide was added to help teachers develop instructional plans. After revision, it was necessary to evaluate the psychometric characteristics and implementation of the 1999 edition of the TPRI. A statewide study was conducted by CARS and TIMES with collaborators at Texas A&M University (TAMU; Jan Hasbrouck and Carolyn Denton) and the University of Texas-San Antonio (UTSA; Rosalind Horowitz). Given the research underlying the development of the TPRI and the analysis of validity completed in the first study (see 1998 Technical Report), validity was not viewed as a major issue. Rather, the

4 4 issues of primary importance involved the reliability of the 1999 TPRI, possible ethnic or gender bias, and its implementation in the field by schools and teachers. Thus, a study was designed that would permit the collection of TPRIs actually administered by teachers in kindergarten, Grade 1, and Grade 2, along with teachers evaluations of the TPRI. The study was designed to sample across the diverse types of school districts characteristic of Texas. Therefore, we defined four types of school districts: urban, suburban, small city, and rural. School districts in each of these district types that were using the TPRI were invited to participate. Within each of the consenting districts, schools were randomly selected; within each school, two classrooms per grade were randomly selected, although some small schools had only one class in some grades. As Table 1 shows, two urban districts, one suburban district, two small city districts, and nine rural districts participated in the implementation study. There were 52 schools, 299 classrooms, and over 4500 children. The sample was roughly comparable in gender and ethnically diverse, including over 900 students who were African-American, over 700 who were Hispanic, and over 2000 who were White. The sample sizes were adequate to estimate sources of bias for each item of the TPRI, a major goal of the implementation study. In each school, research assistants coded the child s TPRI after teacher administration using scannable data collection forms developed by TIMES. The schools database was used to code gender and ethnicity data, which were missing or not available for about 15% of the students, accounting for the discrepancies in the total sample size relative to the gender and ethnicity analyses. Teachers in the selected classrooms completed a confidential questionnaire, representing a total of 287 of the 299 participants. The questionnaire asked about the teachers experience and training, how they administered the TPRI, training experiences with the TPRI, their reaction to different parts of the TPRI, knowledge about the TPRI, and recommendations for its improvement. In addition, one teacher per school was randomly selected for an interview

5 5 to allow more in-depth reports of teacher experiences with the TPRI, with 48 completed interviews. The teachers were asked to represent the experiences of their school with the TPRI. The implementation study took place in the school year. The results are organized into three sections: 1) reliability of the TPRI; 2) item bias by ethnicity and gender; and 3) teacher responses to the TPRI. Although we report reliability data by gender and ethnicity, the sample size was often not adequate to estimate accurately the reliability of many tasks in these subtests of the sample. These data are reported to see if any patterns of differential reliability emerge, but the item bias analyses specifically addresses the issue of response bias by ethnicity and gender. Reliability of the TPRI There are five forms of the TPRI: middle and end of kindergarten, beginning and end of Grade 1, and beginning of Grade 2. The sample size was not adequate for middle of year analyses as only a small subset of students was involved. We did analyze TPRI data from end of Grade 2 to better understand the difficulty level of the items, but the sample size was marginal for these analyses. The teacher s manuals provide explanations for each of these forms and the rationale underlying their development. For each task, Cronbach s alpha was computed. The alpha coefficient ranges from 0 to 1.0 and was reported as the index of internal consistency. High alpha coefficients indicate that item variability is small relative to total test variability, or that all items perform similarly, and were all measuring the same construct. We evaluated the practical significance of the reliability coefficients as follows: Poor (0-.39), Adequate ( ), Good ( ) and Excellent ( ). These estimates of practical significance are arbitrary, but conventional, and provide useful heuristics for interpreting the reliability data. As the screen is used to make decisions about individual children, we required coefficients at least in the "good or excellent range. Since there were often only a few items and smaller samples for the inventory tasks, we expected a range of coefficients. Although we required a median in the

6 6 good or better range, we set a lower bound of.60 ( good ) as acceptable for alpha coefficients based on the entire sample, and.40 for alpha coefficients based on ethnicity or gender. This is in part due to the fact that Cronbach s alpha is a lower-bound estimate of true reliability, and can be strongly affected by restrictions in range of the sample from which it is estimated. Reliability of Kindergarten Forms Table 2 shows the reliability for each of the 11 tasks of the middle of year kindergarten TPRI and the 14 end of year kindergarten tasks collapsing across ethnicity and gender. The screens for both the middle and end of the year have high alpha coefficients in the upper part of the excellent range. Alphas for all the inventory tasks are above.6 (i.e., good) for 19 of the 24 tasks. The exceptions involve the two print awareness tasks and three of the five listening comprehension stories. As reported in the 1998 Technical Report, Book and Print Awareness tasks had inadequate reliability. All of the stories tend to have lower reliability than the other tasks on the TPRI, which we also reported in In Table 3, alphas are computed separately by ethnicity. The results are similar to the overall estimates in showing excellent reliability for the screens, good reliability for most tasks, and less than good reliability for Book and Print Awareness and some of the stories. End of Year Letter identification was adequate (.56) for African-American. End of Year Blending Phonemes was adequate (.54) for Hispanics. Beginning of Year Blending Phonemes (.58) was adequate for Whites. These lower estimates may reflect smaller sample sizes. No estimates were in the poor range. Table 4 reports the reliabilities by gender. There is no evidence for major gender differences in the reliabilities across each task and the patterns parallel those apparent in the overall results. Reliability of Grade 1 Forms

7 7 Table 5 reports the overall reliability results for the Beginning of Grade 1 and End of Grade 1 TPRI tasks, while Tables 6-7 break down the reliabilities by ethnicity and gender, respectively. These are in the adequate range. In Table 5, we see that the five screening tasks all have excellent reliabilities. Of the 31 Grade 1 tasks, only four have reliabilities below the lower bound of good, again involving the two Book and Print Awareness tasks and two of the four reading comprehension tasks. Tables 6 and 7 do not reveal major differences in reliability estimates by ethnicity and gender for the Beginning of Grade 1 TPRI tasks. For the End of Grade 1 tasks, several tasks have adequate to poor reliabilities for the Hispanic and White students, but sample sizes are small for these tasks. There are also tendencies for gender difference showing better reliability for boys than girls on several End of Grade 1 tasks. For neither ethnicity nor gender is any specific pattern apparent and the results most likely reflect the relatively small samples on which to base the reliability estimates. Reliability of Grade 2 Forms Table 8 reports the reliabilities for the overall Grade 2 sample, while Tables 9-10 break down the results by ethnicity and gender, respectively. In Table 8, the Beginning of Grade 2 form shows excellent reliability for the screening task. The reliabilities for the different tasks are all in the good to excellent range except for Final Consonant Substitution (.58) and Orthographic Patterns, Conventions, and Past Tense (.52). For the End of Grade 2 administration, Median Vowel Substitution (.58), Spelling of Long Vowels (.55), one comprehension story (.56) and one expository text (.54) had adequate reliabilities. None were in the poor range. Table 9 shows more variability in reliability estimates for ethnicity than the kindergarten and Grade 1 forms. However, sample sizes are not adequate for many of the tasks. No particular pattern is apparent. Similar variability is apparent in for the gender analyses (Table 10), but the

8 8 differences in reliability tend to occur on tasks with low sample sizes and no pattern is apparent that would indicate systematic bias. Summary: Reliability The reliability analyses consistently show excellent reliabilities for the screens and good to excellent reliabilities for most TPRI tasks at each grade level. As we reported in the 1998 technical report, Book and Print Awareness tasks have weak reliability and some reliability estimates are in the Adequate range for some of the listening and reading comprehension tasks. There are a few end of Grade 2 tasks for which reliability is only adequate, possible reflecting ceiling effects where the tasks are not sufficiently difficult. This is not surprising as the form was designed for the beginning of Grade 2. There are some differences in reliability by ethnicity and gender, but no consistent pattern is apparent. The inconsistencies are most likely related to the small number of items and low sample sizes for some of the gender and ethnicity analyses. While reporting reliability by gender and ethnicity is important for identifying patterns of less than adequate reliability, the key question is whether specific items function differently by virtue of ethnicity and gender. This is addressed in the next section. Differential Item Functioning Analysis Item response theory (IRT) models were employed for detecting differential item bias. IRT permits comparisons of item functioning between groups in terms of the probability that performance of that item for each group is different at the same level of ability. To conduct these analyses, an IRT model is constructed that estimates item parameters for each group of interest (e.g., ethnicity), and compares the parameters obtained for this model to a model in which group membership is ignored. If the models are not different, this indicates that the differences between groups on an item are best explained solely by ability and that group membership does not contribute to differential performance of an item. This would indicate that the item is not biased. It is important to recognize that some items will show evidence for differential item

9 9 functioning (DIF) solely by chance. The goal is to keep the total number of items indicating DIF below 5%. Tables summarize the number of items showing DIF by gender and ethnicity (White/Black, White/Hispanic). Of the 208 items on the Kindergarten form of the TPRI, four showed gender differences, especially on the Blending Phonemes task. Five showed differences between Blacks and Whites, largely on the Middle of Year Phonological Awareness screen. However, three of these favored Whites and two favored Blacks. There were nine items showing DIF for the White/Hispanic analysis, but again the pattern did not show any consistent preference for one or the other ethnicity. The overall rate was below 5% for each analysis. Table 12 shows only two of 237 items demonstrating DIF between genders. In the White/Black comparison, 10 items show DIF. Of three words for the Beginning of Year word reading screen, one favored Whites and the other two favored Blacks. For Whites/Hispanics, there were six items showing DIF, with no consistent pattern of bias. The overall rates of DIF for each analysis were below 4%. A similar pattern is apparent for Grade 2 (Table 13). For the 150 items, 3 show DIF by gender; one for the White/Black comparison; and two for the White/Hispanic comparison. The overall rates are 2% or less for each of these analyses. Summary: Item Bias There is no evidence for systematic item bias by virtue of ethnicity or gender for any of the forms of the TPRI. The overall rates of DIF are uniformly below 5% and no consistent patterns of bias are apparent. Teacher Responses to the TPRI A total of 287 (of a possible 299) teachers completed the different surveys, representing a participation rate of over 95% (Table 14). Table 15 shows that the teachers taught classes of about 20 students, though this was larger and more variable for teachers in the suburban district.

10 10 The teachers are predominantly women (97%). In the rural, small city, and suburban districts, most teachers were White. The urban districts were more comparable in the ethnicity distribution. Teachers generally held B.A. degrees, especially in the rural and small city districts. There was a tendency for more teachers in the suburban and urban districts to have advanced degrees. Most teachers (44%) had 1-5 years teaching experience, but the range was 0-30 and all levels of experience were represented (Table 16). When administering the TPRI, Table 17 shows that over half completed it within one week (55%), with an additional 23% completing it within two weeks. Most other teachers completed it within 3 weeks, with a few mostly in urban districts requiring more time. Table 18 shows that about half administered the screens and inventory simultaneously, while the remainder administered the screen first before administering any of the inventory tasks. This is important because teachers will save time if the screening is done first. This may have reflected a requirement in one large district that teachers administer the screen and the inventory to all children, counter to the recommendations for administration of the TTRI in the Teachers Guide. Most teachers administered the inventory in the middle of the year to at least a subset of students (Table 19). The most common place to administer the TPRI was in the classroom with other students present (42%), with many completing it outside the classroom in a quieter environment (15%). In additional data not in the tables, most teachers administered the TPRI themselves (84%), found the TPRI easy to administer with easily accessible results (74%), and that the TPRI booklets were stored in the teacher s room (67%) or in an easily accessible room (15%). Table 21 shows that 88% of teachers received training that usually exceeded an hour, with the largest number receiving 4-6 hours. Training consisted of hands-on experience for over half the teachers (Table 21). Teachers were asked to respond to eight questions concerning their knowledge of the TPRI. Most teachers responded correctly to these questions, with a range of 60-91% correct, including: number of items in specific tasks (60%), what to do when a child could not accurately

11 11 read the stories (68%), to whom the screen is administered (86%), identifying different parts of the TPRI (91%), rules for practice items (89%), when items can be repeated (63%), how to administer the instructions (84%), and the role of coaching and praise (88%). These results show that teachers were knowledgeable about TPRI administration. The teachers responded to specific questions about the TPRI, completing ratings on a 1 to 5 scale where 1 indicated complete dissatisfaction and 5 complete satisfaction. A rating of 3 would be neutral. In all of these ratings, there is a consistent tendency for suburban teachers to feel less positive about all aspects of the TPRI, although most ratings are in the middle of the range and indicate overall neutrality. This pattern of responses is clearly apparent in Table 22, where suburban teachers were least happy with the flow chart, layout, and summary sheets. Ratings by teachers in urban, small city, and rural districts were more neutral. Similarly, in Table 23, teachers were largely neutral about the content of the TPRI, with suburban teachers largely neutral, but more negative. In Table 24, the same pattern is apparent for the five components of the TPRI, but teachers in urban, small city, and rural districts gave more positive evaluations of the inventory sections. Table 25 shows that rural, small city, and urban teachers were neutral about the TPRI, with suburban teachers more negative when asked to compare the TPRI with other reading assessments. This finding helps explain the consistently lower rating of suburban teachers: the TPRI had replaced an instrument which many suburban teachers had been using for assessment purposes in their district. Table 26 summarizes responses to a number of specific questions on the survey about the TPRI. Suburban teachers were consistently more negative on virtually all these items, with few differences among urban, rural, and small city teachers. The latter were particularly positive about the use of the TPRI for identifying student strengths and weaknesses and their ability to administer it correctly. All teachers indicated that their responses were not influenced by the reporting of student scores to the school board and did not feel strongly that their teaching would

12 12 be evaluated based on TPRI results. Most were neutral concerning training, clarity of directions, materials, and instructional planning. The response of the subset of 46 teachers who were also interviewed concerning different components of the TPRI are summarized in Tables In responding to questions about training, Table 27 shows that most felt that training was adequate, though urban teachers were more likely to indicate that it was not adequate. Most felt the TPRI booklets were easy to access (Table 27). The most common justifications for early reading assessments provided to the teachers (Table 28) involved the evaluation of strengths and weaknesses, the presence of a state mandate, and early identification. When asked about factors that made it difficult to use the TPRI, a diversity of responses emerged (Table 29). Lack of knowledge and experience, and late receipt of the booklets, were the most commonly cited constraints, with suburban teachers also indicating that a curriculum was already in place. In Table 30, it is apparent that comparably small numbers of teachers completely liked or disliked the TPRI. The most common positive responses involved support for the teacher s informal assessment and the opportunity to see that students had made progress. The most common concern (Table 31) was that the TPRI was time consuming. The teachers also emphasized the need to receive the booklets in a timely manner at the beginning of the year and the need for substitutes to help with coverage of classes (Table 32). In response to other questions not shown in the tables, teachers were roughly equally divided in a) the utility of the TPRI for communicating with parents (46% yes); b) its utility for decisions about the need for tutoring (51%) yes); c) usefulness for planning for low performing students (50% yes); d) help in selecting reading materials (43% yes); and e) help in selecting reading objectives (46% yes). Teachers did not feel the TPRI was useful for deciding student referrals (80% no) or grouping students (68% no). Many teachers indicated that they did not receive the booklets early enough to plan instruction, including the formation of instructional groups. Table 33, for example, shows that of teachers who used the TPRI in some way (76%), half used it to

13 13 indicate a need for additional instruction or tutoring. Table 33 also indicates a broad range of potential uses of the TPRI. Summary Teacher responses to the TPRI in urban, rural, and small city districts were neutral to positive. Suburban teachers were consistently more neutral to negative, which may reflect the fact that the TPRI had replaced an assessment procedure previously in place in the district. In general, teachers indicated that directions were easier to follow and that the TPRI was most useful for evaluating student strengths and weaknesses. They felt comfortable with their ability to administer the TPRI, which was supported by the assessments of knowledge, and did not feel they would be evaluated based on TPRI results. The most significant concerns involved the amount of time required to administer the TPRI, although many did not follow the procedure recommended to save time, which is to administer the screen first and the inventory to those students who had not mastered the concepts on the screen. In one large district, teachers were required to administer the screen and the inventory. Teachers also expressed concern about the late receipt of the TPRI and the need for help covering classes when administering the TPRI. Conclusions With the exception of tasks involving Book and Print Awareness and listening/reading comprehension, the overall reliability of the1999 TPRI tasks meets commonly accepted standards for reliability. There is no evidence for significant item bias by ethnicity and gender. The response of teachers in the urban, small city, and rural districts was neutral to positive. The suburban district was consistently negative. These results indicate that revisions of the TPRI are indicated. We plan to change the Book and Print Awareness tasks to warm-up exercises, as reliability seems quite difficult. Specific items can be revised to improve the reliability of some tasks. A major revision of the approach to assessing listening and reading comprehension is needed. The use of copyrighted, authentic stories is not only cumbersome and expensive, but

14 14 does not produce consistently reliable tasks. This may reflect the difficulty of controlling the vocabulary level of the stories, which is not reflected in readability estimates. Based on informal responses with teachers over the year, we plan to write stories for which the difficulty level can be more tightly controlled. Finally, we plan to add additional training guides and material on intervention in response to suggestions from teachers and others who have interacted with us in the development of the TPRI.

15 15 Table 1. Number of districts, schools, and classrooms in the 1999 TPRI implementation study. Urban Suburban Small City Rural Total Districts Schools Classrooms

16 16 Table 2. Overall reliabilities for kindergarten tasks TPRI Kindergarten Middle of the Year N TOTAL ALPHA Screen 1: Letter Sound: Letter Sound Screen 2: Blending Onset-Phonemes Task 1: Book and Print Awareness Task 2: Rhyming Task 3: Blending Word Parts Task 4: Blending Phonemes Task 5: Detecting Initial Sounds Task 6: Detecting Final Sounds Task 7: Letter Name Identification Task 8: Letter to Sound Linking: Letter Identification Task 8: Letter to Sound Linking: Letter Sound Task 9: Comprehension Task 10: Comprehension Task 11: Comprehension End of the Year Screen 3: Letter Sound: Letter Sound Screen 4: Blending Onset-Rhymes and Phonemes Task 1: Book and Print Awareness Task 2: Rhyming Task 3: Blending Word Parts Task 4: Blending Phonemes Task 5: Detecting Initial Sounds Task 6: Detecting Final Sounds Task 7: Letter Name Identification Task 8: Letter to Sound Linking: Letter Identification Task 8: Letter to Sound Linking: Letter Sound Task 12: Comprehension Task 13: Comprehension Task 14: Comprehension

17 17 Table 3. Reliability by gender for kindergarten tasks. TPRI Kindergarten Ethnicity BLACK HISPANIC WHITE Middle of the Year N ALPHA N ALPHA N ALPHA Screen 1: Letter Sound: Letter Sound Screen 2: Blending Onset-Phonemes Task 1: Book and Print Awareness Task 2: Rhyming Task 3: Blending Word Parts Task 4: Blending Phonemes Task 5: Detecting Initial Sounds Task 6: Detecting Final Sounds Task 7: Letter Name Identification Task 8: Letter to Sound Linking: Letter Identification Task 8: Letter to Sound Linking: Letter Sound Task 9: Comprehension Task 10: Comprehension Task 11: Comprehension End of the Year Screen 3: Letter Sound: Letter Sound Screen 4: Blending Onset-Rhymes and Phonemes Task 1: Book and Print Awareness Task 2: Rhyming Task 3: Blending Word Parts Task 4: Blending Phonemes Task 5: Detecting Initial Sounds Task 6: Detecting Final Sounds Task 7: Letter Name Identification Task 8: Letter to Sound Linking: Letter Identification Task 8: Letter to Sound Linking: Letter Sound Task 12: Comprehension Task 13: Comprehension Task 14: Comprehension

18 18 Table 4. Reliabilities by gender for kindergarten tasks. TPRI Kindergarten MALE FEMALE Middle of the Year N ALPHA N ALPHA Screen 1: Letter Sound: Letter Sound Screen 2: Blending Onset-Phonemes Task 1: Book and Print Awareness Task 2: Rhyming Task 3: Blending Word Parts Task 4: Blending Phonemes Task 5: Detecting Initial Sounds Task 6: Detecting Final Sounds Task 7: Letter Name Identification Task 8: Letter to Sound Linking: Letter Identification Task 8: Letter to Sound Linking: Letter Sound Task 9: Comprehension Task 10: Comprehension Task 11: Comprehension End of the Year Screen 3: Letter Sound: Letter Sound Screen 4: Blending Onset-Rhymes and Phonemes Task 1: Book and Print Awareness Task 2: Rhyming Task 3: Blending Word Parts Task 4: Blending Phonemes Task 5: Detecting Initial Sounds Task 6: Detecting Final Sounds Task 7: Letter Name Identification Task 8: Letter to Sound Linking: Letter Identification Task 8: Letter to Sound Linking: Letter Sound Task 12: Comprehension Task 13: Comprehension Task 14: Comprehension

19 19 Table 5. Overall reliabilities for Grade 1 tasks. TPRI First Grade TOTAL Beginning of the Year N ALPHA SCREEN 1: LETTER SOUND: LETTER SOUND SCREEN 2: WORD READING TASK SCREEN 3: BLENDING PHONEMES TASK 1: BOOK AND PRINT AWARENESS TASK 2: RHYMING TASK 3: BLENDING WORD PARTS TASK 4: BLENDING PHONEMES TASK 5: DETECTING INITIAL SOUNDS TASK 6: DETECTING FINAL SOUNDS TASK 7: INITIAL CONSONANT SUBSTITUTION TASK 8: FINAL CONSONANT SUBSTITUTION TASK 9: MEDIAL VOWEL SUBSTITUTION TASK 10: INITIAL BLEND SUBSTITUTION TASK 11: FINAL BLEND SUBSTITUTION TASK 12: COMPREHENSION TASK 13: COMPREHENSION End of the Year SCREEN 4: WORD READING TASK SCREEN 5: BLENDING PHONEMES TASK 1: BOOK AND PRINT AWARENESS TASK 2: RHYMING TASK 3: BLENDING WORD PARTS TASK 4: BLENDING PHONEMES TASK 5: DETECTING INITIAL SOUNDS TASK 6: DETECTING FINAL SOUNDS TASK 7: INITIAL CONSONANT SUBSTITUTION TASK 8: FINAL CONSONANT SUBSTITUTION TASK 9: MEDIAL VOWEL SUBSTITUTION TASK 10: INITIAL BLEND SUBSTITUTION TASK 11: FINAL BLEND SUBSTITUTION TASK 16: COMPREHENSION TASK 17: COMPREHENSION

20 20 Table 6. Reliabilities by gender for Grade 1 tasks TPRI First Grade Ethnicity BLACK HISPANIC WHITE Beginning of the Year N ALPHA N ALPHA N ALPHA SCREEN 1: LETTER SOUND: LETTER SOUND SCREEN 2: WORD READING TASK SCREEN 3: BLENDING PHONEMES TASK 1: BOOK AND PRINT AWARENESS TASK 2: RHYMING TASK 3: BLENDING WORD PARTS TASK 4: BLENDING PHONEMES TASK 5: DETECTING INITIAL SOUNDS TASK 6: DETECTING FINAL SOUNDS TASK 7: INITIAL CONSONANT SUBSTITUTION TASK 8: FINAL CONSONANT SUBSTITUTION TASK 9: MEDIAL VOWEL SUBSTITUTION TASK 10: INITIAL BLEND SUBSTITUTION TASK 11: FINAL BLEND SUBSTITUTION TASK 12: COMPREHENSION TASK 13: COMPREHENSION End of the Year SCREEN 4: WORD READING TASK SCREEN 5: BLENDING PHONEMES TASK 1: BOOK AND PRINT AWARENESS TASK 2: RHYMING TASK 3: BLENDING WORD PARTS TASK 4: BLENDING PHONEMES TASK 5: DETECTING INITIAL SOUNDS TASK 6: DETECTING FINAL SOUNDS TASK 7: INITIAL CONSONANT SUBSTITUTION TASK 8: FINAL CONSONANT SUBSTITUTION TASK 9: MEDIAL VOWEL SUBSTITUTION TASK 10: INITIAL BLEND SUBSTITUTION TASK 11: FINAL BLEND SUBSTITUTION TASK 16: COMPREHENSION TASK 17: COMPREHENSION

21 21 Table 7. Reliabilities by gender for Grade 1 tasks. TPRI First Grade MALE FEMALE Beginning of the Year N ALPHA N ALPHA SCREEN 1: LETTER SOUND: LETTER SOUND SCREEN 2: WORD READING TASK SCREEN 3: BLENDING PHONEMES TASK 1: BOOK AND PRINT AWARENESS TASK 2: RHYMING TASK 3: BLENDING WORD PARTS TASK 4: BLENDING PHONEMES TASK 5: DETECTING INITIAL SOUNDS TASK 6: DETECTING FINAL SOUNDS TASK 7: INITIAL CONSONANT SUBSTITUTION TASK 8: FINAL CONSONANT SUBSTITUTION TASK 9: MEDIAL VOWEL SUBSTITUTION TASK 10: INITIAL BLEND SUBSTITUTION TASK 11: FINAL BLEND SUBSTITUTION TASK 12: COMPREHENSION TASK 13: COMPREHENSION End of the Year SCREEN 4: WORD READING TASK SCREEN 5: BLENDING PHONEMES TASK 1: BOOK AND PRINT AWARENESS TASK 2: RHYMING TASK 3: BLENDING WORD PARTS TASK 4: BLENDING PHONEMES TASK 5: DETECTING INITIAL SOUNDS TASK 6: DETECTING FINAL SOUNDS TASK 7: INITIAL CONSONANT SUBSTITUTION TASK 8: FINAL CONSONANT SUBSTITUTION TASK 9: MEDIAL VOWEL SUBSTITUTION TASK 10: INITIAL BLEND SUBSTITUTION TASK 11: FINAL BLEND SUBSTITUTION TASK 16: COMPREHENSION TASK 17: COMPREHENSION

22 22 Table 8. Overall reliabilities for Grade 2 tasks. TPRI Grade 2 Beginning of the Year N TOTAL Screen 1: Word Reading Task Task 1: Initial Consonant Substitution Task 2: Final Consonant Substitution Task 3: Medial Vowel Substitution Task 4: Initial Blend Substitution Task 5: Final Blend Substitution Task 6: Spelling of CVC and CVCe Words Task 7: Spelling of Long Vowels Task 8: Orthographic Patterns, Conventions, and Past Tense Task 9: Orthographic Patterns, Conventions, and Inflectional Endings Task 10: Comprehension Task 11: Comprehension End of the Year Task 1: Initial Consonant Substitution Task 2: Final Consonant Substitution Task 3: Medial Vowel Substitution Task 4: Initial Blend Substitution Task 5: Final Blend Substitution Task 6: Spelling of CVC and CVCe Words Task 7: Spelling of Long Vowels Task 8: Orthographic Patterns, Conventions, and Past Tense Task 9: Orthographic Patterns, Conventions, and Inflectional Endings Task 14: Comprehension Task 15: Comprehension Task 16: Expository Test Task 17: Expository Test

23 23 Table 9. Reliabilities by ethnicity for Grade 2 tasks. TPRI Second Grade BLACK HISPANIC WHITE Beginning of the Year N Alpha N Alpha N Alpha Screen 1: Word Reading Task Task 1: Initial Consonant Substitution Task 2: Final Consonant Substitution Task 3: Medial Vowel Substitution Task 4: Initial Blend Substitution Task 5: Final Blend Substitution Task 6: Spelling of CVC and CVCe Words Task 7: Spelling of Long Vowels Task 8: Orthographic Patterns, Conventions, and Past Tense Task 9: Orthographic Patterns, Conventions, and Inflectional Endings Task 10: Comprehension Task 11: Comprehension End of the Year Task 1: Initial Consonant Substitution Task 2: Final Consonant Substitution Task 3: Medial Vowel Substitution Task 4: Initial Blend Substitution Task 5: Final Blend Substitution Task 6: Spelling of CVC and CVCe Words Task 7: Spelling of Long Vowels Task 8: Orthographic Patterns, Conventions, and Past Tense Task 9: Orthographic Patterns, Conventions, and Inflectional Endings Task 14: Comprehension Task 15: Comprehension Task 16: Expository Test Task 17: Expository Test......

24 24 Table 10. Reliabilities by gender for Grade 2 tasks. TPRI Grade 2 MALE FEMALE Beginning of the Year N Alpha N Alpha Screen 1: Word Reading Task Task 1: Initial Consonant Substitution Task 2: Final Consonant Substitution Task 3: Medial Vowel Substitution Task 4: Initial Blend Substitution Task 5: Final Blend Substitution Task 6: Spelling of CVC and CVCe Words Task 7: Spelling of Long Vowels Task 8: Orthographic Patterns, Conventions, and Past Tense Task 9: Orthographic Patterns, Conventions, and Inflectional Endings Task 10: Comprehension Task 11: Comprehension End of the Year Task 1: Initial Consonant Substitution Task 2: Final Consonant Substitution Task 3: Medial Vowel Substitution Task 4: Initial Blend Substitution Task 5: Final Blend Substitution Task 6: Spelling of CVC and CVCe Words Task 7: Spelling of Long Vowels Task 8: Orthographic Patterns, Conventions, and Past Tense Task 9: Orthographic Patterns, Conventions, and Inflectional Endings Task 14: Comprehension Task 15: Comprehension Task 16: Expository Test Task 17: Expository Test

25 25 Table 11. Kindergarten TPRI differential item functioning results by task. Number of items with DIF Subtest Gender White/Black White/Hispanic Total # Items Screen 1: Letter Sound MOY *Screen 2: Blending Onset-Phonemes MOY Screen 3: Letter Name EOY Screen 3: Letter Sound EOY Screen 4: Blending Onset Rhyme and Phonemes EOY Task 1: Book and Print Awareness EOY Task 1: Book and Print Awareness MOY Task 2: Rhyming EOY Task 2: Rhyming MOY Task 3: Blending Word Parts EOY Task 3: Blending Word Parts MOY Task 4: Blending Phonemes EOY Task 4: Blending Phonemes MOY Task 5: Detecting Initial Sounds EOY Task 5: Detecting Initial Sounds MOY Task 6: Detecting Final Sounds EOY Task 6: Detecting Final Sounds MOY Task 7: Letter Name Identification EOY Task 7: Letter Name Identification MOY Task 8: Letter to Sound Linking - Letter Identification EOY Task 8: Letter to Sound Linking - Letter Identification MOY Task 8: Letter to Sound Linking - Letter Sound EOY Task 8: Letter to Sound Linking - Letter Sound MOY Task 9: Listening Comprehension MOY Task 10: Listening Comprehension MOY Task 11: Listening Comprehension MOY Task 12: Listening Comprehension MOY Task 13: Listening Comprehension MOY Task 14: Listening Comprehension MOY Percentage of Total # of items with DIF 1.9% 2.4% 4.3% 208 Note. MOY = Middle of the Year; EOY = End of the Year

26 26 Table 12. First Grade TPRI differential item functioning results by task. Number of items with DIF Subtest Gender White/Black White/Hispanic Total # Items Screen 1: Letter Sound BOY *Screen 2: Word Reading BOY Screen 3: Blending Phonemes BOY **Screen 4: Word Reading EOY Screen 5: Blending Phonemes EOY Task 1: Book and Print Awareness BOY Task 1: Book and Print Awareness EOY Task 1: Book and Print Awareness MOY Task 2: Rhyming BOY Task 2: Rhyming EOY Task 2: Rhyming MOY Task 3: Blending Word Parts BOY Task 3: Blending Word Parts EOY Task 3: Blending Word Parts MOY Task 4: Blending Phonemes BOY Task 4: Blending Phonemes EOY Task 4: Blending Phonemes MOY Task 5: Detecting Intial Sounds BOY Task 5: Detecting Intial Sounds EOY Task 5: Detecting Intial Sounds MOY Task 6: Detecting Final Sounds BOY Task 6: Detecting Final Sounds EOY Task 6: Detecting Final Sounds MOY Task 7: Initial Consonant Substitution BOY Task 7: Initial Consonant Substitution EOY Task 7: Initial Consonant Substitution MOY Task 8: Final Consonant Substitution BOY Task 8: Final Consonant Substitution BOY Task 8: Final Consonant Substitution BOY Task 9: Medial Vowel Substitution BOY Task 9: Medial Vowel Substitution MOY Task 9: Medial Vowel Substitution EOY Task 10: Initial Blend Substitution BOY Task 10: Initial Blend Substitution EOY Task 10: Initial Blend Substitution MOY Task 11: Final Blend Substitution BOY Task 11: Final Blend Substitution EOY Task 11: Final Blend Substitution MOY Task 12: Comprehension BOY Task 13: Comprehension BOY Task 14: Comprehension MOY Task 15: Comprehension MOY Task 16: Comprehension EOY Task 17: Comprehension EOY Percentage of Total # of items with DIF 0.84% 3.80% 1.69% 237 Note. MOY = Middle of the Year; EOY = End of the Year

27 27 Table 13. Second Grade TPRI differential item functioning results by task. Number of items with DIF Subtest Gender White/Black White/Hispanic Total # of items Final Blend Substitution BOY Final Blend Substitution EOY Final Blend Substitution MOY Final Consonant Substition BOY Final Consonant Substition EOY Final Consonant Substition MOY Initial Blend Substitution BOY Initial Blend Substitution EOY Initial Blend Substitution MOY Initial Consonant Substitution BOY Initial Consonant Substitution EOY Initial Consonant Substitution MOY Medial Vowel Substitution BOY Medial Vowel Substitution EOY Medial Vowel Substitution MOY Orthographic Patterns - Inflectional BOY Orthographic Patterns - Inflectional EOY Orthographic Patterns - Past tense BOY Orthographic Patterns - Past tense EOY Orthographic Patterns - Past tense MOY Spelling of CVC BOY Spelling of CVC EOY Spelling of CVC MOY Spelling of Long Vowels BOY Spelling of Long Vowels EOY Spelling of Long Vowels MOY Comprehension BOY Comprehension EOY Comprehension BOY Comprehension EOY Percentage of Total # of items with DIF 1.9% 2.4% 4.3% 208 Note. MOY = Middle of the Year; EOY = End of the Year

28 28 Table 14. Number of teachers providing surveys by district type. DISTRICT TYPE NUMBER OF TEACHERS TOTAL Rural 50 Small City 59 Suburban 58 Urban 120 TOTAL 287

29 Table 15. Characteristics of schools, teachers, and students by district type. DISTRICT TYPE NUMBER OF SCHOOLS NUMBER OF TEACHERS AVERAGE NUMBER OF STUDENTS Grade Level Gender Ethnicity Degree Kindergarten First Grade Second Grade Male Female African-American Hispanic Caucasian Other Missing Values B.A. M.A. Rural (STD = 4.6) Small City (STD = 5.1) Suburban (STD = 8.3) Urban (STD = 2.7) TOTAL (STD = 5.6)

30 Table 16. Number of years teaching experience by district type. DISTRICT TYPE Number of Years Teaching Experience years 6-10 years years year years 26+ years Rural Small City Suburban Urban TOTAL

31 Table 17. Time required to administer the TPRI. DISTRICT TYPE I administered all parts of the TPRI to all students within: 1 week 2 weeks 3 weeks 4 weeks More than 4 weeks No Response Rural Small City Suburban Urban TOTAL

32 Table 18. Teacher administration practices by district type. 32 DISTRICT TYPE Please select the TPRI administration procedure that best describes what you did: Administered screening before administering any inventory portions Administered screening and inventory portions at the same time No Response Rural Small City Suburban Urban TOTAL

33 Table 19. Number of students receiving the TPRI by district type. 33 DISTRICT TYPE If you are a first or second grade teacher, for how many students did you administer the TPRI during the middle of the year? 0 1 to 5 6 to to to 20 More than 20 No Response Rural Small City Suburban Urban TOTAL

34 Table 20. Location of TPRI administrations by district type. 34 DISTRICT TYPE Where did you administer most of the TPRI assessments? In your classroom with other students in the room In your classroom without other students in the room In a place outside your room with few or no distractions In a place outside your room with more than a few distractions No Response Rural Small City Suburban Urban TOTAL

35 Table 21. Training in TPRI administration by district type. 35 DISTRICT TYPE TRAINED TO ADMINISTER TPRI NUMBER OF HOURS TRAINED TRAINING INCLUDE HANDS ON PRACTICE YES NO 0-1 Hours 2-3 Hours 4-6 Hours 7-8 Hours Missing Values YES NO Rural Small City Suburban Urban TOTAL

36 Table 22. Teacher responses to the organization of the TPRI by district type. 36 DISTRICT TYPE STATISTIC How do you feel about the following organizational parts of the TPRI? Flow Chart Directions Layout Scoring Summary Sheets Rural Mean SD Small City Mean SD Suburban Mean SD Urban Mean SD

37 Table 23. Teacher responses to the content of the TPRI. 37 DISTRICT TYPE STATISTIC Rate the quality of the content from the sections of the TPRI? Phonemic Awareness Tasks Graphophonemic Knowledge Tasks Word Reading Comprehension Rural Mean SD Small City Mean SD Suburban Mean SD Urban Mean SD

38 Table 24. Teacher response to the different parts of the TPRI by district type. 38 DISTRICT TYPE STATISTIC The following parts of the TPRI were useful: Screening Portion Book and Print Awareness Phonemic Awareness Graphophonemic Knowledge Spelling Comprehension Rural Mean SD Small City Mean SD Suburban Mean

39 Table 25. Teacher comparisons of the TPRI to other reading instruments by district type. 39 DISTRICT TYPE STATISTIC If you are familiar with another reading assessments, compare it to the TPRI on the following characteristics: Ease of Administration Usefulness for planning instruction Identification to students' reading strengths and weaknesses Worthwhile use of instructional time Rural Mean SD Small City Mean SD Suburban Mean SD Urban Mean SD