AIMSweb National Norms Technical Documentation

Transcription

1 AIMSweb National Norms Technical Documentation AIMSweb introduced National Norms in the fall of 2011 to provide benchmarkmeasure norms that reflect n ational Norms are offered for English-language measures in reading, math, and language arts at the standard grade levels and seasons at which these measures are used for benchmarking, as summarized in Table 1. Currently, National Norms are not available for Spanish-language measures or at preschool or high-school levels because nationally-representative samples have not yet been collected for those measures or grade levels. the performance of the Table 1. Measures and grade levels/seasons for National Norms at grades K through 8. Measure Simultaneously, AIMSweb Reading Curriculum-Based Measurement (R-CBM) implemented the midinterval method of calculating percentiles for all norms, including local norms as well as the National Norms. Grade Range From To (Winter) (Fall) Kindergarten (Winter) Kindergarten (Winter) (Fall) (Spring) Spelling MAZE (TEL): Letter Naming Fluency (LNF) Phoneme Segmentation Fluency (PSF) Nonsense Word Fluency (NWF) Written Expression (CWS) The midinterval method Math Computation (M-COMP) is a superior technique for Math Concepts & Applications (M-CAP) Grade 2 describing where a raw Test of Early Numeracy (TEN) Oral Counting (OCM) Number Identification (NIM) Missing Number (MNM) Quantity Discrimination (QDM) (Spring) (Spring) (Spring) (Spring) score falls in a norm-sample distribution AIMSweb.com AIMSweb and Pearson, design for Psi, and PsychCorp are trademarks, in the U.S. and/or other countries, of Pearson Education, Inc. or its affiliate(s) /121 national student population

2 Method Data for the National Norms were selected from the AIMSweb database, which includes all of the measure scores entered into the system by AIMSweb users. The sampling unit was the grade level of a school. In order for a norm sample to accurately represent the student population, it must not only be appropriately stratified along demographic variables, but it must also represent the full range of student performance. Although many schools administer AIMSweb benchmark measures as universal screeners, some schools assess only a subset of their student population, such as those considered to be at risk of not achieving a favorable academic outcome. Thus, drawing a sample without considering the likelihood of universal screening would be likely to result in an overrepresentation of lower-achieving students. For this reason, the data for each National Norm sample came only from schools that had conducted universal screening. In order to identify schools that had done universal screening with a measure at a grade, the AIMSweb team compared the number of benchmark administrations (at each season) with the total number of students enrolled in that grade, as reported by the National Center for Educational Statistics (NCES). If at least 95% of enrolled students had taken the AIMSweb benchmark measure, then that grade at that school was considered eligible for inclusion in the National Norms sample for that measure. If fewer than 95% of the students at that grade had scores on the measure, then data from that grade at that school were not eligible for inclusion. That is, the scores from a grade at a school were treated as eligible or ineligible as a complete set. As a further requirement, only the scores from students who had taken the AIMSweb measure at all three benchmarking periods during the year were retained. (Written Expression was an exception at Grades 1, 6, 7, and 8.) This simplified the sampling process without sacrificing a significant number of cases, and it strengthens some analyses and interpretations of the data by reducing variations across benchmark periods due to sampling differences. From the eligible data at a grade, the AIMSweb team selected the final norm sample to match the national student population by gender, ethnicity, and socioeconomic status (free/reduced lunch), according to demographic targets based on NCES data. Region was not included as a stratification variable because a preponderance of AIMSweb data meeting the inclusion criteria came from the Midwest and South regions, and the AIMSweb team judged that it was better to have larger samples stratified by gender, ethnicity, and SES than smaller samples that were also stratified by region. At each grade, the NCES-provided percentage of a school s students in a demographic category (e.g., American) was multiplied by the number of AIMSweb scores to yield an estimated number of students in that demographic category. The norm sample for each measure at each grade was constructed to be as large as possible while maintaining a close correspondence to national demographic percentages. For most measures, data from the school year were sufficient to provide adequate sample sizes. However, for Spelling, Written Expression, and (at some grades) MAZE it was necessary to use data from in order to obtain sufficiently large samples with good demographic representation. Table 2 reports the number of students in the norm sample for each measure at each benchmark period, per grade. Tables 2 and 3 report the demographic characteristics of each norm sample. Table 2. Number of cases (per benchmark period) in each norm sample Test of Early Numeracy Grade R-CBM MAZE LNF LSF PSF NWF Spelling WE M-CAP M-COMP OCM NIM MNM QDM K 32,597 50,586 48,100 42,104 18,790 20,746 12,152 11, ,158 3,513 71,316 59,372 39,310 25, ,250* 6,563 12,792 13,841 16,824 18, ,282 13, ,818 14, ,570 21, ,484 14,033 15, ,320 24, ,267 11,335 18, ,373 25, ,131 11,070 18, ,246 11, ,936* 3,163 4, ,468 7, * 3, ,048 6, * 2,746 1,921 * Average across seasons. Ranges: (1,423 to 2,689); Grade 6 (1,443 to 2,274); Grade 7 (644 to 1,159); (183 to 407) 2 AIMSweb National Norms Technical Documentation

3 Table 3. Demographic representation of the norm samples (percentages) Reading Curriculum-Based Measurement (R-CBM) MAZE Measure Female Male Kindergarten LNF LSF PSF NWF LNF LSF PSF NWF AIMSweb National Norms Technical Documentation

4 Table 3 (continued). Demographic representation of the norm samples (percentages) Spelling Written Expression (Correct Word Sequences) 1* * * * Math Concepts & Applications (M-CAP) * Average across seasons. Ranges: (1,423 to 2,689); Grade 6 (1,443 to 2,274); Grade 7 (644 to 1,159); (183 to 407) 4 AIMSweb National Norms Technical Documentation

5 Table 3 (continued). Demographic representation of the norm samples (percentages) Math Computation (M-COMP) Measure Female Male Kindergarten OCM NIM MNM QDM OCM NIM MNM QDM Midinterval percentile norms were calculated separately for each norm sample. Only minor smoothing was applied to the growth curves across benchmark periods (mostly at the top or the bottom of the percentile range) because the samples are the same at all periods during the year and because the probes within a year are similar but not identical in difficulty. Results The raw-score distributions in the National Norms samples tend to be about the same as, or slightly higher than, the Aggregate score distributions. This is consistent with expectation because the Aggregate sample, which includes all AIMSweb users, has a higher proportion of low-achieving students than the National Norms sample which includes only schools doing universal screening. Table 4 compares raw-score means and standard deviations in the National Norm and Aggregate samples for each measure and grade. 5 AIMSweb National Norms Technical Documentation

6 Table 4. Means and standard deviations of raw scores in the Aggregate and National Norm samples, by measure and grade R-CBM MAZE Spelling Written Expression M-CAP M-COMP Grade (Winter) Aggreg National Aggreg National Aggreg National Aggreg National Aggreg National Aggreg National 1 43(35) 47(36) 5(6) 5(5) 40(12) 42(10) 6(7) 7(7) 25(12) 27(12) 2 86(39) 90(38) 11(7) 11(7) 51(14) 53(13) 16(12) 18(13) 17(9) 17(9) 30(11) 30(11) 3 106(42) 110(41) 15(7) 16(7) 85(21) 87(23) 25(14) 26(15) 11(6) 11(6) 40(16) 41(16) 4 121(41) 126(40) 20(9) 21(8) 100(23) 103(20) 33(17) 35(16) 16(8) 17(7) 42(16) 43(16) 5 133(43) 139(41) 22(9) 23(9) 110(24) 117(21) 40(18) 40(17) 11(6) 12(6) 23(15) 25(15) 6 146(43) 154(40) 26(10) 29(10) 114(26) 118(22) 45(19) 47(19) 17(9) 18(9) 27(14) 28(14) 7 146(42) 154(38) 24(10) 27(9) 113(29) 120(22) 49(21) 51(18) 15(8) 16(8) 29(16) 28(15) 8 147(39) 153(34) 22(9) 24(9) 116(27) 123(21) 53(21) 55(19) 12(8) 12(7) 23(14) 24(13) Table 4 (continued). Means and standard deviations of raw scores in the Aggregate and National Norm samples, by measure and grade Letter Naming Letter Sound Phoneme Seg. Nonsense Word Period Aggreg National Aggreg National Aggreg National Aggreg National KINDERGARTEN GRADE 1 Fall 19(16) 22(16) Winter 39(18) 43(17) 25(16) 28(16) 27(18) 29(19) 24(19) 27(19) Spring 49(18) 52(18) 37(16) 39(16) 43(19) 46(20) 38(23) 43(24) Fall 45(17) 46(16) 30(15) 31(15) 37(17) 39(17) 34(24) 39(25) Winter 48(16) 50(16) 56(30) 61(32) Spring 70(33) 75(35) Test of Early Numeracy Oral Counting Number Identification Missing Number Quantity Disc. Period Aggreg National Aggreg National Aggreg National Aggreg National KINDERGARTEN GRADE 1 Fall 39(21) 41(21) 30(19) 31(18) 6(6) 6(6) 11(9) 12(9) Winter 58(20) 63(21) 44(15) 46(13) 11(6) 11(6) 18(9) 19(9) Spring 69(19) 76(19) 51(10) 52(9) 14(6) 15(6) 23(8) 24(7) Fall 67(18) 71(19) 40(17) 41(17) 11(6) 12(6) 20(10) 21(10) Winter 76(16) 83(16) 56(15) 59(15) 17(6) 18(6) 28(8) 30(9) Spring 81(14) 89(14) 59(14) 64(14) 19(6) 20(6) 30(8) 34(7) Because of these differences in score distributions, AIMSweb users will find that percentile scores tend to be slightly lower with the National Norms than with the previous Aggregate Norms. 6 AIMSweb National Norms Technical Documentation