Spring 2017 Center for Urban Education Leadership University of Illinois at Chicago Big changes occurred in student achievement, student demographics and district resources between 2001 and 2016 in the State of Illinois. This report summarizes key changes at grades three through eight in Elmhurst and compares them with changes that occurred in suburban Chicagoland and the state as a whole between 2001 and 2016. CONTENTS Pages 1-4 National data on achievement, socio-economic status and race Pages 5-9 Changes in student demographics in Elmhurst and the State of Illinois, and the close association between standardized achievement and low-income enrollments Pages 10-14 Background information for reading the charts and tables in this report Pages 15-19 Achievement details for grades 3-8 in Elmhurst from 2001 through 2016 SOURCES How is Elmhurst District 205 Doing Compared with the Rest Suburban Chicagoland and the State as a Whole? Pages 20-21 Changes in composite achievement in Elmhurst and 54 other districts in Illinois Large Unit District Association (LUDA) Unless otherwise noted, all of the data contained in this report are derived from publicly-available information posted now or in the past at ftp://ftp.isbe.net/schoolreportcard/ and https://iirc.niu.edu/classic/default.aspx by the Illinois State Board of Education UIC Making Good on the Promise of Public Education Elmhurst
Standardized Achievement and Socio-Economic Status are Closely Connected in Most American School Districts The scatter plot on page 2 tells two big stories about American public schools: There is a powerful connection between standardized achievement and measures of socio-economic status (SES). Where average SES is high, even the lowest-achieving districts score close to two grade levels higher than the highestachieving districts where average SES is low. At most points on the SES continuum, there is difference of 2 to 3 grade equivalents between the highest and lowest achieving districts. Since average SES is more or less the same at each point on the SES continuum, something other than SES has got to be causing the differences. Each point on the scatter plot represents one of over three thousand school districts from a recent study of achievement, socioeconomic status and race in American schools*. The vertical axis shows average achievement scores for grades 3-8 from 2009 through 2013 (shown in grade equivalents). The horizontal axis shows the average socioeconomic status of families in each district. The point labeled Elmhurst 205 shows the intersect between average achievement and average socio-economic status in the district between 2009 and 2013. During those years: Achievement in Elmhurst 205 was well above average for districts with similar measures of socio-economic status Socio-economic status in Elmhurst 205 was well above the national average and achievement was 1.9 grade equivalents above the average for all school districts nationwide *Reardon, S.F., Kalogrides, D., & Shores, L. (2017) The Geography of Racial/Ethnic Test Score Gaps (CEPA Working Paper No. 16-10) Retrieved from Stanford Center for Education Policy Analysis: http://cepa.stanford.edu/wp16-10 Center for Urban Education Leadership, University of Illinois at Chicago Page 1
Relationship between Average District Achievement and the Average Socio-Economic Status of Students Families in Elmhurst 205 and over 3,000 Other U.S. Public School Districts: 2009-2013 3 Years ABOVE Factors included in this study s measure of socio-economic status include: 2 Years ABOVE 1 Year ABOVE Elmhurst 205 Family income Percent of parents with a college degree Percent of single parents Percent food stamp eligibility Percent unemployed AT GRADE LEVEL 1 Year BELOW 2 Years BELOW 3 Years BELOW Higher SOCIO-ECONOMIC STATUS Lower SOURCE: Rich, Motoko, Cox, Amanda and Block, Matthew. Money, Race and Success: How Your School Compares in The Upshot, New York Times April 29, 2016 from Reardon, Sean F., Kalogrides, Demetra and Shores, Ken The Geography of Racial-Ethnic Test Score Gaps at https://www.nytimes.com/interactive/2016/04/29/upshot/money-race-and-success-how-your-schooldistrict-compares.html?action=click&contentcollection=upshot®ion=rank&module=package&version=highlights&contentplacement=1&pgtype=sectionfront&smid=tw-upshotnyt&smtyp=cur&_r=2 Center for Urban Education Leadership, University of Illinois at Chicago Page 2
In Most American School Districts, Standardized Achievement and Socio-Economic Status are Closely Associated with Race The scatter plot on page 4 below illustrates how unsuccessful most districts have been at breaking the close association between race, SES and achievement in American schools. But it also illustrates that school effectiveness... what schools and districts do to improve their impact... varies a lot across districts with similar demographics. The good news on page 4 is that improvements in school effectiveness could reasonably increase achievement in most districts by between 0.75 and 2.0 grade-equivalents. The circles on page 4 describe average achievement and average SES in all U.S. school districts that have at least 100 white, 100 Latino and 100 black students in each of grades 3, 4, 5, 6, 7 and 8. Green circles represent the black population in each district. Blue circles represent Latino populations and pink circles represent white populations. Larger circle sizes reflect groups with larger enrollments; smaller circle sizes denote groups with smaller enrollments. Like the scatter plot on page 2, this one tells two important stories: Within-district differences of achievement and SES are powerfully associated with race. On average, white students in most of the districts shown scored at or above grade level and were in the upper third of the SES continuum. Average Latino achievement in most districts was between grade level and two years below grade level, and average SES was in the middle third of the continuum. Average black achievement was between one and two years below grade level, and average SES was in the lowest half of the SES continuum At all points on the SES continuum, achievement differences WITHIN racial groups vary by as much as three grade equivalents from one district to another. Page 4 highlights two districts where average SES for black, Latino and white students is roughly similar, but where achievement in each group is much higher in one district than it is in the other: o Average white achievement in Charlotte-Mecklenburg is 3.0 grade equivalents higher than it is in Simi Valley o Average black achievement is 1.6 grade equivalents higher, and average Latino achievement is 0.9 grade equivalents higher in Charlotte-Mecklenburg, even though average SES among black and Latino families is lower there than in Simi Valley. These data already control for race and SES. The differences they highlight are measures of school effectiveness. Center for Urban Education Leadership, University of Illinois at Chicago Page 3
Achievement, Socio-Economic Status and Race In a Broad Sampling of U.S. Public School Districts 3 Years ABOVE 2 Years ABOVE 1 Year ABOVE AT GRADE LEVEL Charlotte-Mecklenburg, NC Average Achievement Above/Below Black -0.9 Latino -0.6 White +2.9 Black Latino White/non-Latino 1 Year BELOW 2 Years BELOW Simi Valley, CA Average Achievement Above/Below Black -2.5 Latino -1.5 White -0.1 3 Years BELOW Higher SOCIO-ECONOMIC STATUS Lower SOURCE: Rich, Motoko, Cox, Amanda and Block, Matthew. Money, Race and Success: How Your School Compares in The Upshot, New York Times April 29, 2016 from Reardon, Sean F., Kalogrides, Demetra and Shores, Ken The Geography of Racial-Ethnic Test Score Gaps at https://www.nytimes.com/interactive/2016/04/29/upshot/money-race-and-successhow-your-school-district-compares.html?action=click&contentcollection=upshot®ion=rank&module=package&version=highlights&contentplacement=1&pgtype=sectionfront&smid=twupshotnyt&smtyp=cur&_r=2 Center for Urban Education Leadership, University of Illinois at Chicago Page 4
Changes in Demographics, Enrollment and Instructional Resources Between 2001 and 2016, enrollment in Elmhurst rose to over 3,800 before dropping more recently to a little over 3,700. Lowincome enrollment rose from 2% to 14%, white enrollment declined by 17 points and Latino and multi-racial enrollment rose by 9 points. State-wide, overall enrollment fell and low-income enrollment grew. White enrollment dropped by 11 points and Latino enrollment rose by 11 points. Average, per-pupil spending on instruction rose by 68% in Elmhurst and by 74% state-wide. Elmhurst District 205 All Illinois Per Pupil Instructional Expenditures. Center for Urban Education Leadership, University of Illinois at Chicago Page 5
There is a Strong Connection between Low-Income Enrollments and Standardized Achievement Scores Composite Math Achievement and Low-Income Enrollments for Grades 3 through 8 in Elmhurst and Other Suburban Chicagoland School Districts Blue points in the charts above show the intersect of achievement and low-income enrollment in each of 229 districts located in DuPage, Kane, Lane, McHenry, Suburban Cook and Will counties. The vertical scale shows the percent of students who scored at or above state-wide medians (50 th percentile and above). The horizontal scale shows the percent of students eligible for free or reduced lunch in each district. The green lines show achievement levels that are most typical of each free or reduced lunch level. In 2001, most districts in suburban Chicagoland had low-income enrollments of less 20% or less. Elmhurst had 2% low-income enrollments, 15 points below the regional average. In 2001, 77% of Elmhurst students scored at or above state-wide math medians. That was 18 points above the regional average and slightly above the trend line for districts with similar low-income levels. By 2016, low-income enrollments more than doubled in suburban Chicagoland and average math achievement declined by 3 points. But gains in school effectiveness pushed the trend line up across the entire spectrum of low-income enrollments. In Elmhurst, math achievement fell by 4 points but stayed above the trend line for districts with similar low-income enrollments. Center for Urban Education Leadership, University of Illinois at Chicago Page 6
Changes in Composite Achievement in Elmhurst: 2001-2016 The charts below show changes in achievement ranges for students in grades 3-8 between 2001 and 2016 in Elmhurst. Green bands show the portion of students who scored at or around grade level 1. Pink and tan bands show students who scored one or more years below grade level 1. Blue and purple bands show students who scored one or more years above grade-level 1 and were academically on track for college readiness at the end of grade 11. Changes in the average achievement of all students in grades three through eight are shown at the top of each chart. Reading Math 1 Achievement bands and grade equivalents are based on the stanine and percentile rank of individual student scale scores on state-wide scoring distributions (see chart legends) Center for Urban Education Leadership, University of Illinois at Chicago Page 7
Changes in Third Grade Achievement in Elmhurst: 2001-2016 Third grade achievement is a cumulative measure of student learning and instructional effectiveness for grades PK through 3. For school- and district-size groups, it is also a reliable predictor of future achievement in middle school and high school. The charts below indicate that average third grade achievement district-wide fell by about 7 school months in both reading and math between 2001 and 2016. Students scoring at or above grade level (green, blue and purple bands) declined from 81% to 78% in reading and from 86% to 79% in math. Students who were academically on track for an ACT score of 21 or higher at the end of eleventh grade (blue and purple bands) dropped from 67% to 54% in reading and from 72% to 60% in math. Center for Urban Education Leadership, University of Illinois at Chicago Page 8
Increases in Low-Income Enrollments Have Challenged Instructional Effectiveness in Most Suburban Chicagoland Districts The bottom right quadrants in the charts below show that achievement in suburban Chicagoland school districts often declined compared with statewide norms when low-income enrollments increased. But the upper right quadrant in both charts shows that this trend did not happen everywhere. The opposite occurred in about 28% of wealthier districts (chart on left) and in about 40% of districts with higher rates of free or reduced lunch eligibility in 2001 (chart on right). The yellow diamonds in both charts are large unit (LUDA) districts. The orange diamond (partially hidden) in the chart on the left is Elmhurst District 205. Changes in Composite Math Achievement and Low-Income Enrollments at Grades 3 through 8 in Elmhurst and Other Suburban Chicagoland School Center for Urban Education Leadership, University of Illinois at Chicago Page 9
Center for Urban Education Leadership, University of Illinois at Chicago Page 10
Some Surprising Connections between ISAT and PARCC Results Both Tests Made Similar Estimates of Academic Preparation and Made Similar Predictions of College Readiness There are lots of important differences between the old ISAT and newer PARCC exams. But when it comes to estimating overall academic readiness, ISAT and PARCC results have been pretty much the same. The charts below show the percentages of Illinois third graders 2 who scored at or above state-wide medians on ISAT and PARCC exams for reading and language arts. The chart on the left shows results over time for students who were eligible for free or reduced lunch. The chart on the right shows results for students who were not eligible for free or reduced lunch. The surprising finding for each of the six groups shown is that results from the 2015 and 2016 PARCC were not much different than results from the 2014 ISAT. Some went down a little. Some went up a little. Some stayed the same. 3 rd Grade Reading/ELA 2 Percent of non-ell Third Graders Who Achieved At or Above Statewide Median Scores 2 Students with the temporary designation of English language learner (ELL) have been removed from this report to ensure comparability of results over time Center for Urban Education Leadership, University of Illinois at Chicago Page 11
Low Cut Scores Made the ISAT Look Easier than It Actually Was Low cut scores under No Child Left Behind made the ISAT look like it was an easy test... far less rigorous than the National Assessment of Educational Progress (NAEP), Measures of Academic Progress (MAP) and more recently PARCC. In fact, most NCLB-era tests produced close-to-identical, bell-curve distributions even though they claimed to assess very different standards 3. They were just graded differently. Tests like the ISAT produced more passing grades because they located cut scores at the lower end of the scoring distribution... typically between the 15 th and 25 th percentile. NAEP and PARCC produce fewer passing grades because they set higher cut scores... typically between the 60 th and 70 th percentile. On all three tests, scores between the 60 th and 70 th percentile correlate closely with college readiness benchmarks on the ACT and SAT. Once cut scores are removed from the mix, achievement patterns on the ISAT, NAEP, PARCC and most other standardized tests are not very different. All of these tests predict each other s results with high levels of accuracy and measure progress toward college readiness more or less equally well 4. So What? Close-to-identical results on ISAT and PARCC tests mean that Illinois school districts do not have to start from scratch building new baseline data about students on track for college readiness and other important metrics. Normalized 6 ISAT results can be matched up directly with normalized 5 PARCC results. This gives schools and districts a common yardstick going all the way back to 2001. The charts in this report offer a small sample of how this yardstick can be used to assess long-term achievement changes in Elmhurst and other Illinois school districts. 3 For a more extended discussion of how cut scores misrepresented what meeting standards meant under NCLB, see PART 2, An Alternate Universe of Test Results in Zavitkovsky, Paul et. al. (2016) Taking Stock: Achievement in Illinois under No Child Left Behind http://urbaneducationleadership.org/what-we-do/research 4 For more detailed discussion of the ISAT s ability to predict college readiness on the ACT at the end of 11 th grade, see Allensworth, Elaine et. al. (2014) Looking Forward to High School and College: Middle Grade Indicators of Readiness in Chicago Public Schools http://consortium.uchicago.edu/sites/default/files/publications/middle%20grades%20report.pdf and Easton, John Q. et. al. (2008) From High School to the Future: The Pathway to 20 http://consortium.uchicago.edu/sites/default/files/publications/pathway%20to%2020%20report-final.pdf and Zavitkovsky, Paul (2009) Something s Wrong with Illinois Test Results http://evanstonroundtable.com/ftp/zavitkovsky%20report.pdf 5 All tests with equated scale scores have near-normal, bell-curve distributions. Normalized results describe how students achieve in comparison with all other students tested. By contrast, results based on cut scores report the percentage of students who score at or above a specific scale score. Center for Urban Education Leadership, University of Illinois at Chicago Page 12
Reading the Charts in This Report Students Scoring At or Above Statewide Medians 6, College Readiness Benchmarks 7 & PARCC Proficiency Levels 8 Pages 15 through 19 of this report compare achievement changes over time in Elmhurst: Charts on page 15 show changes over time in composite achievement for all students in grades three through eight Charts on page 16 through 18 show changes over time at individual grade levels Charts on page 19 show changes in achievement over the six years that students move from 3 rd grade through 8 th grade. Data shown are for the 8 th grade graduating classes of 2014, 2015 and 2016 The charts on pages 15 through 19 report achievement in Elmhurst in five ways: Green lines report the percentage of students scoring at or above state-wide medians 6 Blue lines report the percentage of students scoring at or above on-track estimates of college readiness on the ACT 7 Brown squares report the percentage of students scoring at or above PARCC Level 2 in 2015 and 2016 8 Tan circles report the percentage of students scoring at or above PACRCC Level 3 in 2015 and 2016 8 Purple diamonds report the percentage of students scoring at or above PARCC Level 4 in 2015 and 2016 8 In the sample chart at the top of page 14: The green line shows 73% of students scoring at or above state-wide reading medians 6 in 2001 compared with 77% in 2014 The blue line shows 63% of students on track for ACT college readiness 7 in 2001 compared with 67% in 2014 The purple diamonds show 58% of students scoring at or above PARCC Level 4 8 in 2015 and 56% in 2016 The tan circles show 83% of students scoring at or above PARCC Level 3 8 in 2015 and 82% in 2016 The brown squares show 95% of students scoring at or above Level 2 8 in 2015 and 94% in 2016 6 Between 2006 and 2014, median ISAT scale scores for both reading and math rose moderately at all grade levels tested statewide. Average, 8-year increases were 1 scale point in composite 3-8 reading and 3 scale points in composite 3-8 math. This means it was slightly harder to score at or above the statewide median in 2014 than it was in 2006. 7 Since the early 2000 s, the 60 th percentile of the Illinois distribution has reliably predicted a score of 21 on the ACT at the end of 11 th grade; similarly, the 68 th percentile has reliably predicted a 22 on the ACT. Procedures used for estimating students on-track for college readiness from statewide ISAT results are described in Zavitkovsky, Paul (2009) Something s Wrong with Illinois Test Results http://evanstonroundtable.com/ftp/zavitkovsky%20report.pdf 8 PARCC s 2016 Score Report Interpretation Guide says students scoring at Level 2 partially met expectations, students at Level 3 approached expectations and students at Level 4 met expectations. Students scoring at Levels 4 and 5 demonstrate readiness for the next grade level and, ultimately, are on track for college and careers. Center for Urban Education Leadership, University of Illinois at Chicago Page 13
A key take-away from the upper chart is that ISAT and PARCC exams produced very similar estimates of students who are on-track to meet college readiness benchmarks on the ACT and SAT. Composite Reading for Grade 3-8 The lower chart shows the same PARCC information and on-track-for-college readiness estimates as the upper chart does. But it also includes the percentage of students who scored at or above three different sets of ISAT cut scores between 2001 and 2014. Cut-scores adopted in 2013 are back-mapped to 2006. Cut scores used between 2006 and 2012 are mapped forward through 2014. The lower chart illustrates that the same, bell-curve distributions can produce very different results by simply moving cut scores to higher or lower points on the bell curve. Center for Urban Education Leadership, University of Illinois at Chicago Page 14
Composite Achievement over Time for Grades 3 through 8 Composite Reading Composite Math *Within standard errors of measure, median scores for the State of Illinois have reflected median scores for the nation as a whole since the early 2000s **Since the early 2000s, the 60 th percentile of the Illinois reading and math distributions have reliably predicted an ACT score of 21 at the end of 11 th grade; 68 th percentile predicts 22 on the ACT Center for Urban Education Leadership, University of Illinois at Chicago Page 15
Reading Third and Fourth Grade Math *Within standard errors of measure, median scores for the State of Illinois have reflected median scores for the nation as a whole since the early 2000s **Since the early 2000s, the 60 th percentile of the Illinois reading and math distributions have reliably predicted an ACT score of 21 at the end of 11 th grade; 68 th percentile predicts 22 on the ACT Center for Urban Education Leadership, University of Illinois at Chicago Page 16
Reading Fifth and Sixth Grade Math *Within standard errors of measure, median scores for the State of Illinois have reflected median scores for the nation as a whole since the early 2000s **Since the early 2000s, the 60 th percentile of the Illinois reading and math distributions have reliably predicted an ACT score of 21 at the end of 11 th grade; 68 th percentile predicts 22 on the ACT Center for Urban Education Leadership, University of Illinois at Chicago Page 17
Reading Seventh and Eighth Grade Math *Within standard errors of measure, median scores for the State of Illinois have reflected median scores for the nation as a whole since the early 2000s **Since the early 2000s, the 60 th percentile of the Illinois reading and math distributions have reliably predicted an ACT score of 21 at the end of 11 th grade; 68 th percentile predicts 22 on the ACT Center for Urban Education Leadership, University of Illinois at Chicago Page 18
Cohort Achievement from Grade 3 through Grade 8 Reading Math *Within standard errors of measure, median scores for the State of Illinois have reflected median scores for the nation as a whole since the early 2000s **Since the early 2000s, the 60 th percentile of the Illinois reading and math distributions have reliably predicted an ACT score of 21 at the end of 11 th grade; 68 th percentile predicts 22 on the ACT Center for Urban Education Leadership, University of Illinois at Chicago Page 19
Between 2001 and 2016, Composite Achievement in Elmhurst at Grades 3-8 Shifted Substantially Compared with Chicago and Other LUDA Districts Elmhurst and the other 54 member districts of Illinois Large Unit District Association (LUDA) serve close to half of all Illinois public school students and reflect the socio-economic diversity of school districts throughout the state. The charts below show the percent of students in grade three through eight who scored at or above state-wide reading and math medians* in 2001 and 2016. Each dot represents a single LUDA district. In 2001, composite reading achievement in Elmhurst ranked 8 th of 55 among all LUDA districts with 73% of students scoring at or above state-wide medians*. In 2016, composite reading in Elmhurst ranked 9 th of 55 among all LUDA districts. Students scoring at or above state-wide medians decreased 3 points to 70%. Reading *Between 2006 and 2014, median ISAT scale scores for both reading and math rose moderately at all grade levels tested statewide. Average, 8-year increases were 1 scale point in composite 3-8 reading and 3 scale points in composite 3-8 math. This means it was slightly harder to score at or above the statewide median in 2014 than it was in 2006. Center for Urban Education Leadership, University of Illinois at Chicago Page 20
In 2001, composite math achievement in Elmhurst ranked 5 th of 55 among all LUDA districts with 77% of students scoring at or above state-wide medians*. In 2016, composite math achievement in Elmhurst ranked 7 th of 55 among all LUDA districts but students scoring at or above state-wide medians* decreased by 4 points to 73%. Math *Between 2006 and 2014, median ISAT scale scores for both reading and math rose moderately at all grade levels tested statewide. Average, 8-year increases were 1 scale point in composite 3-8 reading and 3 scale points in composite 3-8 math. This means it was slightly harder to score at or above the statewide median in 2014 than it was in 2006. Center for Urban Education Leadership, University of Illinois at Chicago Page 21
For more information, contact: Paul Zavitkovsky Leadership Coach/Assessment Specialist pzavit@uic.edu Steve Tozer Program Director stozer@uic.edu UIC Making Good on the Promise of Public Education Center for Urban Education Leadership, University of Illinois at Chicago Page 22