IMPROVING ANNE ARUNDEL COUNTY PUBLIC SCHOOLS SUMMARY: Anne Arundel County students leave elementary school with skills that are among the best in the state and, possibly, the country. However, their performance declines during middle and high school versus other high-performing school districts, and by the time the students graduate from high school, the average SAT score for Anne Arundel students is no better than the national average. This report investigates factors that could be contributing to why Anne Arundel County middle and high school students are not living up to their potential and recommends the school system embrace a datadriven culture focused on turning this problem around. BACKGROUND: In order to understand what data is telling us, we cannot look at it in a vacuum. To see how well Anne Arundel County is meeting the objective of providing students with a superior education, we need to compare how our students are doing versus those in top-performing school systems like Howard and Montgomery County. Interestingly, all three of these counties enroll a similar percentage of minority students. By focusing on whether our students are exhibiting advanced levels of understanding in math, science and English, we can determine whether our schools are developing students likely to excel in the crucial STEM-related fields of the future. KEY FINDING: At the end of 5 th grade, Anne Arundel students are performing much better than students across the state, comparably to those in Montgomery County and close to those in Howard County in all three subjects tested by the Maryland School Assessment (MSA). Unfortunately, during the three years of middle school, the performance of our students declines fairly precipitously versus those of Maryland s top counties. By the end of 8 th grade, Anne Arundel students are actually scoring below the average for the state of Maryland in math.
The poorer performance of Anne Arundel County students relative to Howard and Montgomery Counties continues in high school. In order to graduate from high school, students have to pass HSA tests in algebra, biology and English. Anne Arundel students do reasonably well on the algebra test, but no better than the state average in biology or English: And by the time they take the SAT test to enroll in college, Anne Arundel students are scoring 150 points below those in Howard and Montgomery Counties and equal to the national average: 2013 Avg. SAT Howard County 1653 Montgomery County 1648 Anne Arundel County 1498 State of Maryland 1483 National Average 1498 Since the average SAT scores for Anne Arundel County students are identical to the national average, we can assume that the College Board s national estimate that only 43% of SAT takers in the class of 2013 graduated from high school academically prepared for the rigors of college-level course work is true for Anne Arundel County s graduating seniors as well. As confirmation of this, 70% of the 1,509 graduates of Anne Arundel County schools who enrolled in Anne Arundel Community College in the fall of 2013 were required to take at least one remedial math course before they could enroll in an entrance-level math class. There are several possible factors that could be keeping our students from living up to their potential in middle and high school. POTENTIAL FACTORS 1. Something is wrong with the way we re teaching middle school math. Unlike other subjects, MSA middle school math performance has not been improving in recent years. In fact, scores dropped alarmingly in 2013 to the point where 33.7% of last year s 8 th graders didn t exhibit sufficient knowledge of 8 th grade math to even pass the MSA:
Since Anne Arundel County led the state in implementing the new Common Core middle school math program in 2012/13, last year s tremendous drop-off in performance is a serious concern. The school system postulates that last year s poor scores were a result of a mismatch between what was taught in Common Core and what was tested by MSA. Other possible explanations are that the Common Core is less effective or that its implementation was hurt by teachers having insufficient time to prepare strong lesson plans. It is important to note, however, that Anne Arundel County students were performing poorly relative to the benchmark counties and the state even before the implementation of the Common Core last year, as the data from 2012 demonstrates: 2. Despite concerted efforts, Anne Arundel County has made no progress in closing the achievement gap over the last four years and, in fact, it worsened last year in math. After staying essentially the same since 2008/09, the achievement gap between Caucasian and African-American students widened last year in math:
Middle school again seems to be the biggest problem area. The gap between white and African- American students widens considerably in middle school, particularly in math. Disturbingly, less than 40% of African-American 8 th grade students had passing math scores last year compared to 76% of white students: There is also a gap between the performance of Hispanic students and white students, although it isn t as large. This gap also widens during middle school. 3. A high percentage of students are being enrolled in advanced or honors classes in middle and high school despite not being advanced in their understanding of the subject matter. Encouraging students to take challenging classes is clearly appropriate, within reason. But there's a tipping point where students are in so far over their heads that they can't catch up, and we've apparently reached this point in some of our middle and high school classes. One would expect the percentage of students enrolled in advanced classes to be related to the percentage of students who show an advanced level of understanding in that subject. Let s call this relationship the Challenge Index: Challenge Index = % of Students Enrolled in an Advanced Class in This Subject % of Students Scoring at an Advanced Level If the percentage of students in advanced classes is identical to the percent scoring advanced in that subject, the Challenge Index is 100. At some of our schools, however, it is not uncommon to see challenge indexes of 300 or more.
The following chart combines year-to-year data for 6 th, 7 th and 8 th grade math classes at all 19 middle schools for each of the last three years. Not surprisingly, it shows that the students have the best results (e.g., the percentage scoring advanced increases while the percentage failing decreases) when the Challenge Index is close to 100. As the Challenge Index increases, the percentage of students scoring at the advanced level plummets and the percentage failing the MSA that year increases dramatically: Level of Challenge The Impact of Level of Challenge on Year-to-Year Math MSA Scores Under Appropriately Moderately Very Challenged Challenged Challenged Challenged Severely Challenged # Student Years of Data 11,116 9,332 9,907 9,147 8,749 Challenge Index: Range 16 79 81 114 116 145 147 219 220-816 Average 49 99 132 175 322 % Scoring Advanced on MSA: Prev. Year 41.5% 38.8% 35.2% 32.5% 25.6% This Year 42.2% 40.5% 33.7% 28.3% 16.9% Change + 0.7% + 1.7% - 1.5% - 4.2% - 8.7% % Failing Math MSA: Prev. Year 19.9% 19.3% 18.2% 20.4% 27.1% This Year 18.5% 17.1% 17.3% 23.1% 34.5% Change - 1.4% - 2.2% - 0.9% + 2.6% + 7.4% A push to encourage a lot more students to take advanced classes is also occurring in high school. For example, in the last five years, the number of AACPS students taking Advanced Placement (AP) exams has increased by 57%. Unfortunately, a lot of AACPS students are not doing well on the exams. AP exams are graded on a five-point scale, with a grade of 3 and above considered passing. Nationwide, 60% of students who take an AP exam receive a score of 3 or above, which is almost exactly the percentage of Maryland students who passed their AP exams in 2013. However, only 46.8% of Anne Arundel County exam-takers passed their AP exams last year, which is far below the rates in Howard and Montgomery Counties: 2013 AP Exam Results % Scoring 3 or Higher on 2013 AP Exam AP Tests Taken as % of 9 th -12 th Grade Enrollment Anne Arundel 46.8% 30.5% Howard 81.7% 25.5% Montgomery 73.0% 37.8% Maryland 60.3% 22.3% The data looks even worse when compared on a school-by-school basis. Only students at Severna Park and Broadneck High Schools passed at least 60% of their exams, and half of the Anne Arundel high schools had passing rates of less than 36%.
2013 AP Exam Results by High School % Scoring 3 or Higher High School on 2013 AP Exam Severna Park 66.9% Broadneck 60.8% Arundel 55.8% South River 52.4% Chesapeake 47.7% Southern 45.0% Old Mill 35.9% Annapolis 32.1% Meade 30.8% Northeast 26.8% North County 24.6% Glen Burnie 21.5% Total Anne Arundel 46.8% The push to enroll more students in AP classes is driven by the belief that exposure to advanced-level classes will improve college performance. While research consistently shows that students who receive a score of 3 or higher on AP exams will likely do better in college, the impact of participating in an AP class for those who score below a 3 is less clear. It is hard to believe, for example, that the 45% of exam-takers at Meade High School who received the lowest-possible score of 1 on an AP exam last year were positively affected by their participation in the AP class. 4. There is a tremendous variation in student achievement among the twelve high schools, which grades don t totally reflect. In high school, the same benchmark exam is given to all students in English 10, Algebra 1, and Biology at the end of each marking period to gauge their understanding of the material. The following chart compares the scores on last year s end-of-year benchmark exams to the grades students in those classes received in the 4 th marking period. Benchmark Performance vs. Grades Top 4 Scoring High Schools in Subject Middle 4 High Schools in Subject Bottom 4 High Schools in Subject % of Students Scoring 70% or better on 2012/13 4 th quarter benchmark exam: English 10 79.0 69.5 59.6 Algebra 1 55.8 36.0 28.2 Biology 79.5 65.4 45.9 % of Students Scoring 50% or below on 2012/13 4 th quarter benchmark exam: English 10 3.6 7.1 12.2 Algebra 1 14.3 23.5 40.8 Biology 4.8 11.0 22.9 Average 4 th Period Grade: English 10 80.5 80.4 76.9 Algebra 1 74.8 70.9 65.5 Biology 82.1 80.5 76.9
55.8% of the students at the 4 top-performing schools (which were South River, Severna Park, Arundel & Broadneck for Algebra 1) scored at least 70% on the 4 th quarter Algebra 1 benchmark and only 14.3% of students at these schools scored below a 50%. This is quite a contrast with the 4 lowestperforming schools in Algebra (Annapolis, North County, Glen Burnie & Meade) where only 28.2% of the students scored 70% or better and 40.8% scored 50% or below on the benchmark. Despite the very wide variation in understanding of the subject matter between the students at these two groups of schools, however, the average student in the poorer-performing schools received a 4 th period grade of 65.5, not that much lower than the 74.8 average grade at the top four schools. Parents who are worried about whether their students have truly mastered the subject matter should look at their child s benchmark test results as the grades can be misleading. KEY RECOMMENDATION AACPS needs to follow the lead of progressive school systems across the country who are improving academic performance and reducing their achievement gaps by implementing data-driven cultures that provide teachers and administrators with the data and resources to improve how well their students are learning. All three of the large school systems who won the coveted Broad Foundation award in recent years for dramatically improving academic performance and reducing their achievement gaps attribute a large part of their success to strong real-time, performance-focused data systems. For example, teachers working in the Houston Independent School District, the 2013 winner, routinely analyze and use data to determine the effectiveness of their instructional strategies. In fact, many of the teachers replicate each other s lessons after the data shows who was most successful at teaching a concept. Teachers also use real-time data to identify students who need more help understanding a particular topic. Businesses have been investing for decades in data-driven systems to continuously improve their operations. Anne Arundel County Public Schools needs to move to a data-driven culture and invest in the necessary tools and resources so our teachers and administrators can help our students reach their full potential. Joanna L. Conti April 14, 2014