Charter School Performance in Los Angeles Unified School District

Transcription

1 Charter School Performance in Los Angeles Unified School District A District and Neighborhood Matched Comparison Analysis June 2008 California Charter Schools Association

2 Acknowledgements This report is a product of the School Achievement Data Team of the California Charter Schools Association Elizabeth Robitaille, Director Rebecca Tomasini, Deputy Director Aisha N. Toney, Ph.D., Senior Data Analyst Danessa Murdock, Analyst Written by: Aisha N. Toney, Ph.D., Senior Data Analyst Danessa Murdock, Analyst Peer reviewed by: Vicki E. Murray, Ph.D. Senior Policy Fellow, Education Studies Pacific Research Institute 1 Charter School Performance in Los Angeles Unified School District

3 Table of Contents Inside the Report Introduction...3 Methodology Key Findings Discussions and Implications Conclusion...22 Information Report Outline Terms and Definitions FAQ & A About California Charter Schools Association...33 June 2008 California Charter Schools Association 2

4 Introduction Charter School Performance in Los Angeles Unified School District In 1992, California became the second state to enact charter school legislation and opened its first charter school in That legislation allowed these public schools to operate under a five-year, renewable contract or charter with their district, county office of education, or the State Board of Education. Charter schools are public schools that operate outside of district control, giving them autonomy to determine their own instructional program. This freedom and flexibility enables charter schools to better address the requirements of the communities they serve and be responsive to students needs. The nature of the California charter school legislation also holds charter schools accountable for improved student achievement. State law mandates specific performance requirements outlined in the charter that must be met for that charter to be renewed. Charter schools also have to recruit and retain students to remain viable. In California, the freedom and flexibility of operating a charter comes with highstakes accountability. The charter school movement in California has seen tremendous growth since the California Charter Schools Act of In the past 10 years, charter schools have consistently grown by an average of 50 schools per year. Currently there are 687 charter schools operating in California serving nearly 240,000 students. Los Angeles is an area of California where charter schools have flourished. In 2006, the Los Angeles Unified School District (LAUSD) became the first school district in the country to have 100 charter schools in operation. Today, there are 125 charter schools serving nearly 41,000 public charter school students, making Los Angeles the largest charter movement in the state. Los Angeles is also the seventh largest charter movement in the nation. Only Arizona, Florida, Michigan, Ohio, and Texas have more charter students in their state than Los Angeles has in its county. Given its importance in the state and national charter school movements, the California Charter Schools Association 3 conducted an analysis of the Los Angeles Unified public schools to understand how the charter public schools in LAUSD were performing in comparison to the traditional public schools. The Association s first ever matched comparison analysis was also conducted to provide a more nuanced assessment of charter performance to better inform neighborhood educational choices. This report addresses two important questions: 1. How does the academic performance of charter schools in Los Angeles Unified differ from that of traditional public schools? 2. How does charter school performance in Los Angeles Unified compare to that of matched traditional public schools in the same neighborhood? These questions were answered by examining state API and federal AYP data from the academic school year to conduct a district analysis of LAUSD charter and traditional public schools in aggregate and at the elementary, middle, and high school levels, as well as by CDE specified subgroups. A micro level, neighborhood matched comparison analysis comparing a sample of LAUSD charter schools to their three most similar traditional public schools was used to assess whether charter schools in Los Angeles Unified are a better academic option for parents and students. 3 The California Charter Schools Association is the membership and professional organization serving charter schools in California. The Association s mission is to increase student achievement by supporting and expanding California s quality charter public school movement. 3 Charter School Performance in Los Angeles Unified School District

5 Methodology The Data Cleaning Process The California Charter Schools Association annually analyzes state achievement data describing charter public school versus traditional public school performance. In this process, the August release of the state s Accountability Progress Report achievement data is downloaded from the California Department of Education s (CDE) Web site, cleaned, and analyzed. The state s Academic Performance Index (API) and federal Adequate Yearly Progress (AYP) 2 data files are merged to gather an accurate population of school data and to cross-reference it to identify missing schools or misclassified schools. Before any analyses are conducted, the Association engages in a rigorous data cleaning process, comparing the state s API and AYP data files to our internal charter database to ensure that all charter schools are correctly classified. After merging the state files with our internal database, any identified discrepancies are resolved by working with the Association s knowledgeable regional charter school managers who have a close relationship with their regional charter schools, as well as by checking schools status and available data via Dataquest and Quickquest on the CDE Web site. Additionally, the Association works in partnership with the Administrator of the Academic Accountability Unit at the CDE to clarify questions and permanently correct issues identified during the data cleaning process. Inactive or closed charter schools and alternative and special education (ASAM) schools are then removed from the analysis to ensure an appropriate comparison of school performance. These exclusions are in-line with those used by the CDE and most other researchers. CDE, however, also excludes small schools and direct funded charter schools; the Association does not, but rather analyzes a more complete sample of the California charter population. This year s data cleaning process identified 18 traditional public schools in Los Angeles that were misclassified as charter schools (see Table 1 in the appendix for a list of these schools). These misclassified schools were properly coded as traditional public schools in our data file. Additionally, charter and traditional school total numbers were tracked during each merge and cleaning iteration (see Table 2 in the appendix for those exact numbers). This cleaning process yielded a sample of 102 charter and 698 traditional public schools in Los Angeles Unified, of which 79 charter schools and 593 traditional public schools had valid 2006 API base scores; 98 charter schools and 614 traditional public schools had valid 2007 API growth scores; and 79 charter schools and 593 traditional public schools had valid 2007 API change scores. 2 Data terms are defined in the appendix. June 2008 California Charter Schools Association 4

6 Methodology District Analysis With this sample, 2007 API variables were assessed for Los Angeles Unified School District charter and traditional public schools, as well as their meeting of state API growth targets and making federal AYP for the school year. Median API scores were calculated in accordance with the CDE s methodology. The percentage of schools meeting their targets was calculated for four types of target variables: 1) school wide API targets, 2) comparable improvement API targets, 3) both school wide and comparable improvement API targets combined, and 4) meeting AYP. These growth targets are established for each school as a whole and for each numerically significant subgroup 3 in the school. School wide API targets assess whether or not a school met their API growth target for the school year. An API score of 800 is the statewide performance target. The annual API growth target is defined as 5% of the difference between the school s Base API and the statewide performance target. Comparable improvement API targets assess whether or not a school met its subgroup API growth targets for the school year. To meet all state API growth target requirements and make AYP, each numerically significant subgroup in a school must demonstrate comparable improvement in meeting API targets. This measure applies to ethnic/racial, socioeconomically disadvantaged, English learner, and students with disabilities subgroups. API growth targets are calculated separately for each numerically significant subgroup and set at 5% of the difference between the subgroup s Base API and 800. An important charter specific variable was also assessed for a number of our analyses a charter s number of years in operation or its maturity. Charter maturity differentiates schools that are six years old or older (mature) from those that are five years old or younger (young). This distinction is important because a mature charter will have successfully been reauthorized, proving their academic and operational performance and solidifying their future. Both internal and external research has yielded important differences in charter academic performance based on charter maturity. 3 CDE subgroups are parent identified categories that cannot assure accurate designations. 5 Charter School Performance in Los Angeles Unified School District

7 Methodology Matched Comparison Analysis While the annual analysis conducted by the Association is primarily descriptive, this year the Association conducted its first ever matched comparison analysis using the API data. Building on the research and methodology of Hoxby (2004) and Roy & Mishel (2005), the California Charter Schools Association conducted a matched comparison analysis to assess how charter schools performance compared to neighboring traditional public schools in Los Angeles. This more nuanced examination of school performance provides a micro level, neighborhood analysis of a family s likely school choices and assesses the value charter schools add to the educational arena, as well as parents and students lives. In so doing, this methodology greatly improves on aggregate analyses of school level data and does a better job of assessing charter quality. The matched comparison analysis began with identifying three peer traditional public schools (where data permitted) for each charter in the sample. Charter to peer matching was done using Geographical Information Systems (GIS) 9.2 software to map all of California s schools and their demography. Each charter school with available demographic and API test data was matched with every proximal (within a five-mile radius) traditional school most similar in racial composition, parental educational attainment, and free/reduced-price lunch participation. This matching yielded 304 traditional public schools for the Los Angeles Unified analysis. Alternative and special education schools were excluded for both charter schools and non-charter schools. Schools were also matched for grade level (i.e., elementary matched with elementary, etc.) and school type (e.g., Internet-based charter schools were omitted). Schools latitude and longitude points and their addresses were used to plot schools using GIS mapping. Those that were missing longitude and latitude, but had a physical street address, were also geocoded. If there was a match based on an 80% spelling sensitivity test, then the geocode address match was accepted. For those that did not geocode automatically, they were attempted by manually geocoding. If an acceptable match was found, then the geocode address match was accepted. If there was not a longitude/latitude and no street address, the school was not included in the analysis. The outcome was 98.45% school addresses that were geocodable. To ensure that the geocoding was accurately mapping the schools correct location, the file was spot-checked. Fifty addresses from all areas of California were verified by clicking on the address in ArcGIS and visually seeing where it was located on the street and looking up this address on Google Maps to confirm that both addresses were in the same approximate location. All geocodes were in an acceptable place. In ArcGIS 9.2, Hawths Tools Distance Between Points (Between Layers) was used to create a distance matrix for each school type. The tool derived the closest 100 traditional school geocodes per each individual charter geocode. A distance radius was then set to five miles to isolate the most proximal traditional public schools to each charter to identify a family s likely school options. In ArcGIS, distance is calculated as Euclidean distance ( as the crow flies ) versus drive time. June 2008 California Charter Schools Association 6

8 Methodology Matched Comparison Analysis In order to complete the second part of the GIS analysis, which was to determine what three traditional public schools within a five-mile radius were a demographic match to each charter, a formula was created to determine the least differences between the schools. This was derived by calculating the absolute value between the charter and traditional public schools for the three demographic variables (percent students of color, free/ reduced price lunch participation, and average parent education) and summing up the differences. Those with the least differences were considered most similar and were rank ordered as such in the data file. Given that race/ethnicity is one of the strongest and most persistent demographic variables affecting academic performance (independent of SES) and that it highly correlates with SES, neighborhood, and education the racial composition of the school (percentage of students of color) accounted for twice as much as the other two variables. The other two SES variables were not weighted, serving as a balanced measure of socioeconomic status. Two types of analyses were conducted with this data. The first compared each charter to its matches individually, calculating the difference between the charter schools API base, growth, and change scores and each one of its matches (via subtraction). Those differences were then averaged to get an average difference score, which indicated whether the charter performed better than (a positive average) or worse than (a negative average) its matched traditional public school peers. The second analysis used regression (a type of statistical analysis using SPSS statistical software) to predict how many more or fewer points in API scores were likely if the school was a charter. These results were reported if that difference was significant, meaning statistically significant as defined by a P value of.05 or less (*p.05, **p.01, ***p.001). The following school demographic variables were controlled for in the regression analyses: racial composition (percentage students of color), free/ reduced-price lunch participation, average parental education, and enrollment. 7 Charter School Performance in Los Angeles Unified School District

9 Methodology LAUSD Sample Demographics The Los Angeles Unified School District has 125 charter schools and 742 traditional public schools. Within the district, over forty thousand students attend charter public schools (34,764 in independent 4 charter schools and 5,558 in dependent charter schools) and over 600,000 attend traditional public schools (662,650), yielding a total enrollment of over 700,000 students in LAUSD. These students are diverse (see Graph 1 below) and are almost equally split between the primary and secondary grade levels in both traditional (48% elementary, 23% middle, and 29% high school) and charter schools (47% elementary, 17% middle, and 36% high school). The matched comparison analysis of the Los Angeles Unified School District, including independent and dependent charter schools, yielded 102 charter schools and 304 traditional public schools with the data needed to match schools and perform analyses. Only one charter had less than three matches (it had two). There were 20 mature charter schools (schools that are six years old or older) and 82 young charter schools (schools less than six years old) in this sample. Maturity was assessed in this analysis because it has a statistically significant positive relationship with API growth (.141**), significant at the p =.01 level using 2- or 1-tailed Pearson s correlations. The results section that follows will present analyses of the Los Angeles district level data first, followed by highlights from the matched comparison analysis in each section. Results sections are organized by key findings that summarize the overarching trend of data described in each section. 4 Independent or direct funded charter schools receive funding from the state. Therefore, they are autonomous from their local district. Dependent or indirect funded charter schools are locally funded from their district. Therefore, they are an extension of and dependent on the district for funding, governance, etc. Graph 1: Student Ethnicity in LAUSD *Other includes American Indian, Pacific Islander, Filipino, and Multi-Racial June 2008 California Charter Schools Association 8

10 Key Findings Charter schools in LAUSD outperform traditional public schools on a variety of student achievement measures. As discussed in the methodology, API results were analyzed to assess the academic performance of charter schools in LAUSD and to compare charter performance to that of traditional public schools. Overall, this analysis demonstrates a trend of higher performance by charter schools in LAUSD in median API base and growth, as well as in meeting API and AYP targets. Charter schools in Los Angeles Unified earned a higher median API base of 711 compared to traditional public schools, where the median API base was 697. That was also true at the elementary, middle, and high school levels by an average of 63 greater points. That difference was smallest at the elementary school level where the charter median API base was 730 and the traditional school median API base was 718 (a difference of 12 points) and greatest at the middle school level, where the charter median API base was 729 and the traditional school median API base was 629 (a difference of 100 points). At the high school level, charter schools in LAUSD median API base was 682 and that of traditional high schools was 605 (a difference of 77 points). While the differences in API scores between these charter and traditional public schools is impressive, it is important to note the decreasing trend in performance from elementary to high school for both charter schools and traditional public schools, such that the median API scores decrease as students progress to each grade level. This trend will continue to emerge throughout the analyses. While charter schools outperformance of traditional public schools in median API growth is masked at the aggregate district level (706 for charter and 712 for traditional public schools), charter schools earned higher median API growth at the elementary, middle, and high school levels by an average of 56 points 5. Identical to the API base analysis, that difference was also smallest at the elementary school level where the charter median API growth was 738 and the traditional school API growth was 729 (a difference of 9 points) and greatest at the middle school level where the charter median API growth was 729 and the traditional school median API growth was 634 (a difference of 95 points). At the high school level, charter schools in LAUSD median API growth was 682 and that of traditional high schools was 618 (a different of 64 points). As noted above, here too the decreasing trend in performance from elementary to high school persists. Charter schools in Los Angeles Unified also did a better job of meeting their API targets than traditional public schools. All three types of API targets were assessed and are described in Table 3. Comparable improvement targets measure if the school met their growth targets for student subgroups for which they had enough students African American, Latino, SES disadvantaged, EL, and students with disabilites. School wide API targets capture the school s aggregate growth goal for the year and the both variable calculates the percentage of schools who met both types of targets. Overall, charter schools in LAUSD were 12% more likely to meet comparable improvement API targets, 6% more likely to meet their school wide targets, and 9% more likely to meet both types of targets than traditional public schools. 9 Charter School Performance in Los Angeles Unified School District

11 Key Findings Overall, charter schools in LAUSD also did a better job than traditional public schools of meeting AYP, but only by one percentage point (69% of charter versus 68% of traditional public schools). This difference was similarly small at the elementary school level (77% of charter versus 76% of traditional public schools) and reversed at the high school level (58% of charter versus 61% of traditional public schools). Charter middle schools, however, were 24% more likely to meet AYP than traditional middle schools (64% of charter versus 40% of traditional public schools). 5 The median statistic does not control for the sample size, therefore the four calculations are not interrelated. The middle point in a distribution of districtwide (98) cases will be different than the middle points of the elementary (42), middle (21), and high school (35) cases respectively. Table 3: Percent of LAUSD Charter & Traditional Public Schools Meeting API Growth Targets Type of School Number of Schools Number (and %) of Schools with API Growth Data Percent that Met Both School Wide and Comparable Improvement Targets Percent that Met School Wide Targets Percent that Met Comparable Improvement Targets Overall Charter (77%) 54% 72% 57% Traditional (90%) 45% 66% 45% Elementary Charter (78%) 69% 83% 75% Traditional (92%) 54% 74% 55% Middle Charter (86%) 22% 56% 22% Traditional (98%) 14% 33% 14% High Charter (71%) 56% 68% 56% Traditional (74%) 14% 55% 14% LAUSD charters did a better job of meeting API targets Source: California Department of Education data; California Charter Schools Association Analysis June 2008 California Charter Schools Association 10

12 Key Findings Matched Comparison Analysis Looking specifically at the matched comparison analysis sample of charter in LAUSD and traditional public schools, 41.2% of sampled charter schools earned higher 2006 API base than their three traditional matched peers and 44.1% earned higher 2006 API base than their most similar traditional match. Those percentages greatly improve for API growth, where 62.8% of charter schools in LAUSD in the sample earned higher 2007 API growth than their three traditional matched peers and 70.6% earned higher 2007 API growth than their most similar traditional match. Charter schools compared performance on API change was less impressive. Only 49% of sampled charter schools earned higher 2007 API change than their three traditional matched peers and 47.1% earned higher 2007 API change than their most similar traditional match. At the aggregate level, these charter schools performed better on API base by 14 points, API growth by 37 points, and on API change by 7 points than their traditional matched peers. Regression analyses showed a significant effect in favor of charter schools for API change, indicating that being a charter predicted 7.70** more points in API change. Controlling for demographic variables, regression analyses showed no significant results for API base or growth. Without those controls, charter schools in LAUSD predict 58.71*** more API growth points than their traditional school counterparts. 11 Charter School Performance in Los Angeles Unified School District

13 Key Findings Charter schools in LAUSD are more likely than traditional public schools to improve their Academic Performance Index (API) at a faster rate. In addition to assessing schools API growth scores, we also identified schools that increased their API growth score by 50 points or more. This analysis demonstrated that charter schools in LAUSD were more likely to achieve significant increases in their annual API growth than traditional public schools. Eighteen (18) percent of charter schools increased their API growth score by 50 points or more while only five (5) percent of traditional public schools did the same. Among this 18 percent of charter schools, the average increase in API growth was 78 points (73 points for traditional public schools) and the median increase was 72 points (64 for traditional public schools). This key finding is further supported by an examination of API change for LAUSD schools. API change, which is the 2007 API growth minus the 2006 API base, was higher for charter schools in LAUSD, indicating a faster rate of growth for these charter schools (see Graph 2 below). The charter versus traditional school API change difference is greatest at the high school level (10 points) and averages 8 points across the three grade levels. Charter schools in LAUSD rate of growth is also evidenced in the matched comparison analysis by the increase in the percent of charter schools outperforming their traditional matched peers from API base to API growth. The percent of charter schools outperforming their three traditional matched peers increased from 41.2% on API base to 62.8% for API growth (a 21.6 percentage point increase). Likewise, the percentage of charter schools outperforming their most similar traditional matched peer increased from 44.1% on API base to 70.6% for API growth (a 26.5 percentage point increase). These increases in percentages and thus in the number of schools outperforming their traditional matched peers highlight charter schools ability to increase their academic performance over the course of a school year. Graph 2: Median API Change from 2006 Base to 2007 Growth for LAUSD Public Schools Charter schools in LAUSD had greater API change scores indicating a faster rate of growth for charter schools Source: California Department of Education data; California Charter Schools Association Analysis June 2008 California Charter Schools Association 12

14 Key Findings API for African American students is higher in charter schools in LAUSD than in traditional public schools. API results for other traditionally disadvantaged groups are higher at the middle and high school levels, but not at the elementary school level. African American, Latino, and White Students An assessment of 2006 API base scores for numerically significant subgroups showed that Los Angeles Unified charter schools add value to the African American academic experience, as well as the middle and high school academic experience across subgroups (see Table 4 below). Charter schools in LAUSD earned higher median API base for African American (691 for charter and 637 for traditional public schools) and Caucasian (868 for charter and 847 for traditional public schools) students across all grade levels. For Latino students, this trend of higher median API base was found at the middle (88 points higher) and high school levels (94 points higher), but not at the district (30 points below traditional public schools) or elementary school levels (9 points below traditional public schools). On average, charter schools in LAUSD outperform traditional public schools in 2006 median API base scores by 82 points for African American students and 50 points for Caucasian students across the three grade levels. While Latino students elementary API base is lower than that of traditional schools, it is greater at the middle and high school levels. Overall, charter schools median API base is higher than that of traditional public schools for all three racial subgroups at the middle (an average of 67 greater points) and high school levels (an average of 77 greater points). Table 4: Median 2006 API Base by Subgroup and Grade Level for LAUSD Public Schools Type of School African American Latino Caucasian SES Disadvantaged English Learner Students with Disabilities LAUSD Charter Traditional Elementary Charter * Traditional Middle Charter Traditional High Charter Traditional *Insufficient test data 13 Charter schools in LAUSD earned higher median API base for many subgroup populations Source: California Department of Education data; California Charter Schools Association Analysis Subgroups are specified by CDE; Racial subgroups assessed represent the majority populations in LAUSD Charter School Performance in Los Angeles Unified School District

15 Key Findings An assessment of 2007 API growth scores yields a very similar pattern as that of the median API base scores (see Table 5 on page 16). Charter schools in LAUSD earned higher median API growth than traditional public schools for African American (718 for charter and 644 for traditional public schools) and Caucasian (858 for charter and 854 for traditional public schools) students across all grade levels. For Latino students, this trend of higher median API growth was found at the middle (723 for charter and 628 for traditional public schools) and high school levels (688 for charter and 602 for traditional public schools), but not at the elementary school level (701 for charter and 722 for traditional public schools). On average, charter schools in LAUSD outperform district traditional public schools in 2007 API growth scores by 80 points for African American students and 52 points for Caucasian students across the three grade levels. Overall, charter schools median API growth is higher than that of traditional public schools for all three racial subgroups at the middle and high school levels. An examination of the achievement gap, comparing the median API base and growth scores for Caucasian students to that of African American and Latino students, showed that the racial achievement gap persists for both charter and traditional public schools. Charter schools in LAUSD were doing a better job of closing that gap for African American students, evidenced by a 177 point gap in API base and a 140 point gap in API growth for charter schools versus a 210 point gap in API base and growth in traditional public schools (see Graph 3). Traditional public schools, however, are doing a better job of closing the racial achievement gap for Latino students (a 189 point gap in API base and a 161 point gap in API growth for charter schools versus a 189 point gap in API base and a 149 point gap in API growth for traditional public schools). Those gap differentials, however, are not equal in size. While the traditional public school black-white gap in median API growth scores is 70 points higher than that in charter schools, the charter white-latino gap is only 12 points higher than that in traditional public schools. The racial achievement gaps in charter schools decrease from API base to growth for both African American and Latino students, while remaining unchanged for African American traditional school students. Graph 3: LAUSD Charter vs. Traditional Median API Racial Gaps Charter schools in LAUSD have a smaller White/Black gap for median API & do a better job of decreasing it. Source: California Department of Education data; California Charter Schools Association Analysis Subgroups are specified by CDE; Racial subgroups represent the majority populations in LAUSD June 2008 California Charter Schools Association 14

16 Key Findings It is also important to note the trend of decreasing performance from elementary to middle to high school that was discussed earlier. Here too this trend persists for both charter and traditional public schools, however, with one exception: Charter middle schools in LAUSD API scores for Latino students increased from the elementary to middle school scores. That increase, however, does not continue at the high school level, where instead the trend of decreasing performance persists. Socioeconomically Traditionally Disadvantaged Students Another important subgroup to consider is students who are socioeconomically disadvantaged. A similar pattern was found with this subgroup as with the racial subgroups just discussed (see Table 5 on page 16). Charter schools in LAUSD median API base scores were greater than that of traditional public schools at the district (by only 1 point), middle (by 89 points), and high school levels (by 86 points). While charter schools in LAUSD median API growth scores were not greater at the aggregate district level nor the elementary school level, they were greater than that of traditional public schools at the middle (720 for charter and 627 for traditional public schools) and high school levels (678 for charter and 615 for traditional). Here, the trend of decreasing performance persists for traditional public schools but not for charter schools. Socioeconomically traditionally disadvantaged students attending charter schools in LAUSD demonstrate increased median API scores from elementary to middle school. However, that trend does not continue at the high school level. English Learners Median API results for English learners were very similar to that of socioeconomically traditionally disadvantaged students. Charter schools in LAUSD median API base scores for EL students were greater than those of traditional public schools at the middle (102 greater points) and high school levels (91 greater points), but not at the aggregate district level nor the elementary school level (see Table 4 on page 13). Charter schools in LAUSD median API growth scores for EL students followed the same pattern being greater than those of traditional public schools at the middle (656 for charter and 600 for traditional public schools) and high school levels (660 for charter and 558 for traditional), but not at the aggregate district level nor the elementary school level (see Table 5). The trend of decreasing performance persists for traditional public schools. EL students attending charter schools in LAUSD demonstrate an increase in median API base of 50 points from elementary to middle school and a small increase (4 points) in median API growth from middle to high school. Students with Disabilities Students with disabilities attending charter schools in LAUSD earned higher median API base scores than traditional public schools at the aggregate district level (106 greater points), the middle school level (173 greater points), and the high school level (131 greater points). This pattern continues for median API growth, where charter schools earned higher scores than traditional public schools at the aggregate district level (596 for charter and 447 for traditional public schools), the middle school level (596 for charter and 433 for traditional public schools), and the high school level (574 for charter and 408 for traditional public schools). The number of charter school students tested from this subgroup was insufficient to produce data at the elementary school level. While these API scores are low for charter and traditional public schools, the charter-traditional score differences are substantial. The trend of decreasing performance persists for these students as well. 15 Charter School Performance in Los Angeles Unified School District

17 Key Findings Table 5: Median 2007 API Growth by Subgroup and Grade Level for LAUSD Public Schools Type of School African American Latino Caucasian SES Disadvantaged English Learner Students with Disabilities LAUSD Charter Traditional Elementary Charter * Traditional Middle Charter Traditional High Charter Traditional *Insufficient test data Source: California Department of Education data; California Charter Schools Association Analysis Subgroups are specified by CDE; Racial subgroups assessed represent the majority populations in LAUSD Charter schools in LAUSD earned higher median API base for many subgroup populations June 2008 California Charter Schools Association 16

18 Key Findings Charter middle schools in LAUSD consistently outperform traditional public schools Throughout this analysis, charter middle schools in LAUSD have consistently outperformed traditional public schools at the highest levels.charter middle schools in LAUSD median API base was 100 points higher than that of traditional public schools. Median API growth differences were greatest at the middle school level as well (95 points greater than traditional middle schools). Furthermore, charter middle schools were 24% more likely to meet AYP than traditional middle schools. Among the subgroups assessed, charter middle schools in LAUSD demonstrated the highest median API base scores for Latino, socioeconomically disadvantaged, EL, and students with disablities as well as for API growth with Latino and students with disablities. Median API scores increase as charters in LAUSD mature. In , young charter schools in LAUSD had strong results increasing student achievement. As discussed in the methodology, charter maturity is an important variable that often distinguishes schools API performance. This distinction continued to hold true in the API results. Mature charter schools in LAUSD outperformed young charter schools and traditional public schools on API base and growth (see Graph 4). While traditional public schools earned 13 greater points than young charter schools, mature charter schools earned 92 greater points than traditional public schools. This trend continues at the elementary and high school levels, but shifts at the middle school level such that young charter schools outperform mature charter schools and traditional public schools (see Table 6). For 2007 API growth, traditional public schools earned 6 greater median API growth points than young charter schools, while mature charter schools earned 69 greater points than traditional public schools (see Table 6 and Graph 4). This trend continues at the elementary school level, where traditional public schools earned 38 greater points than young charter schools and mature charter schools earned 59 greater points than traditional public schools. The order of performance between traditional, young charter, and mature charter schools changes at the middle and high school levels for median API growth. For middle schools, young charter schools in LAUSD outperform mature charter schools (by only 1 point) and traditional public schools (by 96 points). These mature charter schools also substantially outperform traditional public schools (by 95 points). For high schools, the trend shifts yet again with mature charter schools outperforming young charter schools (by 101 points) and traditional public schools (by 163 points), and young charter schools outperforming traditional public schools as well (by 62 points). 17 Charter School Performance in Los Angeles Unified School District

19 Key Findings Taken together, these results show that charter schools in LAUSD API growth performance increases as charter schools age. The exact opposite is true for 2007 API change scores (see Table 6 and Graph 4). Young charter schools earned greater API change scores than mature charter schools (by 6 points) and traditional public schools (by 8 points), except at the middle school level where mature charter schools outperform young charter schools (by 7 points) and traditional public schools (by 10 points). Across grade levels, charter schools outperform traditional public schools in API change scores (see Table 6). Table 6: Median API Base, Growth, and Change by LAUSD Charter Maturity Young Charter Schools Mature Charter Schools Traditional Public Schools 2006 Median API Base LAUSD Elementary Middle High Mature charter schools in LAUSD typically outperform all other schools on median API base 2007 Median API Growth LAUSD Elementary Middle High Mature charter schools in LAUSD typically outperform all other schools on median API growth 2007 Median API Change LAUSD Elementary Middle High Young LAUSD charter schools typically outperform all other schools on median API change Source: California Department of Education data; California Charter Schools Association Analysis Graph 4: Median API Base, Growth, and Change by LAUSD Charter Maturity LAUSD charter maturity affects median API score outcomes Source: California Department of Education data; California Charter Schools Association Analysis June 2008 California Charter Schools Association 18

20 Key Findings Matched Comparison Analysis Maturity also proved to be an important factor in the matched comparison analysis. The percent of charter schools in LAUSD outperforming their matched traditional peers in API base, growth, and change scores is generally greater for mature charter schools (see Table 7). This was true in every case except for the percentage of charter schools earning higher API growth scores than their most similar matched traditional peer, which was 0.6 percentage points lower than for all charter schools in the sample. Table 7: Percent of LAUSD Charter Schools Earning Higher Scores than Their % Earned Higher Score than Their 3 Matched Traditional Peers % Earned Higher Score than Their Most Similar Matched Traditional Peer 2006 Median API Base 2007 API Base All Charter Schools in the Sample 41.2% 44.1% Mature Charter Schools in the Sample 70% 65% 2007 API Growth All Charter Schools in the Sample 62.8% 70.6% Mature Charter Schools in the Sample 65% 70% 2007 API Change All Charter Schools in the Sample 49% 47.1% Mature Charter Schools in the Sample 55% 55% Mature charter schools in LAUSD are more likely to outperform their matched traditional public school peers Source: California Department of Education data; California Charter Schools Association Analysis 19 Charter School Performance in Los Angeles Unified School District

21 Discussions and Implications In this analysis of California State Academic Performance Index data, charter schools in the Los Angeles Unified School District demonstrated greater academic performance than traditional public schools in a variety of key areas. Under the key finding one, we saw that charter schools in LAUSD outperformance was demonstrated in median API base, growth and change scores, meeting API and AYP targets, making substantial gains in API growth, and in a matched comparison analysis of district schools. This data, while only descriptive, indicates that charter schools in LAUSD are doing a better job of serving the same students. The population of students in independent charter schools 5 and traditional schools is similarly diverse (see Graph 1 on page 8); however, the data suggests that charter schools in LAUSD more effectively serve that similar demographic population. This finding, as well as the demographic data, helps to refute the creaming theory often lobbied against charter schools, accusing them of primarily serving more advantaged students. This analysis of public schools in Los Angeles Unified suggests otherwise and supports the continued expansion of charter schools in LAUSD. These charter schools seem to be fulfilling the promise of expanded educational options from the 1992 charter legislation by offering improved school choices for families and students. Key finding two demonstrated the faster rate at which charter schools in LAUSD were improving their API scores. This may be a result of the greater autonomy and flexibility charter schools often enjoy, which likely allows them to innovate quickly to meet students needs. More research is needed to understand how charter schools autonomy affects their academic results. In addition to demonstrating greater academic performance in aggregate, key finding three showed that charter schools in LAUSD also displayed important benefits for the district s student subgroup populations. The value-add of charter schools was particularly strong and consistent for African American students. African American charter school students earned higher API base and growth scores with impressive point differences and charter schools in LAUSD did a better job of closing the racial achievement gap between black and white students. Charter schools in LAUSD also did a better job than traditional public schools of serving the other middle and high school subgroup populations; this was true for Latino, socioeconomically disadvantaged, English learner (EL), and students with disablities. These findings are particularly important given the achievement and opportunity gaps pervasive in California, and help to further refute the argument that charter schools serve a specialized and more privileged population of students. Further research needs to be done to clearly identify exactly who charter school students are (demographically), in more detail than currently exists or is possible with test data. Research should also be done to better understand why these charter schools are so effective with African American students to identify the structures and practices that lead to this very important success. The data presented under key finding four also revealed charter schools in LAUSD capacity to serve the secondary school populations well. Charter middle schools were more likely to meet AYP and earned some of the highest median API scores in the sample. Charter middle schools were also the one population where young charters outperformed the others. Furthermore, charter middle and high schools consistently outperformed traditional public schools among the subgroup populations, and were also the only subpopulations to break the trend of decreasing performance typically found as students progress from elementary to high school. These results were particularly pronounced at the middle school level, demonstrating that charters in LAUSD are working very effectively with this population. More research is needed to identify the structural and curricular practices being used in these successful middle schools and how they might be replicated, particularly in charter elementary schools. June 2008 California Charter Schools Association 20

22 Discussions and Implications Key finding five demonstrated the importance of a charter s maturity, highlighting a charter s age as an essential variable in any analysis of charter schools. As is true at the aggregate state level, charter schools in LAUSD improve their performance with time, earning higher API base and growth scores as they mature. Equally important, young charter schools earned higher API change scores, displaying greater academic progress in the course of one school year. As new schools, charter schools should reasonably be given time to establish themselves in comparison to traditional public schools with many years of history. However, this data indicates that if you give them the necessary time to solidify their academic program, charter schools in LAUSD will outperform traditional public schools. Furthermore, results for the mature charter schools indicate that it is not simply more years of operation that leads to greater success, but rather more years of effective innovation which distinguishes mature charter schools from mature traditional schools. The finding of young charter schools higher median API change further supports the charter movement argument that flexibility and autonomy are tools for school improvement. Future research and policy efforts should try to identify the factors that help charter schools build a strong program quickly, to aid charter schools progress to maturity. Amidst the findings of LAUSD charter effectiveness throughout this analysis, some areas of improvement were also uncovered. LAUSD charter results at the elementary school level were much less consistent and not as positive. Elementary charter schools median API scores were lower than that of traditional public schools for the socioeconomically disadvantaged, EL, and disabled subgroups and the difference between elementary charter and traditional public schools was small at the district level. Charter schools in LAUSD effectiveness with Latino students was also inconsistent. These charter schools did not outperform traditional public schools with elementary Latino students or in aggregate with this subgroup, and traditional public schools were doing a better job of closing the White-Latino achievement gap. While there were some exceptions for charter schools, in general charters in LAUSD shared the trend of decreasing performance found in traditional public schools. At each grade level, school performance as measured by median API scores decreased. Analysis Limitations Being primarily a descriptive analysis, this report cannot provide an understanding of how significant the results discussed are or the relationships that may exist among the various variables that were assessed. That said, point differences in API measures relate to actual student achievement on the micro level and are very significant to the children and families they represent. This analysis is also limited by the use of aggregate school data, which cannot describe the type of progress actual students may be making or the myriad of school quality variables that are not assessed by state tests. Student level data would likely provide charter schools a better opportunity to demonstrate the value they add to individual students lives. 5 Independent charter schools are the appropriate comparison to traditional public schools because they operate outside of district auspices and are genuinely non-district schools that more accurately fit the charter school mission of serving underserved students well. They also serve a similar population as traditional public schools. 21 Charter School Performance in Los Angeles Unified School District

23 Conclusion This analysis paints a clear picture of the many ways charter schools in Los Angeles Unified are promoting students academic enrichment, as well as ways the local charter school movement in Los Angeles can improve to effect even greater change. Future research should analyze student level data to better demonstrate the value charter schools add by following actual students performance over time. Researchers should also conduct inferential analysis (mean differences and regressions with significance statistics) to further support these Los Angeles Unified findings. Case studies of successful charter middle schools and charter schools serving African American students are also needed to tie actual structures and practices to the academic performance of these two populations. The Los Angeles charter movement would also benefit greatly from an assessment of the charter student population in detail to further prove charter schools diversity and success with diverse populations. Charter schools in Los Angeles Unified provide academic rigor and educational choice, and therefore are an educational option to invest in and further expand. The Los Angeles charter movement serves as a good model of effective educational reform for other charter movements across the nation. June 2008 California Charter Schools Association 22

24 Information References Hoxby, C.M. (2004). Achievement in charter schools and regular public schools in the United States: Understanding the differences. Harvard University and the National Bureau of Economic Research. Roy, J., & Mishel, L. (2005). Advantage none: Re-examining Hoxby s finding of charter school benefits. Economic Policy Institute: Washington, D.C. California Department of Education data: California Charter Schools Association member database (imis). Appendix I. Table 1: List of LAUSD Traditional Public Schools Incorrectly Coded as Charter schools II. Table 2: LAUSD School Population Numbers for Each Data Cleaning Iteration III. Report Outline IV. Terms and Definitions V. Frequently Asked Questions and Their Answers 23 Charter School Performance in Los Angeles Unified School District

25 Information Table 1: List of LAUSD Traditional Public Schools Incorrectly Coded as Charter Schools School Name CDS Code District Date Closed Crenshaw/Dorsey: Audubon Charter Middle School Los Angeles Unified 7/1/2003 Crenshaw/Dorsey: Baldwin Hills Charter Elementary Los Angeles Unified 7/1/2003 Crenshaw/Dorsey: Tom Bradley Environmental Science Los Angeles Unified 7/1/2003 Crenshaw/Dorsey: Coliseum Street Elementary School Los Angeles Unified 7/1/2003 Crenshaw Learn Charter High School Los Angeles Unified 7/1/2003 Crenshaw/Dorsey: Fifty-Fourth Street Charter Elementary Los Angeles Unified 7/1/2003 Crenshaw/Dorsey: Fifty-Ninth Street Elementary Los Angeles Unified 7/1/2003 Crenshaw/Dorsey: Forty-Second Street Charter School Los Angeles Unified 7/1/2003 Crenshaw/Dorsey: Hyde Park Charter School Los Angeles Unified 7/1/2003 Crenshaw/Dorsey: Marlton Charter School Los Angeles Unified 7/1/2003 Crenshaw/Dorsey: Mid-City Charter Magnet School Los Angeles Unified 7/1/2003 Crenshaw/Dorsey: Seventy-Fourth Street Charter School Los Angeles Unified 7/1/2003 Crenshaw/Dorsey: Sixth Avenue Elementary Los Angeles Unified 7/1/2003 Palisades: Temescal Canyon Continuation High School Los Angeles Unified 7/1/2000 Crenshaw/Dorsey: View Park Continuation High School Los Angeles Unified 7/1/2003 Crenshaw/Dorsey: Virginia Road Charter Elementary Los Angeles Unified 7/1/2003 Crenshaw/Dorsey: Western Avenue Charter School Los Angeles Unified 7/1/2003 Crenshaw/Dorsey: Whitney Young Continuation High School Los Angeles Unified 7/1/2003 June 2008 California Charter Schools Association 24

26 Information Table 2: LAUSD School Population Numbers for Each Data Cleaning Iteration 25 Charter School Performance in Los Angeles Unified School District

27 Report Outline I. Executive Summary II. Introduction a. Charter school definition and brief history of the California charter movement b. CCSA history and role in the movement c. Study methodology III. Results a. Los Angeles Unified charter schools outperform traditional public schools on a variety of student achievement measures. LAUSD charter schools had higher median API base overall and across grade levels LAUSD charter schools had higher median API growth across grade levels, but not at the district level A greater percentage of LAUSD charter schools met comparable improvement (sub-group) targets, across grade levels A greater percentage of LAUSD charter schools met school wide targets across grade levels A greater percentage of LAUSD charter schools met both targets across grade levels Matched Comparison Analysis: 41.2% of LAUSD charter schools earned higher 2006 API base (70% mature and 34.2% young charter schools) than their three matches o and 44.1% higher than their most similar match (65% mature and 41.5% young charter schools) 62.8% of LAUSD charter schools earned higher 2007 API growth (65% mature and 62.2% young charter schools) than their three matches o and 70.6% higher than their most similar match (70% mature and 73.2% young charter schools) 49% of LAUSD charter schools earned higher API change (55% mature and 47.6% young charter schools) than their three matches o and 47.1% higher than their most similar match (55% mature and 51.2% young charter schools) Overall, these charter schools performed better on API base (14 points), growth (37 points) and change (7 points) averages Regression showed being a charter predicted 7.70** more points in API change and no significant results for base or growth. Without demographic controls, LAUSD charter schools predict 58.71*** more API growth points than their traditional school counterparts. b. LAUSD charter schools are more likely than traditional public schools to improve Academic Performance Index (API) at a faster rate. A greater percentage of charter schools had 50 points or higher API growth increase (18% charter to 5% traditional) Charter schools are improving student achievement at a faster rate evidenced by higher API change scores June 2008 California Charter Schools Association 26

28 Report Outline Matched Comparison Analysis: The percent of charter schools outperforming their matched traditional peers on API base increased for API growth. This increase from base to growth in the charter percentages was 21.6% for three matches and 26.5% for the most similar match. c. API for African American students is higher in LAUSD charter schools than in traditional public schools. API results for other traditionally disadvantaged groups are higher at the middle and high school levels, but not at the elementary school level. LAUSD charter schools had higher median API base at the district level for African American and white students, but not for Latino students o The African American median API base was greater across grade levels o The white median API base was greater across grade levels o The Latino median API base was greater for middle and high school and the traditional public schools was greater at district and elementary school levels o The middle and high school median API base was greater across racial groups LAUSD charter schools had higher median API growth for African American and white students across all grade levels. For Latino students median API growth was higher at the middle and high school levels, but not at the elementary school level; o Charter schools median API growth is higher for all three racial groups at the middle and high school levels, but not at the elementary school level LAUSD charter SES traditionally disadvantaged students API growth is greater in middle and high school, not elementary or overall o API base is greater overall, and in middle and high school, but not in elementary school LAUSD charter EL students API growth is greater in middle and high school, not in elementary school or overall o API base is greater in middle and high school, not in elementary school or overall LAUSD charter students with disablities API growth is greater in middle and high school and overall (no data for elementary) o API base is greater overall, and in middle and high school (no data for elementary) d. API growth increases as LAUSD charter schools mature. In , young LAUSD charter schools had strong results increasing student achievement. Mature charter schools outperform non-mature charter schools and traditional public schools on API base o Except in middle school, where non-mature outperforms mature; still both mature and nonmature outperform traditional 27 Charter School Performance in Los Angeles Unified School District

29 Terms and Definitions About the API: The purpose of the Academic Performance Index (API) is to measure the year-over-year growth in academic performance for California schools. The API summarizes a school s standardized test scores into a single number, which ranges from 200 to The statewide API goal is 800 for all schools; higher numbers generally indicate better performance on the tests. Why do the API results matter? As a fundamental part of the Public Schools Accountability Act (PSAA), API scores are used to meet state and federal requirements for school accountability and are an important component for measuring a school s Adequate Yearly Progress (AYP) under the federal No Child Left Behind (NCLB) Act. These ratings can have substantial consequences for schools. Under-performing schools are given additional funds to encourage improvement and high-ranking schools may be eligible for additional acknowledgement. Why do some schools not have API s? There are several reasons why some schools don t have API results. For example, if a school failed to test at least 85% of eligible students or if the school tested fewer than 11 students, the state does not calculate an API for that school. Schools that serve specific kinds of students, such as special education schools, are currently accountable under a different model. Base API: The 2006 Base API was calculated using each school s test results from the state s Standardized Testing and Reporting (STAR) program and the California High School Exit Examination (CAHSEE) taken during the school year. The Base API is used to measure academic improvement from one year to the next by comparing it to the Growth API. Growth API: The 2007 Growth API was calculated using the same test criteria as the 2006 Base API. However, the 2007 Growth API was calculated from the standardized test results, while the 2006 Base was calculated using the test results. The Growth API is used as a measure of improvement in academic performance when compared to the Base API. Growth in API (Change): Growth from the beginning to the end of a school year on the 2007 API is calculated by subtracting the 2006 Base API from the 2007 Growth API. API Growth targets: Each school is assigned an API Growth target by the state. The Growth target is determined by calculating 5% of the difference between the Base API and 800. Schools with APIs at or above 800 must maintain a minimum API of 800. A school meets its overall API Growth target if it meets its schoolwide target and its comparable improvement (all numerically significant subgroups) targets. Improvement is measured by subtracting the 2006 Base API from the 2007 Growth API. Positive numbers mean that standardized test scores improved, while negative numbers indicate that test scores declined. To meet comparable improvement targets and state API growth target requirements, each numerically significant subgroup in a school must demonstrate comparable improvement in meeting API targets. This concept applies to ethnic/racial, socioeconomically disadvantaged, English learner, and students with disabilities subgroups. A subgroup must be numerically significant in both the Base year and Growth year in an API reporting cycle to have subgroup growth and target information. June 2008 California Charter Schools Association 28

30 Terms and Definitions DataQuest Will give you more specific information on California schools and districts API Home AYP Home Accountability Progress Reporting (APR) Includes links to both API (state accountability) and AYP (federal accountability) EdData (Education Data Partnership) Fiscal, demographic, and performance data on California s K-12 schools EdSource Credible and respected source of K 12 education information, research, analysis, and data 29 Charter School Performance in Los Angeles Unified School District

31 FAQ & A Frequently Asked Questions and Their Answers Q: Don t charter schools pick/recruit the best students? A: No, charter schools serve a diverse demographic. Q: Why is the Association s data different from CDE s? A: The Association and CDE s method of analysis are different. The Association utilizes a rigorous cleaning process to ensure only active charter schools are included; our sample size includes small schools and all funding types. Furthermore, the information reflected is from the CDE s original August release of the 2007 Growth Data File. Q: Are the results significant? A: This analysis is used for descriptive purposes only. Q: What are your methods? A: The matched comparison analysis began with identifying three peer schools for each charter in the sample. Charter to peer matching was done using Geographical Information Systems (GIS) software to map all of California s schools and their demography. Each charter school with available demographic and API test data was matched with three proximal (within a five-mile radius) traditional public schools most similar in racial composition, parental educational attainment, and free/reduced-price lunch participation. This matching yielded traditional public schools for the Los Angeles regional analysis. Alternative and special education schools are excluded for both charter schools and non-charter schools. Q: How is MCA different than Similar School Rank? A: Similar School Rank is limited to only academic performance comparisons, whereas the Association s MCA analysis is based on geography and socioeconomic factors. This is critical, as it presents realistic school choice for parents. By comparing a charter against its neighboring traditional public school allows real data for student and parents to decide on school attendance. June 2008 California Charter Schools Association 30

32 FAQ & A CDE s Similar School Rankings are the school s API Base compared with the Base APIs of other similar schools in the comparison group (based on demographic School Characteristics Index*). Q: How are Matched Comparison Analysis comparison schools different than LAUSD s comparison schools? A: The Association utilized GIS software to match a charter school based on a three-to-five mile radius, parent educational level, and the percentage of students on the Free/Reduced Meals Program. In contrast to gauge the success of the charter school during the renewal term, LAUSD s Research and Evaluation division formerly known as PERB calculates a group of comparison district schools to be selected that meet the following criteria. The District will identify the comparison schools and will inform the charter of the names of the schools and the specific data used to identify them. The analysis of the school s academic performance will include a comparison of the academic achievement of the charter school s students to the academic achievement of two sets of comparison District schools: 1. The resident schools the charter school students would have otherwise attended ( Resident Schools ); and 2. District schools of similar demographic characteristics ( Demographically Similar Schools ). Comparison resident schools will be selected by using the charter school students home addresses to identify the District schools they otherwise would have attended. The District schools most represented at the charter school will be chosen as comparison resident schools. Demographically Similar Schools will be selected by using a modified version of the formula utilized by the state for creating its similar schools list. 31 Charter School Performance in Los Angeles Unified School District

33 FAQ & A Q: Why aren t all your charter schools represented in your sample? A: This is based on our methodology as noted above. Q: What about other studies and online sources (e.g., GreatSchools)? A: Each institution employs a different methodology. The Association prides itself on utilizing a rigorous crossreference and identification of missing school information to gather an accurate population of school level data from the CDE. Before we conduct any analyses, we engage in a rigorous data cleaning process, comparing the state s API and AYP data files to our internal charter database. After merging the state file with our database, any identified discrepancies are resolved by working with our knowledgeable General Managers who have a close relationship with their regional charter schools. Additionally, we work in partnership with the Administrator of the Academic Accountability Unit at the CDE to clarify questions and permanently correct issues identified by the School Achievement Data Team. Detailed steps of this cleaning methodology are outlined below. Q: What about the 2007 Base results? A: 2007 Base data will be released in mid-may. June 2008 California Charter Schools Association 32

34 About Us About the California Charter Schools Association The California Charter Schools Association is the membership and professional organization serving charter schools in California. The Association s mission is to increase student achievement by supporting and expanding California s quality charter public school movement. The Association advances the charter school movement through state and local advocacy efforts, leadership on quality and extensive resources. The Association began operations in 2003 launched from the work of its predecessor organization, the California Network of Educational Charter schools (CANEC), which was in operation from Charter School Performance in Los Angeles Unified School District