2015 WAITLIST DATA & METHODOLOGY

Transcription

1 2015 WAITLIST DATA & METHODOLOGY A. Introduction: How do we estimate the unmet demand for charter schools? There are not enough charter schools to accommodate every student who applies. In order to address this unmet demand in California, CCSA seeks to estimate the number of students on charter school waitlists. CCSA developed its methodology in 2013 in cooperation with state and national charter association leaders from New York City and the National Alliance for Public Charter Schools. We continuously improve our process of data collection and analysis, allowing us to be increasingly confident in our estimates. CCSA can now trace the growth of charter demand in California over three years. Every year, CCSA launches a Fall Data Campaign in which we distribute a survey to all active charters. We contact all school administrators electronically, and we engage in several follow up attempts via and phone to maximize the overall response rate. A portion of the FDC is dedicated to our effort in understanding the scope of demand for charters. Last year, we asked Did your school hold an admissions lottery for the year? Additionally, we asked respondents to fill out a table with the given instructions: For the school year, please indicate the number of open seats and applications by grade level. If you do not know the exact totals, please estimate to the best of your ability. Many respondents misinterpreted the second question by inputting the number of total seats instead of number of available seats. We corrected for the unclear questions in order to collect accurate responses for this year: Capacity: What is the enrollment capacity for your school in ? Continued Enrollment: How many students who attended your school in are enrolled in your school for ? New Applications: How many new applications for enrollment did you receive for the school year? B. Data: How many charters responded to the survey? This year, the response rate for our survey was 29% of all charters. 1 Table 1 below looks at the distribution of our survey results compared to the population of schools for which we are estimating waitlist data points. Generally, the survey sample falls within 0% 6% difference from the population distribution. However, it is important to note that our sample has 11% more representation on autonomous schools, 11% less representation in semi autonomous schools, 14% less representation in 1 29% (343/1,184) response rate for Capacity survey question, 30% response rate for Continued Enrollment and New Application survey questions

2 schools between 2 3 years old, 8% more representation in schools less than 1 years old, and 8% more site based schools. 2 Table 1. Descriptive Statistics: Sample Size Summary Autonomy Status Management Model Start Type Site Type Age Region Districts Survey Sample (N=354) Population (N=1,184) Difference Autonomous 81% 70% 11% Semi Autonomous 16% 27% 11% Non Autonomous 4% 3% 1% CMO 38% 32% 6% Freestanding 51% 55% 4% Network 11% 13% 2% Conversion 12% 17% 5% Start up 88% 83% 5% Combination 4% 6% 2% Independent Study 10% 16% 6% Site Based 86% 78% 8% < 1 years 12% 4% 8% 2 3 years 15% 29% 14% 4+ years 73% 67% 6% Greater LA 32% 32% 0% NE & Central Valley 24% 27% 3% North Coast & Bay Area 27% 24% 3% Southern California 17% 17% 0% LAUSD 25% 24% 1% OUSD 5% 3% 3% FUSD 1% 1% 0% SCUSD 2% 1% 1% SFUSD 2% 1% 1% SDUSD 4% 5% 1% C. Methodology How did CCSA account for 71% of the charters who did not respond to all of the survey questions? We estimated the rest of the population data by testing out a number of variables: Enrollment Management Model Level of Autonomy Age of the school 2 See Appendix for definition of key terms

3 Geographic Region Major Districts Academic Performance Given the nature of the count data, the appropriate statistical model of choice is a negative binomial regression. We used this model to adequately assess what variables are significant in predicting new applications and open seats based on the survey sample. 3 After arriving at the appropriate models, we used multiple imputation to calculate 100 data points for new applications and open seats for every school based on the model. We then take the average of the 100 imputed data points to arrive at our final new application and open seats numbers for the rest of the data. How did CCSA actually measure the unduplicated (i.e. discounted) waitlist for California? To quantify the number of students who wanted to attend a California charter school but could not do so, we: 1. Divided the number of applications in each charter school by a discount factor (1.63 if there was another charter serving the same grade levels within 10 miles, 1.00 if not) to obtain the number of applicants, i.e. demand. 4 This reflects the fact that applicants to non isolated charters can apply to multiple schools. 2. Estimated open seats by subtracting continued enrollment from capacity Multiplied the number of open seats by a discount factor (0.98) to obtain the number of open seats with a matching application, i.e. the supply. This reflects our estimate that 2% of open seats lack matching applications Subtracted the number of open seats with matching applications from the number of applicants in each school in order to calculate number of students on waitlists per school Summed the waitlists for all schools to obtain the total unduplicated (i.e. discounted) waitlist for California. Given that we are estimating over two thirds of the data, all counts of open seats, applications, and waiting lists should be regarded as best estimates. D. Results & Findings This section presents the estimated supply and demand for charter schools in Using the methodology described in Appendix A, missing responses for the three relevant survey questions were 3 See Appendix B for detailed information about statistical methodology, variables, etc. 4 See New York City Charter schools: Enrollment Lottery Estimates and NAPCS National Public School Waiting List Estimate for more details about the discount factor. 5 To correct for self reported error, we replace any negative values with zero. 6 Self reported grade level data from nearly 200 charter schools from CCSA s Fall Data Campaign showed that 2% of all open seats lacked matching applications. For example, a high school with a large waitlist for its freshman class might have no applications for its 10 open seats in 12 th grade. 7 Theoretically, schools cannot have negative waitlists. We replace any negative value as well as any positive value for charters that are fully virtual or virtual with an optional site based component with zero.

4 estimated. Therefore, the results in this section are intended to represent charter supply and demand among the entire universe of 1,184 CA charters active in the academic year We estimated a total of over 358,000 applications and 226,000 unique applicants for about 78,700 open seats in California charters schools in In other words, we estimate: 4.5 applications submitted for every open charter seat. Nearly 3 unique (i.e. unduplicated) applicants for every open seat. 2. The estimated unmet demand for charter schools is approximately 158,000 students. Given that we estimate 226,000 unique applicants and 78,700 open seats, it may appear that we should estimate 147,300 (226,000 78,700) students on waitlists instead of 158,000. The 10,700 student gap represents all schools in which there were no applicants to fill open available seats. For example, if a school had 10 open seats but only 1 applicant, we get a waitlist value of 9. This means that there are still vacant seats in the school. Since the 9 does not truly represent a waitlist, we give each school with negative waitlist value a value of zero; in other words, zero is an indicator for no waitlist for that school. 3. The table below compares waitlists to , broken up by geographic location. Table 2. California Charter School Estimated Waitlist by Geography, Charters w/ Data 8 Avg. Waitlist per Charter # Students on Waitlists Regions Greater LA ,883 53,200 Northeast & Central Valley ,237 42,800 North Coast & Bay ,836 26,700 Southern CA ,336 36,200 School Districts Los Angeles USD ,300 41,830 San Diego USD ,080 5,440 San Jose ,360 5,620 Oakland USD ,600 2,680 Counties Los Angeles ,000 49,840 San Diego ,680 16,470 Santa Clara ,830 6,390 Alameda ,360 4,520 Sonoma ,140 4,530 Sacramento ,330 6,380 San Joaquin ,780 4,940 Fresno ,630 4,090 San Bernardino ,210 7,530 Stanislaus ,660 8 We obtained self reported or estimated data for 1104 schools in and 1129 schools in See Appendix for more information. 9 Includes the following school districts: San Jose Unified, East Side Union High, and six elementary districts: Alum Rock Union, Cambrian, Campbell Union, Franklin McKinley, Moreland, Mt. Pleasant.

5 Riverside ,450 7,810 Placer ,630 4,240 Bay Area ,570 9,940 E. Limitations There are several important limitations of this data that may affect the accuracy of the findings: 1. The demand numbers in this brief are estimated based on the number of open seats and applicants as reported by charter schools. As such, they do not account for students and families who ultimately chose not to attend a charter school despite being offered admission. In other words, it is assumed that everyone offered admission elects to go to a charter school, which is likely to be untrue. a. Our methodology addresses this concern. We believe that the discount factor of 1.63 we applied is conservative, as it discounts the number of applications at most charters by close to 40% 2. Since the data we used to estimate 71% of California charter schools are self reported, we had to correct for self reported error. For example, we found that many people reported a lower maximum capacity rate than their total enrollment published on the California Department of Education. We correct these instances by replacing reported capacity with the total enrollment. 3. As mentioned in the introduction, Table 1 looks at our sample survey compared to the population distribution. The comparison shows that there are significant sampling biases in our data. For example, Autonomous, Semi Autonomous, Schools between 1 3 years old, CMOs, Site Based and Independent schools are all disproportionately represented in our survey compared to the entire population of California charters. All of the reasons above show that we make a set assumptions that may or may not be violated due to the nature of our data and methods. Since we are ultimately estimating a majority of the data, CCSA only releases estimates for groups of 20 or more charters. 10 Includes the following school 16 districts: San Francisco Unified, Oakland Unified, Hayward Unified, San Lorenzo Unified, West Contra Costa Unified, Mt. Diablo Unified, Pittsburg Unified, Antioch Unified, San Mateo Foster City, Redwood City Elementary, Ravenswood City Elementary, Alum Rock Union Elementary, East Side Union High, Franklin McKinley, San Jose Unified, and Novato Unified.

6 Appendix A Variables With the exception of the enrollment variables, we dummy coded each of the subsets of nominal variables: 1=yes, 0=no Variable Group Enrollment Management Model Autonomy Status Age of School Geographic Region Major Districts Independent Variables Academic Performance API below 700* API in the 700s API in the 800s API in the 900s NO API score * Reference category for variable group Variables total enrollment to total enrollment growth enrollment by grade level to enrollment growth by grade level to enrollment growth by grade level Charter Management Organization (CMO) Network Freestanding* Autonomous Semi Autonomous Non Autonomous* Age 1 or less Age 2 3 years* Age 4 or more years Greater LA Northeast & Central Valley* North Coast & Bay Area Southern California Los Angeles Unified LAUSD Oakland Unified OUSD San Francisco Unified SFUSD Sacramento City Unified SCUSD Fresno Unified FUSD San Diego Unified SDUSD In addition to the variables noted above, we wanted to assess whether there were any interaction effects. With interaction variables, we ask: are there added values in combination of two categories? For example, would schools in Greater LA Autonomous schools have a greater added value in waitlist than if we just considered those two variables separately? We interacted all nominal variables (Management level to Academic performance). Given that our sample is so small, we only included interaction variables in which at least 20% of the data points fell into that interaction category.

7 Interaction Variables Site Based X CMO CMO X Autonomous Autonomous X Schools 4 years or older Site Based X Schools 4 years or older Site Based X Autonomous Autonomous X North Coast & Bay Area Autonomous X LAUSD Site Based X North Coast & Bay Area Greater LA X Schools 4 years or older API in 700s X Schools 4 years or older Autonomous X API in 800s Autonomous X Greater LA Autonomous X API in the 700s Models The appropriate model to handle our count data is negative binomial regression. Our dependent variables are new applications and open seats. We first ran negative binomial regressions on each of the dependent variables, controlling for all of the variables listed above. We prune out the variables without statistical significance at a generous α=0.10. We run the models a second time with significant variables to check for redundancy in remaining variables. We prune a second time. We continue this process until all remaining independent variables are statistically significant. We arrive at the final models: Open Seats Independent Variables: 4 th grade enrollment in 2014, 6 th grade enrollment growth between 2012 and 2013, Northeast Central Valley, CMO X Age 4 & older interaction, Site Based School X Age 4 & older interaction New Applications Independent Variables: 2015 total enrollment, 12 th grade enrollment growth between 2012 and 2013, CMO, Autonomous, API in the 700s, API in the 800s, API in the 900s, Age 1 or less, North Coast & Bay Area, CMO X Age 4 & older interaction, and CMO X autonomous interaction. These are the final models we use to predict the 71% of schools with missing survey data. Multiple Imputation After we arrive at the appropriate models to estimate both new applications and open seats, we predict the 71% of missing data using these models. We use a method called multiple imputation to predict values 100 times for each of our independent variables based on the negative binomial regressions. We then average the 100 predicted values in order to arrive at one single value per school.

8 Final Waitlist Analysis Open Seats When new applications are collected, not every application matches perfectly with an open seat. For example, a student who is in the 4 th grade may submit an application for a school for her 5 th grade year. That school may have a total of 10 open seats, but no open seats for 5 th grade. CCSA did an internal study to determine the rate in which application match the open seats at any particular school. We account for this error by multiplying open seats by a value of Applications vs. Applicants It is likely that in order to increase chances of getting into a charter school, a parent may submit applications to multiple schools. To calculate the waitlist, we need to estimate a 1 to 1 match for applicants per open seats, but our data is duplicated (multiple applications per 1 student). New York City Charter School Center and National Alliance Public Charter Schools published reports on their waitlist numbers and how they addressed multiple applications. 11 We divide the new applications variable with a discount factor of 1.63, discounting approximately 40% of the original value. We only apply this discount factor for schools that are not isolated. 12 Waitlist To create the waitlist number, we take the corrected applicants number and subtract the open seats with match number. When we generate the waitlist, we have to correct for a few things. 1. As addressed in the results and findings (pg 3), there are some schools in which there are vacant seats at a particular school, but no applicants to fill those seats. Our formula would then generate a negative value for a waitlist. Every negative waitlist value actually represents that those particular schools do not have existing waitlists. We correct for this by replacing every negative waitlist value with a We assume that virtual schools have no maximum capacity. 13 For every virtual school with a generated waitlist number, we also replace those schools waitlists with a See New York City Charter schools: Enrollment Lottery Estimates & NAPCS National Public School Waiting List Estimate for more details about the discount factor. 12 Isolated schools are charters that do not have any other charters that serve any overlapping grades within a 10 mile radius. 13 By virtual schools, we mean charters that exclusively serve students online or charters that serve students online and offer some optional site based instruction.

9 Appendix B: Definitions of Charter School Types Management Model (CCSA Definition): CMO school: School that is part of a charter management organization (CMO), which is an organization that operates three or more schools linked by a common philosophy and centralized governance or operations. Network school: School that is part of a Network, which is a group of schools linked by a common philosophy but not centralized governance or operations. Networks are also entities that would otherwise fit definition of CMO but have fewer than three schools. Freestanding: Freestanding schools include both start up single site schools and traditional district schools that have converted to charters that are not part of a network or CMO affiliation. Autonomy (CCSA Definition) Autonomous charters: Schools that appoint their board of directors, do not use the local school district s collective bargaining agreement, are directly funded and are likely to be incorporated as a 501(c)3. Non autonomous charters: Schools that either have the majority of their board appointed by their authorizer or are under a school district s collective bargaining agreement, are indirectly funded, and are not incorporated as a 501(c)3. Semi autonomous charters: Schools that appoint their own board and is incorporated as a 501(c)3. In addition to these characteristics, a semi autonomous charter school either uses their authorizing district s collective bargaining agreement and is directly funded or is indirectly funded and does not use the district s collective bargaining agreement. Site Type (California Department of Education CDE Definition): Non classroom based: Schools where less than 80% of instructional time is offered at the school site when students are, engaged in educational activities required of those pupils and are under immediate supervision and control of an employee of the charter school who possesses a valid teaching certificate (EC ). Classroom based: Schools where at least 80% of instructional time is offered at the school site. Start Type (CDE Definition): Conversion: Schools that converted from a traditional public school into a charter school. Start up: Schools that started organically without converting from an existing school.