Choosing Your Validity: Analyzing the Impacts of Charter Schools in a U.S. State Using Two Types of Matching Designs

Size: px
Start display at page:

Download "Choosing Your Validity: Analyzing the Impacts of Charter Schools in a U.S. State Using Two Types of Matching Designs"

Transcription

1 Choosing Your Validity: Analyzing the Impacts of Charter Schools in a U.S. State Using Two Types of Matching Designs Leesa M. Foreman, Kaitlin P. Anderson, Gary W. Ritter, Patrick J. Wolf University of Arkansas Presented at the Association for Education Finance and Policy, 41 st Annual Conference, Denver, CO March 19, 2016 Abstract In this study, we consider situations in which imperfect lotteries are conducted for a subset of grades within schools. While these lotteries unfortunately do not create opportunities for full-scale randomized experiment evaluations, they do provide opportunities for improvements to quasiexperimental designs. The goal of the paper is to examine charters schools using two distinct matching methods, to check the congruence of those results, and to use the very small lottery subsample as a robustness check for both. We assess the effectiveness of the charter schools in a single mid-size U.S. state using two quasiexperimental modeling methods that can be used in combination to address both internal and external validity concerns when lotteries do not turn out to be randomized experiments. The first method matches charter students to traditional public school students from the same district feeder schools/areas; in this method, the population from which the matches are drawn includes all students in the feeder schools. The second matches charter students from oversubscribed areas to waitlist students; in this method, the population from which the matches are drawn includes all only those students who applied for admission to at least one charter school within the region. The first method (the TPS-Matching method), has stronger external validity as it compares students statewide, while the second method (the Waitlist-Matching method), has stronger internal validity as it compares a set of students who have all applied to charter schools, thus addressing the issue of selection bias. Data from state math and literacy exams for 4 th -8 th grade students from are used for the analyses. Results are compared and found to be similar between methods for both math and reading. We find modest positive results for charter school students which are strongest for the first year of analysis. Robustness checks are performed and also reported. In the end, we find similar results using multiple methods; this gives us greater confidence in our quasiexperimental strategies used to assess the effectiveness of charter schools throughout the state. Keywords: school choice; charter schools; research methods; quasi-experimental design 1

2 Choosing Your Validity: Analyzing the Impacts of Charter Schools in a U.S. State Using Two Types of Matching Designs 1. Introduction When a Lottery Isn t a Randomized Control Trial In education research, we attempt to find causal relationships. We are concerned with whether estimates are biased and if they can be generalized. What can we learn about these students, teachers, schools, programs, and interventions in this place and time that might be useful and which we might be able to use with other students, teachers, schools, and programs to improve them? Randomized experiments are considered the gold standard for education research as they address the issues of selection bias many quasi-experimental methods face. However, randomized experiments are often neither feasible nor generalizable. They have strong internal validity but lack external validity. When studying charter schools (or other oversubscribed programs), researchers look for oversubscribed schools that perform lotteries as a means of student selection. Lotteries can be used as randomized experiments because winners are chosen at random from a group of applicants who have all applied. Though some may win and others may lose, the fact that all of them applied indicates there is something similar about these individuals which is different from those that did not apply. Due to the lotteries, we would be able to compare outcomes for students who were randomly admitted to the school to the outcomes for students who were randomly not admitted to the school. With this method, we would have a Randomized Control Trial (RCT), which is the most rigorous research design for evaluating the causal impacts of a program (Rossi et al., 2004). This method is particularly strong because it allows for a comparison of students who all are invested in attending a charter school (by applying to the school). In the context of charter school evaluation research, the differences between students performance should be attributed to the impact of attending a charter school and not on differences in parent motivation. Experimental approaches have been lauded for their strong internal validity, as they have the ability to generate unbiased causal estimates of programmatic effects. Experiments also have been properly criticized for their limited external validity, due to the generalizability of those causal estimates to the entire sample of program participants and, ultimately, to potential participants in other places and circumstances. With experimental charter evaluations, we likely get an unbiased estimate of the impact of higher-quality charter schools on student achievement. Quasiexperimental design (QED) studies, such as those involving student matching and longitudinal analysis employed in our primary analysis here, tend to be weaker than experiments in internal validity but much stronger than experiments in external validity. While we would prefer experimental approaches that use oversubscription to conduct an RCT evaluation, having lotteries does not guarantee that an RCT analysis is possible. In the paragraphs 2

3 that follow, we highlight four issues connected to random assignment designs that can limit their usefulness. 1. Oversubscribed schools are not representative of all charter schools. 2. Oversubscribed schools can have very few seats open for lottery admission after year one. 3. In the initial year of operation where lotteries can best be exploited for study, however, this is not necessarily the best year to study the school. 4. Lottery analyses require better record keeping than is often done in regular schools. First, the requirement for oversubscription indicates a particular desirability for these schools and thus is not necessarily representative of other charters. It can be argued that oversubscribed schools are not representative of the true charter effects because of the fact that they are oversubscribed while others are not. Perhaps there is something different about these schools, which leads them to be oversubscribed higher achieving students, better teachers, more involved parents, different curriculum, etc. Working with oversubscribed schools may not be representative of all charters but it provides the best opportunity for a randomized experiment and offers the ability to address selection bias issues. Second, if we could ignore the external validity challenges of an oversubscribed charter school, we must then deal with the fact that, in most cases, there are often very few lottery seats available in an oversubscribed school in any year but the first. Often there are only a few grades where lotteries occur and there may be only a few seats available as most children in each grade move to the next. In the best cases, a school will add additional classes to a grade which would provide for a larger lottery. Third, for new schools and programs just beginning, the opportunity to implement a randomized control trial via lotteries is optimal. However, when looking to examine charter effects, we find that new schools don t often perform well in the first years of operation and the estimates we obtain may not reflect the true effects of that program over time (CREDO, 2009). If we are looking at a charter school system district-wide or state-wide, relying on new schools with large oversubscription may not give accurate estimates of the charter effect as first year studies aren t representative of schools at maturity. Finally, careful RCT s require excellent data management. That is, charter school personnel need to be diligent in tracking initial applicants, initial admits, waitlist admits, and those who declined offers of enrollment. Many charters do not have the types of data management systems required to carefully organize such information. Additionally, some states do not require charter schools to report lottery and waitlist results, or maintain them over time. Thus, perhaps the best opportunity for careful lottery studies exists in charter middle schools, where there is a pipeline grade each year in which all seats in the initial grade are open for 3

4 lottery admission. For this reason, many of the best known charter school lottery studies are conducted at middle schools, where there is a pipeline grade, and the set of entering students have previously attended traditional public schools and have pre-data available. 1.1 Study Origin In this analysis, we use lottery and waitlist information from the school year. Only schools that reported their waitlists were included in the analysis. It is possible that some schools had a waitlist but did not report it 1. It is also possible that a school used a lottery admission process but, upon enrolling students, had no waitlist because various parents who received admissions declined to enroll and all waitlisted students eventually were admitted to the school. We received waitlisted student data for eight oversubscribed schools. Only waitlisted students with previous public school enrollment were included (out of state, private school, or home school applicants were not reported). In addition, it was not indicated whether any students were awarded automatic admission outside of the lottery (for sibling preference or previous mid-year transfer students). We found very few winners in very few grades with very large waitlists in comparison; thus, estimates of the charter effect were not reliable. (Details and findings of our lottery analysis can be found in the Findings section and Appendix A). So what can be done? We have imperfect lotteries, a broken RCT if you will. However, we have waitlist information that could still be useful. Several of the charters in our state have large waitlists even in grades that are most likely filled with continuing students. Thus, we have experiments in which few people get the actual treatment, but the control population is very large. Can we somehow exploit this? We decided to use the fact that both enrolled charter students and reported waitlist students were all applicants to charter schools. Instead of comparing outcomes for winners and losers of lotteries (because we have so few lottery winners), we compare outcomes of all enrolled charter students and all reported waitlist students in By making use of the waitlist students in this way, we could still address concerns of selection bias and increase internal validity by comparing applicants. We use this information as the robustness check to the larger TPS-matching evaluation. In this study, we address two overarching questions, one practical and one methodological: (1) Are charter schools in this state effective at improving student achievement? (2) Should we believe these results? Does our strategy of using waitlist students as the comparison population yield similar results as a matching study comparing charter students to similar students in TPS schools? 1.2 Study Motivation 1 We believe that there is only one school that was oversubscribed but did not provide waitlist data to the evaluation team. 4

5 Initially, we hoped to evaluate numerous state charter schools using RCT s based on the student admission lotteries. At least, we hoped to use the lotteries that occurred at a few oversubscribed schools as a robustness check to the larger TPS-matching study that was completed adding internal validity to the external validity provided by the statewide matching analysis. Unfortunately, only a few schools were reported as oversubscribed and, within those schools, there were only seats available in a few grades. Thus, we were not confident that results from the lottery-based evaluation of charter schools though a gold standard model would fairly represent the effectiveness of charter schools statewide. As we cannot rely on the results of the lottery analysis, we turn to quasi-experimental models of estimating charter school effects that are used when randomized control trials are not possible. Matching methods attempt to address issues of selection bias by finding the closest student match based on observable characteristics. Because matching studies use large, often longitudinal, administrative data sets that represent the entire population of students in a district or state, both the study power of the evaluation and the external validity are improved. In addition, Bifulco (2012) found that propensity score matching methods used with Ordinary Least Squares regression, that include students from the same district with variables added for pretreatment achievement measures, can provide similar effect estimates as randomized control trials for evaluating the impacts of school choice programs. We use this framework for our main evaluation methodology (TPS-matching analysis). Moreover, to address reasonable concerns about selection bias threatening the validity of matching studies (Rossi et al., 2004; Cook & Campbell, 1979), we conduct an additional analysis focused specifically on the sample of charter applicants (waitlist-matching analysis). We used the same matching method for both matching analyses and the same OLS regression model. This paper is organized in the following way. A review of the relevant studies of charter schools that have used both randomized control trials and matching methods can be found in the Review of Literature section that follows. Details on the matching method and regression model used can be found in the Methods section. Results of our analyses are reported in the Findings. We conclude with a discussion of the limitations, policy implications, and plans for future research. 2. Review of Literature - Charter School Studies Educational choice as a school improvement strategy has been seriously contemplated since the 1960s. Providing choice to families and students who otherwise are often subject to the monopolistic traditional public schools could, in theory, create competition that spurs innovation in traditional public schools. Nobel laureate economist Milton Friedman from these early days was encouraging policy makers to introduce competition and give the customers alternatives in the education sector, saying that the injection of competition would do much to promote a healthy variety of schools (Friedman, 1983; 1962). 5

6 One prominent form of school choice is public charter schooling, developed in Minnesota in the early 1990s. Charter schools are distinctive public schools that are allowed the freedom to be more innovative while being held accountable for advancing student achievement. Because they are public schools, they are open to all children, do not charge tuition, and do not have special entrance requirements (National Alliance for Public Charter Schools). These schools provide parents with an alternative public school option to the traditional public schools in their neighborhoods. Currently, 42 states and the District of Columbia have charter school laws and charter school support in each state varies widely (Center for Education Reform). To frame our findings and to provide context, we consider the relevant studies of charter schools that have used both randomized control trials and matching methods. The literature review included here is by no means exhaustive. For a more complete review of the literature on charter school research, see Betts & Tang (2014). They find that charter schools are performing better in math than traditional public schools in most grades, with middle schools producing the largest gains. They find reading achievement to be positive but insignificant, though this appears to be driven by a few studies with large negative effect sizes. They note that impacts in the charter sector vary considerably, in particular across geographic areas. 2.1 Charter RCT Studies Mathematica Policy Research (MPR) conducted an evaluation of 36 charter middle schools in 15 states (Clark, et al., 2011). MPR limited its evaluation to charter schools that were oversubscribed and used random lotteries to determine which students were and were not admitted. Such lotteries provide the foundation for a Randomized Controlled Trial (RCT) or experiment. The MPR study found that the charter middle schools in the sample produced achievement gains that were, on average, similar to those of the control group. Urban charters tended to have statistically significant positive effects on student achievement while rural charters tended to have statistically significant negative effects. Lower-income students realized more positive achievement gains from charters while higher-income students experienced more negative achievement effects. Fortson et al. (2012) conducted an RCT which examined a sample of middle school students from 15 charter schools in six states over a two-year period. They found in the full sample, students randomly selected to attend charter schools through the lottery had nearly identical average math and reading test scores as students in the control group. From a restricted subsample, they found charter school students had identical average math test scores as students in the control group, but negative reading test scores which were statistically significant. Hoxby et al. (2009) conducted a random assignment evaluation of New York City charter schools that included 93% of NYC charter students in grades 3-12 over an eight-year period. They found that on average, students who attended a charter school for all grades, kindergarten through eight, could close about 86 percent of the black-white achievement gap in math and 66 percent of that 6

7 achievement gap in English. Students who attended fewer years would improve by commensurately smaller amounts. They also found that students who attended a charter high school were more likely to receive a Regents diploma and to score higher on the Regents exams than their control group counterparts. In an RCT study done by Abdulkadiroglu et al. (2009), which included students in Boston s middle and high charter schools in grades 6-12 over a seven-year period, the authors found large positive effects for charter schools students in reading and math at both the middle and high school levels. They found particularly large effects in math at the middle school level. Not surprisingly, these studies of oversubscribed charters, which families intentionally select, the authors find generally positive results. 2.2 Charter Matching Studies Many studies are conducted on charters that are not necessarily oversubscribed. The Center for Research on Educational Outcomes (CREDO) at Stanford University has performed three national evaluations of charter school performance. In all three studies, CREDO used a Virtual Twin Matching (VTM) method, which matches each charter school student with a composite of multiple traditional public school students that, collectively, reflect the charter student s observable characteristics that are assumed to affect achievement. The first of these studies reported in 2009 examined charter school populations in 15 states and the District of Columbia with data available from through That evaluation concluded that 17 percent of charter schools in the sample generated achievement outcomes that were significantly better than the outcomes for the comparison students, 37 percent of charters delivered achievement outcomes that were significantly worse, and for 46 percent of the charters there was no statistical difference between the outcomes of their students and the virtual twins. Charter performance was somewhat more positive for low-income students and at charters that had been open longer (commonly referred to as the maturity advantage). The second study reported in 2013 served as a follow-up to the 2009 CREDO study, evaluating the same states as previously as well as 10 new states and the city of New York, with data that had been released since the 2009 report. The study concluded that charter schooling generated a very small but statistically significant benefit in reading gains that amounted to 7 extra days of achievement growth but no difference in math. Low-income, English Language Learner (ELL), and special needs students appeared to benefit the most from charter schools in terms of achievement gains, according to the study. Evaluators noted that school closure rates had some impact on the findings overall. In 2015, CREDO released a national study focused on 41 urban areas with substantial concentrations of public charter schools. The study concluded that urban charters deliver average 7

8 achievement benefits in both reading and math that are statistically significant and substantially meaningful, amounting to 28 additional days of learning growth in reading and 40 days in math. Summary The results from these national studies of charter school effects on student test scores are remarkably similar. They all suggest that the average effect of charter schooling in general on student achievement is modest. The national studies with longitudinal components indicate that average charter effects tend to be somewhat negative in the earlier years of charter schooling but tend to be somewhat positive now. The studies all agree that urban charters demonstrate much larger and more consistent positive achievement benefits than do rural charters, especially with disadvantaged populations of students. 2.3 Contributions of this Paper In this analysis, we present the results of a charter school evaluation of a medium-sized U.S. state using a student matching methodology similar to the one used in the CREDO studies except that we match students 1:1 instead of 1: many to avoid the potential problem of bias raised in critiques of the CREDO approach. 2 We discuss the extent to which our primary findings from that analysis are consistent with the general patterns of charter school effects from the national studies reviewed above. Importantly, we conduct sensitivity and robustness checks of our primary results using alternative methodologies and analytic samples. A primary goal of the paper to assess the congruence of two distinct matching methods (a general matching study and a matching study using only a sample of charter applicants as comparison kids) to assess the effectiveness of charter schools. Bifulco (2012) indicates that longitudinal student matching studies that include baseline test scores and geography as matching variables can produce effect estimates that are less than 4 percent different from those obtained through experimental analysis. We use Bifulco s approach for our main analysis and build upon it for our secondary analysis by applying the same matching method but now only matching charter students with students on charter waitlists. Matching within the population of students who sought entrance to charters presumably controls for parent motivation to seek charter schooling and thereby eliminates the main threat to the internal validity of QED charter studies. Furthermore, as we could not verify the conduct and validity of the charter school lotteries performed in the state that we study, but we can confirm which students were eligible applicants and which students were left on the waitlist, we classify the experimental analysis of the charter effects as a Broken RCT which could contain selection bias. The lottery analysis is not at all 2 The most salient criticism of the Virtual Twin Matching is that the use of a larger number of students in the comparison group in an estimation of achievement gains over time biases the charter effects estimates towards 0 because the estimations are more precise on the comparison group side of the analysis than on the charter group side (Hoxby, 2009). 8

9 useful for telling us overall program impacts, because it is based on a small subsample of students in lottery grades in a subsample of non-representative schools. However, we can use the lottery analysis in these grades as robustness checks for our two matching strategies. Therefore, a student matching approach within the pool of charter applicants can address the issue of selection bias and increase internal validity. We compare our primary QED results with the results of the waitlistmatching analysis and lottery evaluation with additional robustness checks. We now turn to a description of the data used for the analyses. 3. Data The research team was provided de-identified student level data for a mid-sized U.S state, for years to Each ID was paired with information for each school year including the school attended, race/ethnicity, gender, free and reduced lunch status, English language learner status, special education status, and test scores for math and literacy. The student test scores came from the State Criterion Referenced Exams in both Math and Literacy. In this state, assessments are taken by students in 3 rd through 8 th grade. We use the prior year test scores as the baseline measure and compare outcomes of student achievement in math and literacy for and Scores are standardized within grade with mean equal to zero and standard deviation equal to one. There are two types of charter schools in this state: start-up and conversion charter schools. Conversion charter schools are schools that were formerly traditional public schools which later converted to charter. These conversion schools, while technically called charter schools in this state, are not always schools of choice as they are often the only school available to a community. Start-up charter schools are created from scratch to be their own school district and are open to all students who apply, regardless of residence. These start-up charters are the types of schools that most researchers and observers are referring to when they study and debate the pros and cons of charter schools. As a result, our analyses here focus solely on the start-up charter schools. Throughout the rest of this paper, when use the term charter schools, we refer to start-up charter schools, since these are the schools that are known as charter schools. Charter students represent about 1.6% to 1.8% of all K-12 students statewide depending on the year (Tables 1 and 2 below). Charter students share of total enrollment has increased over the two years covered by this study. Seventeen start-up charter schools, feeder schools, and approximately 5,500-7,000 students, respectively, were included in each year of analysis of the TPS-matching evaluation. Ten charter schools and approximately 2,500-3,200 students, respectively, were included in each year of analysis of the waitlist-matching evaluation. 9

10 Table 1. Student Demographics: Charters Students Compared to State, State (All Start-up Charter Students) Students Charter Students in TPS-Matching Analysis Charter Students in Waitlist Matching Analysis Enrollment 471,867 7,402 3,624 3,313 Charter as % 1.6% 0.7% 0.7% Total FRL % 61% 52% 49% 82% Minority % 36% 60% 57% 85% N of Schools Table 2. Student Demographics: Charters Students Compared to State, State (All Students) Start-up Charter Students Charter Students in TPS-Matching Charter Students in Waitlist Matching Enrollment 474,995 8,346 4,748 3,439 Charter as % 1.8% 1% 0.7% Total FRL % 61% 51% 70% 76% Minority % 37% 57% 55% 99% N of Schools The subpopulation of charter students differs in some observable ways from the state as a whole, in that it includes a smaller proportion of low-income students but a larger proportion of minority students. For the TPS-matching analysis, for which all start-up charters in the state were included, the numbers are much closer when comparing charter schools with their local traditional public school districts which serve as their feeder districts those districts where the students would have otherwise been assigned had they not attended the public charter school. Charter schools included in the waitlist-matching analysis have greater proportions of low-income and minority students than the subpopulations of charters and students statewide. (Baseline equivalency comparisons are displayed in the following section.) Appendix A shows some of the basic details for charter schools including the year the school opened, the grade levels served during the school years covered in this paper, the enrollment of each charter school, the percentage of minority students by race/ethnicity, the percentage of students who qualify for free or reduced lunch, and the number of feeder districts from which traditional public school students were drawn for comparison in the TPS-matching analysis. We now turn to the details of the different methods and model used for the analyses. 4. Methods 4.1 Quasi-Experimental Design 10

11 As we review the methods used, we should consider the main issues we are addressing: How well do charter students do relative to their peers and do the results of multiple QED methods converge? Both of our primary analyses here rely on developing a set of matching students who are not in charter schools for each individual student attending a charter. What does it mean to create a student match? The goal of this method is to create a set of students that are in traditional public schools but are essentially the same as the group of public charter school students when comparing observable characteristics such as income and race/ethnicity. Any differences will not be based on observable student characteristics (such as race, income, gender, or prior test scores) as matched twins will be intentionally selected to be nearly identical on these characteristics. Reasons for a specific student not being included in the analysis include, but are not limited to: not having test scores from both the baseline test year and the outcome year, being in an untested grade in either the baseline or outcome year, not being enrolled in a public school during either year, or if a student missed the test day, among other reasons. Thus, the results should be interpreted as the impacts for the matched student population, which may not generalize to the broader student population. 3 It should be noted that we report single year estimates and students were re-matched the second year to account for movement between the groups. For the TPS-matching analysis, the sample available for matching includes all charter students and traditional public school students in the geographic feeder districts. Not all students are included in the analysis however. For the TPS- comparison groups in particular, many students were not used for a one-to-one match. Charter students are matched to students from the same feeder district they would have attended had they not been admitted to a charter school. Seventeen of 20 charters in the state were included in this analysis. Of course, a clear weakness in this method remains. Despite the similarity in observable characteristics between the charter students and the comparison students, these students differ on one fundamental unobservable attribute the motivation (due to positive or negative traits or past experiences) to seek out a seat in a charter school. However, having access to the waitlist data from all oversubscribed charter schools in our sample allows us to attempt to address this shortcoming in our second analysis. For the waitlist-matching analysis, the sample consists of all charter students in the geographic area of the oversubscribed schools and all waitlist students. Most oversubscribed schools fall within a central urban metropolitan area, and one is located in a rural area. All charter students in these areas are included in the analysis as treatment schools, whether or not they allotted any open seats in the previous year using a lottery. Of the 17 total charters in the state, the 10 charters in regions with waitlists were included in this analysis. 3 In the TPS-matching analysis, 66-74% were actually matched and for the waitlist-matching analysis, 38-41% were actually matched. 11

12 The key difference (in addition to the fact that there are seven more schools in the broad matching analysis) is that, in the waitlist analysis, the population from which the matched twins were selected is drawn entirely form the charter school waitlists that have been provided to us. In the TPS-matching analysis, matched twins were drawn from the full population of students. When looking to compare the results of these two analyses we find that we are not comparing apples-to-apples as there are 17 schools included in the TPS-matching analysis and 10 schools in the waitlist-matching analysis. For greater comparability of results, and to check whether we can replicate results with these two different matching methods, we restrict the sample of the TPSmatching analysis to include the same 10 schools. As we may be concerned that by restricting the full TPS-matching sample we may be excluding very different types of schools, we examine the baseline characteristics below. We now turn to the means by which we created matched comparison groups of students. 4.2 Matching Method In order to complete the matching process for charter students, students who have received the treatment of being in the charter school are matched on observable characteristics from the previous school year, so that the academic growth they experience in each year can be properly studied. For those students who are not promoted from one grade to the next, accommodations are made to match properly, as described in step 1 below. Treatment students are matched with students in traditional public schools using the following matching procedure (fully outlined in Appendix B). This identical process is used for both the TPS-matching and waitlist-matching analyses and is outlined as follows: Matching Process (Conducted Separately for Math and Literacy) 1. Students are first matched with a student in the same grade in both the outcome year and baseline or matching year (always the year before). 2. For the math and literacy analyses, separately, all students are matched based on previous year scores on the same subject test, rounded to the nearest 0.01 z-score unit. The other subject test score is used as part of the propensity score in step 4 (below), as having a matched test score in the same subject is more relevant for controlling for prior performance. Therefore, the math analysis matches first on math examination scores and later factors in literary scores, while the literacy analysis matches first on literacy examination scores and later factors in math scores. 3. A propensity score is then created using free and reduced lunch (FRL) status, race/ethnicity (African-American, Asian-American or Pacific Islander, Hispanic- American, Native American, White, or Two or more races ), gender, and the other test score (literacy for the math analysis and math for the literacy analysis). 4. Finally, all matches are based on guaranteeing exact matches from step 1 and 2, and the closest available propensity score match from step 3. 12

13 In order to test whether or not this process worked for the purposes of generating an appropriate comparison group, we conducted baseline equivalency analyses to test how similar the two groups were to each other. The average measure of each of the observable variables is reported for both the charter treatment group and for the matched student comparison group. Any difference between the two is reported, and the statistical p-value is reported to show if that difference is statistically significant. P-values below 0.05 indicate statistically significant differences that might raise concerns about the comparability of our samples. For our major comparisons, shown in Tables 3 through 10, in some instances we had to use broader matches 4 in order to capture a large enough sample size for the analysis. For this reason, in all cases, and especially in cases where there are significant differences at baseline, our regressions will account for any of the small differences observed in these matching variables. 4.3 Student Samples in Each Method Tables 3 and 4 show the math and literacy baselines respectively, for schools included in the full TPS-matching analysis, for each year. For the combined set of matches for all charter schools included in the TPS-matching analysis, it appears that there were some significant differences in the percent of FRL students in , although these differences were slight in size. In the tables that follow, Charter refers to the treatment group and TPS refers to the comparison group. Table 3. Baseline Equivalency for TPS-Matching Analysis in Math, N of Schools = Charter TPS Difference Charter TPS Difference Number of Observations Grades Served Prior Year Math Z-Score Prior Year Literacy Z-Score (0.03) % FRL (0.04) *** (0.04) % Minority % Female Note: *** p<0.01, ** p<0.05, * p<0.1 Table 4. Baseline Equivalency for TPS-Matching Analysis in Literacy, N of Schools = Charter TPS Difference Charter TPS Difference Number of Observations Grades Served Prior Year Math Z-Score (0.02) Prior Year Literacy Z-Score (0.01) % FRL (0.08) *** (0.01) % Minority % Female Note: *** p<0.01, ** p<0.05, * p<0.1 4 Broader matches were accomplished by relaxing the degree of similarity of the baseline test score for the two students. 13

14 Turning our attention to the ten schools included in the restricted TPS-matched sample, in Tables 5 and 6, again we see some slight but significant differences in the percent of FRL students in However, we find that the restricted TPS-matched sample has more minority students than the full TPS-matching sample for both years. Table 5. Baseline Equivalency for Restricted TPS-Matching Analysis in Math, N of Schools = Charter TPS Difference Charter TPS Difference Number of Observations Grades Served Prior Year Math Z-Score Prior Year Literacy Z-Score (0.05) * % FRL % Minority % Female Note: *** p<0.01, ** p<0.05, * p<0.1 Table 6. Baseline Equivalency for Restricted TPS-Matching Analysis in Literacy, N of Schools = Charter TPS Difference Charter TPS Difference Number of Observations Grades Served Prior Year Math Z-Score (0.01) Prior Year Literacy Z-Score % FRL (0.04) *** (0.01) % Minority * % Female Note: *** p<0.01, ** p<0.05, * p<0.1 We consider that we may be excluding very different set of schools, so we next examine the baseline equivalencies of the seven omitted schools in Tables 7 and 8. Here we find significant and somewhat large differences in the percent of low-income students and modest but significant differences in minority students for Relative to both the full TPS-matched sample and the restricted TPS-matched sample, there are far fewer minority and low-income students in both years. Table 7. Baseline Equivalency for Omitted TPS-Matching Analysis in Math, N of Schools = Charter TPS Difference Charter TPS Difference Number of Observations Grades Served Prior Year Math Z-Score Prior Year Literacy Z-Score % FRL (0.22) *** (0.01) % Minority % Female (0.03) Note: *** p<0.01, ** p<0.05, * p<0.1 14

15 Table 8. Baseline Equivalency for Omitted TPS-Matching Analysis in Literacy, N of Schools = Charter TPS Difference Charter TPS Difference Number of Observations Grades Served Prior Year Math Z-Score (0.08) Prior Year Literacy Z-Score (0.00) % FRL (0.24) *** % Minority (0.06) ** (0.02) % Female Note: *** p<0.01, ** p<0.05, * p<0.1 Tables 9 and 10 show the math and literacy baselines for schools included in the waitlist-matching analysis. It appears there were some significant differences in the percent of female students in , otherwise the two groups are equally balanced. When we examine the similarities between the restricted TPS-matched sample and the waitlist-matched sample, we find that they are comparable. Table 9. Baseline Equivalency for Waitlist-Matching Analysis in Math, N of Schools = Charter Waitlist Difference Charter Waitlist Difference Number of Observations Grades Served Prior Year Math Z-Score (0.00) Prior Year Literacy Z-Score (0.02) (0.08) *** % FRL % Minority % Female (0.06) *** (0.02) Note: *** p<0.01, ** p<0.05, * p<0.1 Table 10. Baseline Equivalency for Waitlist-Matching Analysis in Literacy, N of Schools = Charter Waitlist Difference Charter Waitlist Difference Number of Observations Grades Served Prior Year Math Z-Score Prior Year Literacy Z-Score (0.01) % FRL % Minority % Female Note: *** p<0.01, ** p<0.05, * p<0.1 These summary statistics show that we were able to match our samples on most characteristics, though our charter students were slightly more likely to be economically disadvantaged, despite the fact that prior test scores are identical. This is due to the fact that our primary matching indicator was prior year academic ability. By comparing estimates based on the same schools, we can assess whether multiple methods generate similar conclusions. If they do, we will have greater confidence in our findings. 15

16 4.4 Regression Model Once the baseline equivalency is examined, the resulting matches were run through statistical testing to see how much of the academic growth for students can be attributed to attending charter schools. The method used is a straightforward Ordinary Least Squares regression analysis, with robust standard errors. The same regression model is used for both matching analyses. OLS Regression Model. y i = β 0 + β 1 charter + β 2 θ i + β 3 X i + β 4 δ + ε (1) Where, y i represents a given outcome of interest (math or literacy achievement) for student i, charter is an indicator for charter school treatment, θ i represents controls for prior year test scores (both math and literacy achievement), X i represents controls for student level characteristics (gender, race, FRL status), δ represents control for switched schools, and ε represents the error term. Our regression analyses statistically control for any minor differences in demographic characteristics. 4.5 Robustness Checks As a robustness check, we use the same sample of schools used for the waitlist-matching analysis to make an additional comparison. We use the full sample prior to matching, and run an OLS regression where we control for prior achievement, student characteristics, and schools for which students applied. As this model includes ALL students who applied to these charter schools (those attending and those on the waitlist), we refer to this as the Applicant model. Results of the TPS-matching model(s), waitlist-matching model, and applicant model are reported together in the following section. As a further check for robustness, we used the lottery analysis to compare to all models for Results of this robustness check can be found in Appendix A. 5. Findings The focus of our research was to answer the following questions: (1) Are charters effective in this state? (2) Should we believe these results? Does our strategy of using waitlist students as the comparison population yield similar results as a matching study comparing charter students to similar students in TPS schools? The academic impacts represented in Table 11 indicate that, for all 17 start-up charter schools in the state, students in public charter schools demonstrated positive and statistically significant impacts of 0.09 standard deviations on math scores and 0.07 standard deviations on literacy 16

17 scores, in In , we find no clear effect of attending a charter school on math scores, while there was a somewhat negative and statistically significant impact of standard deviations on literacy scores. To assess the extent to which we should believe these results, we consider results for the limited sample of students attending charter schools in the same region as charters with waitlists. Thus, we restrict the sample to the same schools included in the waitlist-matching and applicant analyses for comparison. If the results of the restricted sample provide similar estimates to those of the waitlist-matching, this convergence of the results of the overall TPS-matching analysis enhances our confidence that our matching strategy is not significantly threatened by self-selection. In the restricted TPS-matching analysis, we see the same results for as those found in the full TPS-matching sample, statistically significant impacts of 0.09 standard deviations in math and 0.07 standard deviations in literacy. For , we find statistically significant impacts on both math and literacy scores of 0.03 standard deviations. Comparing these results to the findings from the waitlist-matching analysis, we find similar results with statistically significant impacts for both math and literacy of 0.05 standard deviations, in For , we find no significant effects for math or literacy. This gives us greater confidence in the results of the full TPS-matching analysis. Looking to the applicant model analysis, we see that this analysis appears to support the findings of both matching analyses. For , we see positive and statistically significant impacts in math of 0.11 standard deviations, and 0.06 standard deviations in literacy. For , we find no clear effects in math, but positive and significant effects in literacy of 0.04 standard deviations. It appears that it may be possible to use applicant information to evaluate charter school effects. We find that both the waitlist-matching and the applicant analyses provided similar estimates to the TPS-matching results in most instances. With a combination of analyses, it may be possible for a combination of quasi-experimental methods to provide robust estimates in the absence of randomized experiments using data from oversubscribed schools. Furthermore, results from these analyses are consistent with the general patterns of modest charter school effects from the national studies reviewed in the literature. 17

18 Table 11. Charter Effect by Assessment, Model Comparisons, N of Schools= TPS- Matching TPS- Matching (Restricted) Waitlist- Matching TPS- Matching TPS- Matching (Restricted) In the following section, we conclude with a discussion of the limitations and policy implications. Waitlist- Matching Model Applicant Applicant MATH Charter Effect 0.09 *** 0.09 *** 0.05 ** 0.11 *** * Treatment n= N= LITERACY Charter Effect 0.07 *** 0.07 *** 0.05 ** 0.06 *** ** 0.03 ** ** Treatment n= N= Control for prior reading/math tests x x x x x x Control for student characteristics x x x x x x Control for lottery (grade/school) Control for school applied x x Control for school switchers x x x x *p<0.10, **p<0.05, ***p<

19 6. Discussion This evaluation sought to offer an overview of the academic impacts of charter schools in a medium-sized state for the to school years. Due to the limitations of data collection, we were not able to make firm conclusions about oversubscribed start-up charter schools through a Randomized Control Trial (RCT) analysis. Using a careful student matching method, charter students in each school were matched with similar students in their feeder districts in each of these years in the TPS-matching analysis. The waitlist-matching analysis was intended to complement the TPS-matching analysis. The same student matching method, in which charter students in each school were matched with similar traditional public school students who applied for charter schools but were not admitted (waitlisted) in the school year, was used to create approximately equivalent comparison groups. The applicant analysis was added as a robustness check for the waitlist-matching estimates. Separate matches and analyses were conducted for Math and Literacy (outcomes in grades 4-8). Given the data available, the quasi-experimental model is the best form of analysis available to us. Further, this study has the advantage of combining two matching methodologies thereby improving both the external and internal validity of the findings. Reasonable conclusions that can be drawn from this study are that charters in vibrant charter markets with waitlists in this state have a modest positive effect on student test scores in math and literacy, however, this finding is not consistent over both years of analysis. The school year appeared to be the strongest individual year for charter school performance, compared with With the evaluation that has been performed, there were certain limitations that can be improved upon in future studies. The key weakness of this study is the requirement that it be quasiexperimental, as not enough charter schools provided clear, reliable information about admissions lotteries for us to conduct a gold standard experimental evaluation. A second limitation of this study was our small sample of oversubscribed schools and our relatively low student match rates. Most oversubscribed charters are found within the central urban metropolitan area. Several charter schools, by design or for other reasons, maintain low student populations and therefore have low numbers of students tested. Researchers should continue to analyze the academic impacts of public charter schools. One of the most celebrated aspects of charter schools anywhere is that they are held accountable for their outcomes. This current evaluation seeks to add to that accountability and provide a means of checking the robustness of results found in the TPS-matching analysis. In doing so, we have increased confidence in our traditional matching results. While academic impacts do not encompass the entire mission of a charter school, or any school, these results help provide information on the performance of public charter schools. 19

An Empirical Analysis of the Effects of Mexican American Studies Participation on Student Achievement within Tucson Unified School District

An Empirical Analysis of the Effects of Mexican American Studies Participation on Student Achievement within Tucson Unified School District An Empirical Analysis of the Effects of Mexican American Studies Participation on Student Achievement within Tucson Unified School District Report Submitted June 20, 2012, to Willis D. Hawley, Ph.D., Special

More information

The Effects of Statewide Private School Choice on College Enrollment and Graduation

The Effects of Statewide Private School Choice on College Enrollment and Graduation E D U C A T I O N P O L I C Y P R O G R A M R E S E A RCH REPORT The Effects of Statewide Private School Choice on College Enrollment and Graduation Evidence from the Florida Tax Credit Scholarship Program

More information

Peer Influence on Academic Achievement: Mean, Variance, and Network Effects under School Choice

Peer Influence on Academic Achievement: Mean, Variance, and Network Effects under School Choice Megan Andrew Cheng Wang Peer Influence on Academic Achievement: Mean, Variance, and Network Effects under School Choice Background Many states and municipalities now allow parents to choose their children

More information

School Competition and Efficiency with Publicly Funded Catholic Schools David Card, Martin D. Dooley, and A. Abigail Payne

School Competition and Efficiency with Publicly Funded Catholic Schools David Card, Martin D. Dooley, and A. Abigail Payne School Competition and Efficiency with Publicly Funded Catholic Schools David Card, Martin D. Dooley, and A. Abigail Payne Web Appendix See paper for references to Appendix Appendix 1: Multiple Schools

More information

Educational Attainment

Educational Attainment A Demographic and Socio-Economic Profile of Allen County, Indiana based on the 2010 Census and the American Community Survey Educational Attainment A Review of Census Data Related to the Educational Attainment

More information

Shelters Elementary School

Shelters Elementary School Shelters Elementary School August 2, 24 Dear Parents and Community Members: We are pleased to present you with the (AER) which provides key information on the 23-24 educational progress for the Shelters

More information

Transportation Equity Analysis

Transportation Equity Analysis 2015-16 Transportation Equity Analysis Each year the Seattle Public Schools updates the Transportation Service Standards and bus walk zone boundaries for use in the upcoming school year. For the 2014-15

More information

2013 TRIAL URBAN DISTRICT ASSESSMENT (TUDA) RESULTS

2013 TRIAL URBAN DISTRICT ASSESSMENT (TUDA) RESULTS 3 TRIAL URBAN DISTRICT ASSESSMENT (TUDA) RESULTS Achievement and Accountability Office December 3 NAEP: The Gold Standard The National Assessment of Educational Progress (NAEP) is administered in reading

More information

Longitudinal Analysis of the Effectiveness of DCPS Teachers

Longitudinal Analysis of the Effectiveness of DCPS Teachers F I N A L R E P O R T Longitudinal Analysis of the Effectiveness of DCPS Teachers July 8, 2014 Elias Walsh Dallas Dotter Submitted to: DC Education Consortium for Research and Evaluation School of Education

More information

The Talent Development High School Model Context, Components, and Initial Impacts on Ninth-Grade Students Engagement and Performance

The Talent Development High School Model Context, Components, and Initial Impacts on Ninth-Grade Students Engagement and Performance The Talent Development High School Model Context, Components, and Initial Impacts on Ninth-Grade Students Engagement and Performance James J. Kemple, Corinne M. Herlihy Executive Summary June 2004 In many

More information

A Comparison of Charter Schools and Traditional Public Schools in Idaho

A Comparison of Charter Schools and Traditional Public Schools in Idaho A Comparison of Charter Schools and Traditional Public Schools in Idaho Dale Ballou Bettie Teasley Tim Zeidner Vanderbilt University August, 2006 Abstract We investigate the effectiveness of Idaho charter

More information

NCEO Technical Report 27

NCEO Technical Report 27 Home About Publications Special Topics Presentations State Policies Accommodations Bibliography Teleconferences Tools Related Sites Interpreting Trends in the Performance of Special Education Students

More information

Kansas Adequate Yearly Progress (AYP) Revised Guidance

Kansas Adequate Yearly Progress (AYP) Revised Guidance Kansas State Department of Education Kansas Adequate Yearly Progress (AYP) Revised Guidance Based on Elementary & Secondary Education Act, No Child Left Behind (P.L. 107-110) Revised May 2010 Revised May

More information

Serving Country and Community: A Study of Service in AmeriCorps. A Profile of AmeriCorps Members at Baseline. June 2001

Serving Country and Community: A Study of Service in AmeriCorps. A Profile of AmeriCorps Members at Baseline. June 2001 Serving Country and Community: A Study of Service in AmeriCorps Cambridge, MA Lexington, MA Hadley, MA Bethesda, MD Washington, DC Chicago, IL Cairo, Egypt Johannesburg, South Africa A Profile of AmeriCorps

More information

ABILITY SORTING AND THE IMPORTANCE OF COLLEGE QUALITY TO STUDENT ACHIEVEMENT: EVIDENCE FROM COMMUNITY COLLEGES

ABILITY SORTING AND THE IMPORTANCE OF COLLEGE QUALITY TO STUDENT ACHIEVEMENT: EVIDENCE FROM COMMUNITY COLLEGES ABILITY SORTING AND THE IMPORTANCE OF COLLEGE QUALITY TO STUDENT ACHIEVEMENT: EVIDENCE FROM COMMUNITY COLLEGES Kevin Stange Ford School of Public Policy University of Michigan Ann Arbor, MI 48109-3091

More information

Like much of the country, Detroit suffered significant job losses during the Great Recession.

Like much of the country, Detroit suffered significant job losses during the Great Recession. 36 37 POPULATION TRENDS Economy ECONOMY Like much of the country, suffered significant job losses during the Great Recession. Since bottoming out in the first quarter of 2010, however, the city has seen

More information

School Year Enrollment Policies

School Year Enrollment Policies 1 2018 19 School Year Enrollment Policies BASIS Schools, Inc. operates open-enrollment public charter schools which do not charge tuition and do not administer entrance examinations. BASIS Schools, Inc.

More information

Psychometric Research Brief Office of Shared Accountability

Psychometric Research Brief Office of Shared Accountability August 2012 Psychometric Research Brief Office of Shared Accountability Linking Measures of Academic Progress in Mathematics and Maryland School Assessment in Mathematics Huafang Zhao, Ph.D. This brief

More information

Status of Women of Color in Science, Engineering, and Medicine

Status of Women of Color in Science, Engineering, and Medicine Status of Women of Color in Science, Engineering, and Medicine The figures and tables below are based upon the latest publicly available data from AAMC, NSF, Department of Education and the US Census Bureau.

More information

A Guide to Adequate Yearly Progress Analyses in Nevada 2007 Nevada Department of Education

A Guide to Adequate Yearly Progress Analyses in Nevada 2007 Nevada Department of Education A Guide to Adequate Yearly Progress Analyses in Nevada 2007 Nevada Department of Education Note: Additional information regarding AYP Results from 2003 through 2007 including a listing of each individual

More information

Student Mobility Rates in Massachusetts Public Schools

Student Mobility Rates in Massachusetts Public Schools Student Mobility Rates in Massachusetts Public Schools Introduction The Massachusetts Department of Elementary and Secondary Education (ESE) calculates and reports mobility rates as part of its overall

More information

READY OR NOT? CALIFORNIA'S EARLY ASSESSMENT PROGRAM AND THE TRANSITION TO COLLEGE

READY OR NOT? CALIFORNIA'S EARLY ASSESSMENT PROGRAM AND THE TRANSITION TO COLLEGE READY OR NOT? CALIFORNIA'S EARLY ASSESSMENT PROGRAM AND THE TRANSITION TO COLLEGE Michal Kurlaender University of California, Davis Policy Analysis for California Education March 16, 2012 This research

More information

University-Based Induction in Low-Performing Schools: Outcomes for North Carolina New Teacher Support Program Participants in

University-Based Induction in Low-Performing Schools: Outcomes for North Carolina New Teacher Support Program Participants in University-Based Induction in Low-Performing Schools: Outcomes for North Carolina New Teacher Support Program Participants in 2014-15 In this policy brief we assess levels of program participation and

More information

Sector Differences in Student Learning: Differences in Achievement Gains Across School Years and During the Summer

Sector Differences in Student Learning: Differences in Achievement Gains Across School Years and During the Summer Catholic Education: A Journal of Inquiry and Practice Volume 7 Issue 2 Article 6 July 213 Sector Differences in Student Learning: Differences in Achievement Gains Across School Years and During the Summer

More information

Miami-Dade County Public Schools

Miami-Dade County Public Schools ENGLISH LANGUAGE LEARNERS AND THEIR ACADEMIC PROGRESS: 2010-2011 Author: Aleksandr Shneyderman, Ed.D. January 2012 Research Services Office of Assessment, Research, and Data Analysis 1450 NE Second Avenue,

More information

Public School Choice DRAFT

Public School Choice DRAFT Public School Choice DRAFT Why Public School Choice? The educational ecosystem continues to see different types of schools and instructional choices being offered by private schools, charter organizations,

More information

Evaluation of a College Freshman Diversity Research Program

Evaluation of a College Freshman Diversity Research Program Evaluation of a College Freshman Diversity Research Program Sarah Garner University of Washington, Seattle, Washington 98195 Michael J. Tremmel University of Washington, Seattle, Washington 98195 Sarah

More information

BENCHMARK TREND COMPARISON REPORT:

BENCHMARK TREND COMPARISON REPORT: National Survey of Student Engagement (NSSE) BENCHMARK TREND COMPARISON REPORT: CARNEGIE PEER INSTITUTIONS, 2003-2011 PREPARED BY: ANGEL A. SANCHEZ, DIRECTOR KELLI PAYNE, ADMINISTRATIVE ANALYST/ SPECIALIST

More information

Coming in. Coming in. Coming in

Coming in. Coming in. Coming in 212-213 Report Card for Glenville High School SCHOOL DISTRICT District results under review by the Ohio Department of Education based upon 211 findings by the Auditor of State. Achievement This grade combines

More information

Karla Brooks Baehr, Ed.D. Senior Advisor and Consultant The District Management Council

Karla Brooks Baehr, Ed.D. Senior Advisor and Consultant The District Management Council Karla Brooks Baehr, Ed.D. Senior Advisor and Consultant The District Management Council This paper aims to inform the debate about how best to incorporate student learning into teacher evaluation systems

More information

Wisconsin 4 th Grade Reading Results on the 2015 National Assessment of Educational Progress (NAEP)

Wisconsin 4 th Grade Reading Results on the 2015 National Assessment of Educational Progress (NAEP) Wisconsin 4 th Grade Reading Results on the 2015 National Assessment of Educational Progress (NAEP) Main takeaways from the 2015 NAEP 4 th grade reading exam: Wisconsin scores have been statistically flat

More information

RAISING ACHIEVEMENT BY RAISING STANDARDS. Presenter: Erin Jones Assistant Superintendent for Student Achievement, OSPI

RAISING ACHIEVEMENT BY RAISING STANDARDS. Presenter: Erin Jones Assistant Superintendent for Student Achievement, OSPI RAISING ACHIEVEMENT BY RAISING STANDARDS Presenter: Erin Jones Assistant Superintendent for Student Achievement, OSPI Agenda Introductions Definitions History of the work Strategies Next steps Debrief

More information

The Impacts of Regular Upward Bound on Postsecondary Outcomes 7-9 Years After Scheduled High School Graduation

The Impacts of Regular Upward Bound on Postsecondary Outcomes 7-9 Years After Scheduled High School Graduation Contract No.: EA97030001 MPR Reference No.: 6130-800 The Impacts of Regular Upward Bound on Postsecondary Outcomes 7-9 Years After Scheduled High School Graduation Final Report January 2009 Neil S. Seftor

More information

Updated: December Educational Attainment

Updated: December Educational Attainment Updated: Educational Attainment Among 25- to 29-year olds, the proportions who have attained a high school education, some college, or a bachelor s degree are all rising, according to longterm trends.

More information

A Diverse Student Body

A Diverse Student Body A Diverse Student Body No two diversity plans are alike, even when expressing the importance of having students from diverse backgrounds. A top-tier school that attracts outstanding students uses this

More information

Iowa School District Profiles. Le Mars

Iowa School District Profiles. Le Mars Iowa School District Profiles Overview This profile describes enrollment trends, student performance, income levels, population, and other characteristics of the public school district. The report utilizes

More information

Access Center Assessment Report

Access Center Assessment Report Access Center Assessment Report The purpose of this report is to provide a description of the demographics as well as higher education access and success of Access Center students at CSU. College access

More information

IS FINANCIAL LITERACY IMPROVED BY PARTICIPATING IN A STOCK MARKET GAME?

IS FINANCIAL LITERACY IMPROVED BY PARTICIPATING IN A STOCK MARKET GAME? 21 JOURNAL FOR ECONOMIC EDUCATORS, 10(1), SUMMER 2010 IS FINANCIAL LITERACY IMPROVED BY PARTICIPATING IN A STOCK MARKET GAME? Cynthia Harter and John F.R. Harter 1 Abstract This study investigates the

More information

5 Programmatic. The second component area of the equity audit is programmatic. Equity

5 Programmatic. The second component area of the equity audit is programmatic. Equity 5 Programmatic Equity It is one thing to take as a given that approximately 70 percent of an entering high school freshman class will not attend college, but to assign a particular child to a curriculum

More information

Cooper Upper Elementary School

Cooper Upper Elementary School LIVONIA PUBLIC SCHOOLS http://cooper.livoniapublicschools.org 215-216 Annual Education Report BOARD OF EDUCATION 215-16 Colleen Burton, President Dianne Laura, Vice President Tammy Bonifield, Secretary

More information

Segmentation Study of Tulsa Area Higher Education Needs Ages 36+ March Prepared for: Conducted by:

Segmentation Study of Tulsa Area Higher Education Needs Ages 36+ March Prepared for: Conducted by: Segmentation Study of Tulsa Area Higher Education Needs Ages 36+ March 2004 * * * Prepared for: Tulsa Community College Tulsa, OK * * * Conducted by: Render, vanderslice & Associates Tulsa, Oklahoma Project

More information

George Mason University Graduate School of Education Program: Special Education

George Mason University Graduate School of Education Program: Special Education George Mason University Graduate School of Education Program: Special Education 1 EDSE 590: Research Methods in Special Education Instructor: Margo A. Mastropieri, Ph.D. Assistant: Judy Ericksen Section

More information

Trends & Issues Report

Trends & Issues Report Trends & Issues Report prepared by David Piercy & Marilyn Clotz Key Enrollment & Demographic Trends Options Identified by the Eight Focus Groups General Themes 4J Eugene School District 4J Eugene, Oregon

More information

Effectiveness of McGraw-Hill s Treasures Reading Program in Grades 3 5. October 21, Research Conducted by Empirical Education Inc.

Effectiveness of McGraw-Hill s Treasures Reading Program in Grades 3 5. October 21, Research Conducted by Empirical Education Inc. Effectiveness of McGraw-Hill s Treasures Reading Program in Grades 3 5 October 21, 2010 Research Conducted by Empirical Education Inc. Executive Summary Background. Cognitive demands on student knowledge

More information

Australia s tertiary education sector

Australia s tertiary education sector Australia s tertiary education sector TOM KARMEL NHI NGUYEN NATIONAL CENTRE FOR VOCATIONAL EDUCATION RESEARCH Paper presented to the Centre for the Economics of Education and Training 7 th National Conference

More information

Financing Education In Minnesota

Financing Education In Minnesota Financing Education In Minnesota 2016-2017 Created with Tagul.com A Publication of the Minnesota House of Representatives Fiscal Analysis Department August 2016 Financing Education in Minnesota 2016-17

More information

Statistical Peers for Benchmarking 2010 Supplement Grade 11 Including Charter Schools NMSBA Performance 2010

Statistical Peers for Benchmarking 2010 Supplement Grade 11 Including Charter Schools NMSBA Performance 2010 Statistical Peers for Benchmarking 2010 Supplement Grade 11 Including Charter Schools NMSBA Performance 2010 September 2010 River Dunavin 1 ALBUQUERQUE PUBLIC SCHOOLS BOARD OF EDUCATION PAULA MAES Vice

More information

Review of Student Assessment Data

Review of Student Assessment Data Reading First in Massachusetts Review of Student Assessment Data Presented Online April 13, 2009 Jennifer R. Gordon, M.P.P. Research Manager Questions Addressed Today Have student assessment results in

More information

A Pilot Study on Pearson s Interactive Science 2011 Program

A Pilot Study on Pearson s Interactive Science 2011 Program Final Report A Pilot Study on Pearson s Interactive Science 2011 Program Prepared by: Danielle DuBose, Research Associate Miriam Resendez, Senior Researcher Dr. Mariam Azin, President Submitted on August

More information

learning collegiate assessment]

learning collegiate assessment] [ collegiate learning assessment] INSTITUTIONAL REPORT 2005 2006 Kalamazoo College council for aid to education 215 lexington avenue floor 21 new york new york 10016-6023 p 212.217.0700 f 212.661.9766

More information

The Condition of College & Career Readiness 2016

The Condition of College & Career Readiness 2016 The Condition of College and Career Readiness This report looks at the progress of the 16 ACT -tested graduating class relative to college and career readiness. This year s report shows that 64% of students

More information

Data Glossary. Summa Cum Laude: the top 2% of each college's distribution of cumulative GPAs for the graduating cohort. Academic Honors (Latin Honors)

Data Glossary. Summa Cum Laude: the top 2% of each college's distribution of cumulative GPAs for the graduating cohort. Academic Honors (Latin Honors) Institutional Research and Assessment Data Glossary This document is a collection of terms and variable definitions commonly used in the universities reports. The definitions were compiled from various

More information

PEER EFFECTS IN THE CLASSROOM: LEARNING FROM GENDER AND RACE VARIATION *

PEER EFFECTS IN THE CLASSROOM: LEARNING FROM GENDER AND RACE VARIATION * PEER EFFECTS IN THE CLASSROOM: LEARNING FROM GENDER AND RACE VARIATION * Caroline M. Hoxby NBER Working Paper 7867 August 2000 Peer effects are potentially important for understanding the optimal organization

More information

Research Design & Analysis Made Easy! Brainstorming Worksheet

Research Design & Analysis Made Easy! Brainstorming Worksheet Brainstorming Worksheet 1) Choose a Topic a) What are you passionate about? b) What are your library s strengths? c) What are your library s weaknesses? d) What is a hot topic in the field right now that

More information

Evidence for Reliability, Validity and Learning Effectiveness

Evidence for Reliability, Validity and Learning Effectiveness PEARSON EDUCATION Evidence for Reliability, Validity and Learning Effectiveness Introduction Pearson Knowledge Technologies has conducted a large number and wide variety of reliability and validity studies

More information

Cooper Upper Elementary School

Cooper Upper Elementary School LIVONIA PUBLIC SCHOOLS www.livoniapublicschools.org/cooper 213-214 BOARD OF EDUCATION 213-14 Mark Johnson, President Colleen Burton, Vice President Dianne Laura, Secretary Tammy Bonifield, Trustee Dan

More information

Institution of Higher Education Demographic Survey

Institution of Higher Education Demographic Survey Institution of Higher Education Demographic Survey Data from all participating institutions are aggregated for the comparative studies by various types of institutional characteristics. For that purpose,

More information

BUILDING CAPACITY FOR COLLEGE AND CAREER READINESS: LESSONS LEARNED FROM NAEP ITEM ANALYSES. Council of the Great City Schools

BUILDING CAPACITY FOR COLLEGE AND CAREER READINESS: LESSONS LEARNED FROM NAEP ITEM ANALYSES. Council of the Great City Schools 1 BUILDING CAPACITY FOR COLLEGE AND CAREER READINESS: LESSONS LEARNED FROM NAEP ITEM ANALYSES Council of the Great City Schools 2 Overview This analysis explores national, state and district performance

More information

Evaluation of Teach For America:

Evaluation of Teach For America: EA15-536-2 Evaluation of Teach For America: 2014-2015 Department of Evaluation and Assessment Mike Miles Superintendent of Schools This page is intentionally left blank. ii Evaluation of Teach For America:

More information

Charter School Performance Accountability

Charter School Performance Accountability sept 2009 Charter School Performance Accountability The National Association of Charter School Authorizers (NACSA) is the trusted resource and innovative leader working with educators and public officials

More information

The Impact of Honors Programs on Undergraduate Academic Performance, Retention, and Graduation

The Impact of Honors Programs on Undergraduate Academic Performance, Retention, and Graduation University of Nebraska - Lincoln DigitalCommons@University of Nebraska - Lincoln Journal of the National Collegiate Honors Council - -Online Archive National Collegiate Honors Council Fall 2004 The Impact

More information

Demographic Survey for Focus and Discussion Groups

Demographic Survey for Focus and Discussion Groups Appendix F Demographic Survey for Focus and Discussion Groups Demographic Survey--Lesbian, Gay, and Bisexual Discussion Group Demographic Survey Faculty with Disabilities Discussion Group Demographic Survey

More information

LANGUAGE DIVERSITY AND ECONOMIC DEVELOPMENT. Paul De Grauwe. University of Leuven

LANGUAGE DIVERSITY AND ECONOMIC DEVELOPMENT. Paul De Grauwe. University of Leuven Preliminary draft LANGUAGE DIVERSITY AND ECONOMIC DEVELOPMENT Paul De Grauwe University of Leuven January 2006 I am grateful to Michel Beine, Hans Dewachter, Geert Dhaene, Marco Lyrio, Pablo Rovira Kaltwasser,

More information

FTE General Instructions

FTE General Instructions Florida Department of Education Bureau of PK-20 Education Data Warehouse and Office of Funding and Financial Reporting FTE General Instructions 2017-18 Questions and comments regarding this publication

More information

How to Judge the Quality of an Objective Classroom Test

How to Judge the Quality of an Objective Classroom Test How to Judge the Quality of an Objective Classroom Test Technical Bulletin #6 Evaluation and Examination Service The University of Iowa (319) 335-0356 HOW TO JUDGE THE QUALITY OF AN OBJECTIVE CLASSROOM

More information

Norms How were TerraNova 3 norms derived? Does the norm sample reflect my diverse school population?

Norms How were TerraNova 3 norms derived? Does the norm sample reflect my diverse school population? Frequently Asked Questions Today s education environment demands proven tools that promote quality decision making and boost your ability to positively impact student achievement. TerraNova, Third Edition

More information

University of Toronto

University of Toronto University of Toronto OFFICE OF THE VICE PRESIDENT AND PROVOST 1. Introduction A Framework for Graduate Expansion 2004-05 to 2009-10 In May, 2000, Governing Council Approved a document entitled Framework

More information

DATE ISSUED: 11/2/ of 12 UPDATE 103 EHBE(LEGAL)-P

DATE ISSUED: 11/2/ of 12 UPDATE 103 EHBE(LEGAL)-P TITLE III REQUIREMENTS STATE POLICY DEFINITIONS DISTRICT RESPONSIBILITY IDENTIFICATION OF LEP STUDENTS A district that receives funds under Title III of the No Child Left Behind Act shall comply with the

More information

Entrepreneurial Discovery and the Demmert/Klein Experiment: Additional Evidence from Germany

Entrepreneurial Discovery and the Demmert/Klein Experiment: Additional Evidence from Germany Entrepreneurial Discovery and the Demmert/Klein Experiment: Additional Evidence from Germany Jana Kitzmann and Dirk Schiereck, Endowed Chair for Banking and Finance, EUROPEAN BUSINESS SCHOOL, International

More information

Firms and Markets Saturdays Summer I 2014

Firms and Markets Saturdays Summer I 2014 PRELIMINARY DRAFT VERSION. SUBJECT TO CHANGE. Firms and Markets Saturdays Summer I 2014 Professor Thomas Pugel Office: Room 11-53 KMC E-mail: tpugel@stern.nyu.edu Tel: 212-998-0918 Fax: 212-995-4212 This

More information

Financial aid: Degree-seeking undergraduates, FY15-16 CU-Boulder Office of Data Analytics, Institutional Research March 2017

Financial aid: Degree-seeking undergraduates, FY15-16 CU-Boulder Office of Data Analytics, Institutional Research March 2017 CU-Boulder financial aid, degree-seeking undergraduates, FY15-16 Page 1 Financial aid: Degree-seeking undergraduates, FY15-16 CU-Boulder Office of Data Analytics, Institutional Research March 2017 Contents

More information

Principal vacancies and appointments

Principal vacancies and appointments Principal vacancies and appointments 2009 10 Sally Robertson New Zealand Council for Educational Research NEW ZEALAND COUNCIL FOR EDUCATIONAL RESEARCH TE RŪNANGA O AOTEAROA MŌ TE RANGAHAU I TE MĀTAURANGA

More information

Politics and Society Curriculum Specification

Politics and Society Curriculum Specification Leaving Certificate Politics and Society Curriculum Specification Ordinary and Higher Level 1 September 2015 2 Contents Senior cycle 5 The experience of senior cycle 6 Politics and Society 9 Introduction

More information

Learning But Not Earning? The Value of Job Corps Training for Hispanics

Learning But Not Earning? The Value of Job Corps Training for Hispanics Learning But Not Earning? The Value of Job Corps Training for Hispanics Alfonso Flores-Lagunes The University of Arizona Department of Economics Tucson, AZ 85721 (520) 626-3165 alfonso@eller.arizona.edu

More information

STEM Academy Workshops Evaluation

STEM Academy Workshops Evaluation OFFICE OF INSTITUTIONAL RESEARCH RESEARCH BRIEF #882 August 2015 STEM Academy Workshops Evaluation By Daniel Berumen, MPA Introduction The current report summarizes the results of the research activities

More information

Early Warning System Implementation Guide

Early Warning System Implementation Guide Linking Research and Resources for Better High Schools betterhighschools.org September 2010 Early Warning System Implementation Guide For use with the National High School Center s Early Warning System

More information

Moving the Needle: Creating Better Career Opportunities and Workforce Readiness. Austin ISD Progress Report

Moving the Needle: Creating Better Career Opportunities and Workforce Readiness. Austin ISD Progress Report Moving the Needle: Creating Better Career Opportunities and Workforce Readiness Austin ISD Progress Report 2013 A Letter to the Community Central Texas Job Openings More than 150 people move to the Austin

More information

Charter School Performance Comparable to Other Public Schools; Stronger Accountability Needed

Charter School Performance Comparable to Other Public Schools; Stronger Accountability Needed April 2005 Report No. 05-21 Charter School Performance Comparable to Other Public Schools; Stronger Accountability Needed at a glance On average, charter school students are academically behind when they

More information

The Effect of Income on Educational Attainment: Evidence from State Earned Income Tax Credit Expansions

The Effect of Income on Educational Attainment: Evidence from State Earned Income Tax Credit Expansions The Effect of Income on Educational Attainment: Evidence from State Earned Income Tax Credit Expansions Katherine Michelmore Policy Analysis and Management Cornell University km459@cornell.edu September

More information

Omak School District WAVA K-5 Learning Improvement Plan

Omak School District WAVA K-5 Learning Improvement Plan Omak School District WAVA K-5 Learning Improvement Plan 2015-2016 Vision Omak School District is committed to success for all students and provides a wide range of high quality instructional programs and

More information

Best Colleges Main Survey

Best Colleges Main Survey Best Colleges Main Survey Date submitted 5/12/216 18::56 Introduction page 1 / 146 BEST COLLEGES Data Collection U.S. News has begun collecting data for the 217 edition of Best Colleges. The U.S. News

More information

Massachusetts Department of Elementary and Secondary Education. Title I Comparability

Massachusetts Department of Elementary and Secondary Education. Title I Comparability Massachusetts Department of Elementary and Secondary Education Title I Comparability 2009-2010 Title I provides federal financial assistance to school districts to provide supplemental educational services

More information

Common Core Path to Achievement. A Three Year Blueprint to Success

Common Core Path to Achievement. A Three Year Blueprint to Success Common Core Path to Achievement A Three Year Blueprint to Success The Winds of Change Continue to Blow!!! By the beginning of the 2014-2015 School Year, there will be a new accountability system in place

More information

Undergraduates Views of K-12 Teaching as a Career Choice

Undergraduates Views of K-12 Teaching as a Career Choice Undergraduates Views of K-12 Teaching as a Career Choice A Report Prepared for The Professional Educator Standards Board Prepared by: Ana M. Elfers Margaret L. Plecki Elise St. John Rebecca Wedel University

More information

College Pricing. Ben Johnson. April 30, Abstract. Colleges in the United States price discriminate based on student characteristics

College Pricing. Ben Johnson. April 30, Abstract. Colleges in the United States price discriminate based on student characteristics College Pricing Ben Johnson April 30, 2012 Abstract Colleges in the United States price discriminate based on student characteristics such as ability and income. This paper develops a model of college

More information

HDR Presentation of Thesis Procedures pro-030 Version: 2.01

HDR Presentation of Thesis Procedures pro-030 Version: 2.01 HDR Presentation of Thesis Procedures pro-030 To be read in conjunction with: Research Practice Policy Version: 2.01 Last amendment: 02 April 2014 Next Review: Apr 2016 Approved By: Academic Board Date:

More information

Lecture 1: Machine Learning Basics

Lecture 1: Machine Learning Basics 1/69 Lecture 1: Machine Learning Basics Ali Harakeh University of Waterloo WAVE Lab ali.harakeh@uwaterloo.ca May 1, 2017 2/69 Overview 1 Learning Algorithms 2 Capacity, Overfitting, and Underfitting 3

More information

Enrollment Trends. Past, Present, and. Future. Presentation Topics. NCCC enrollment down from peak levels

Enrollment Trends. Past, Present, and. Future. Presentation Topics. NCCC enrollment down from peak levels Presentation Topics 1. Enrollment Trends 2. Attainment Trends Past, Present, and Future Challenges & Opportunities for NC Community Colleges August 17, 217 Rebecca Tippett Director, Carolina Demography

More information

Delaware Performance Appraisal System Building greater skills and knowledge for educators

Delaware Performance Appraisal System Building greater skills and knowledge for educators Delaware Performance Appraisal System Building greater skills and knowledge for educators DPAS-II Guide for Administrators (Assistant Principals) Guide for Evaluating Assistant Principals Revised August

More information

OFFICE OF ENROLLMENT MANAGEMENT. Annual Report

OFFICE OF ENROLLMENT MANAGEMENT. Annual Report 2014-2015 OFFICE OF ENROLLMENT MANAGEMENT Annual Report Table of Contents 2014 2015 MESSAGE FROM THE VICE PROVOST A YEAR OF RECORDS 3 Undergraduate Enrollment 6 First-Year Students MOVING FORWARD THROUGH

More information

https://secure.aacte.org/apps/peds/print_all_forms.php?view=report&prin...

https://secure.aacte.org/apps/peds/print_all_forms.php?view=report&prin... 1 of 35 4/25/2012 9:56 AM A» 2011 PEDS» Institutional Data inst id: 3510 Institutional Data A_1 Institutional Information This information will be used in all official references to your institution. Institution

More information

Jason A. Grissom Susanna Loeb. Forthcoming, American Educational Research Journal

Jason A. Grissom Susanna Loeb. Forthcoming, American Educational Research Journal Triangulating Principal Effectiveness: How Perspectives of Parents, Teachers, and Assistant Principals Identify the Central Importance of Managerial Skills Jason A. Grissom Susanna Loeb Forthcoming, American

More information

Teacher intelligence: What is it and why do we care?

Teacher intelligence: What is it and why do we care? Teacher intelligence: What is it and why do we care? Andrew J McEachin Provost Fellow University of Southern California Dominic J Brewer Associate Dean for Research & Faculty Affairs Clifford H. & Betty

More information

Rural Education in Oregon

Rural Education in Oregon Rural Education in Oregon Overcoming the Challenges of Income and Distance ECONorthwest )'3231-'7 *-2%2') 40%22-2+ Cover photos courtesy of users Lars Plougmann, San José Library, Jared and Corin, U.S.Department

More information

Effective Recruitment and Retention Strategies for Underrepresented Minority Students: Perspectives from Dental Students

Effective Recruitment and Retention Strategies for Underrepresented Minority Students: Perspectives from Dental Students Critical Issues in Dental Education Effective Recruitment and Retention Strategies for Underrepresented Minority Students: Perspectives from Dental Students Naty Lopez, Ph.D.; Rose Wadenya, D.M.D., M.S.;

More information

UK Institutional Research Brief: Results of the 2012 National Survey of Student Engagement: A Comparison with Carnegie Peer Institutions

UK Institutional Research Brief: Results of the 2012 National Survey of Student Engagement: A Comparison with Carnegie Peer Institutions UK Institutional Research Brief: Results of the 2012 National Survey of Student Engagement: A Comparison with Carnegie Peer Institutions November 2012 The National Survey of Student Engagement (NSSE) has

More information

Tun your everyday simulation activity into research

Tun your everyday simulation activity into research Tun your everyday simulation activity into research Chaoyan Dong, PhD, Sengkang Health, SingHealth Md Khairulamin Sungkai, UBD Pre-conference workshop presented at the inaugual conference Pan Asia Simulation

More information

U VA THE CHANGING FACE OF UVA STUDENTS: SSESSMENT. About The Study

U VA THE CHANGING FACE OF UVA STUDENTS: SSESSMENT. About The Study About The Study U VA SSESSMENT In 6, the University of Virginia Office of Institutional Assessment and Studies undertook a study to describe how first-year students have changed over the past four decades.

More information

Graduate Division Annual Report Key Findings

Graduate Division Annual Report Key Findings Graduate Division 2010 2011 Annual Report Key Findings Trends in Admissions and Enrollment 1 Size, selectivity, yield UCLA s graduate programs are increasingly attractive and selective. Between Fall 2001

More information

User Manual. Understanding ASQ and ASQ PLUS /ASQ PLUS Express and Planning Your Study

User Manual. Understanding ASQ and ASQ PLUS /ASQ PLUS Express and Planning Your Study User Manual ADMITTED STUDENT QUESTIONNAIRE ADMITTED STUDENT QUESTIONNAIRE PLUS TM ADMITTED STUDENT QUESTIONNAIRE PLUS EXPRESS Understanding ASQ and ASQ PLUS /ASQ PLUS Express and Planning Your Study About

More information