Economics Letters 58 (1998) 345 350 The effect of school quality on student performance: A quantile regression approach * Eric Eide, Mark H. Showalter Department of Economics, Brigham Young University, Provo, UT 84602, USA Received 29 April 1997; received in revised form 7 October 1997; accepted 22 October 1997 Abstract We use quantile regressions to estimate whether the relation between school quality and performance on standardized tests differs at different points in the conditional distribution of test score gains. Previous work has focused only on average school quality effects. 1998 Elsevier Science S.A. Keywords: School quality; Quantile regression JEL classification: I21 1. Introduction A number of studies have examined the effect of school quality on student achievement (e.g. Ehrenberg and Brewer, 1994, 1995; Hanushek, 1986, 1996). These studies find that, on balance, improving school resources such as the pupil teacher ratio or per pupil spending do not improve students performance on standardized achievement tests. This general conclusion runs counter to the conventional wisdom that the way to improve student achievement at public elementary and secondary schools is to allocate more money to them. Studies which analyze the effectiveness of school quality on student performance have primarily relied on estimation approaches such as Ordinary Least Squares (OLS) or Instrumental Variables (IV), which estimate the mean effect of school resource variables on student achievement. While estimating how on average school resources affect educational outcomes yields straightforward interpretations, the standard methodology may miss what is crucial for policy purposes, namely, how school resources affect achievement differently at different points of the conditional test score distribution. For example, while increases in per pupil spending may not matter for average test scores, it would be useful to know if increased spending increases test scores at the bottom of the conditional distribution. In short, we not only address the question, does money matter? but we also ask the question, for whom does money matter? *Corresponding author. Tel.: 11 801 3785169; fax: 11 801 3782844: e-mail: eide@byu.edu 0165-1765/ 98/ $19.00 1998 Elsevier Science S.A. All rights reserved. PII S0165-1765(97)00286-3
346 E. Eide, M.H. Showalter / Economics Letters 58 (1998) 345 350 2. Methodology and estimation Our approach is based on quantile regressions, which estimate the effect of explanatory variables on the dependent variable at different points of the dependent variable s conditional distribution. Quantile regressions were initially introduced as a robust regression technique which allows for estimation where the typical assumption of normality of the error term might not be strictly satisfied (Koenker and Bassett, 1978); it has also been used to estimate models with censoring (Powell, 1984, 1986; Buchinsky, 1994, 1995). Most recently, quantile regressions have been used simply to get information about points in the distribution of the dependent variable other than the conditional mean (Buchinsky, 1994, 1995; Eide and Showalter, 1997). We use quantile regressions to examine whether the effects of school quality differ across the quantiles in the conditional distribution of test score changes. Our regression specification follows the standard education production function approach of relating student outcomes to a vector of school inputs and a vector of individual-specific controls (e.g. Hanushek, 1986), and is based on data from the High School and Beyond (described below). Our dependent variable is the change in math test score between the sophomore and senior year of high school (or math score gain). This value-added form of estimation is discussed in Hanushek (1986). The school inputs we use are pupil teacher ratio, district per pupil expenditures, the fraction of teachers with an advanced degree, enrollment at the school, and the length of the school year. Our specification also contains individual controls for the initial math test score in the sophomore year, gender, race/ ethnicity, presence of father and mother in the household, educational attainment of father and mother, family income, family size, community residence, and region. We also include dummy variables to control for missing values of the explanatory variables, in which case the 1 explanatory variable was set to zero. As described by Koenker and Bassett (1978), the estimation is done by minimizing Eq. (1): Min O u uy 2 x bu 1 O (1 2u )uy 2 x bu K teht:y $x bj b e R t t t t t t t t teht:y,x bj (1) where yt is the dependent variable, xt is the k by 1 vector of explanatory variables, b is the coefficient vector and 1 is the quantile to be estimated. The coefficient vector b will differ depending on the particular quantile being estimated. 3. Data In estimating the effect of school resources on math score gains, we employ the High School and Beyond, a nationally representative longitudinal data set conducted by the National Center for Education Statistics. We use data on a cohort of students who were high school sophomores during the 1980 base year survey. Our sample consists of students who attended public high school in the sophomore year, did not drop out or transfer schools by the senior year, and have valid math scores in both the sophomore and senior years. 1 We also estimated all regressions excluding all missing observations on explanatory variables, and the results were nearly identical to those reported here.
E. Eide, M.H. Showalter / Economics Letters 58 (1998) 345 350 347 Table 1 Descriptive statistics Mean St. dev. School variables: Pupil teacher ratio 19.14 4.56 School year length (days) 180.22 5.44 % teachers with advanced degrees 47.86 23.99 School enrollment 1353.21 787.02 Per pupil district expenditures 1576.38 662.31 % black 14.59 23.41 % hispanic 10.73 21.60 Urban 0.22 0.42 Rural 0.33 0.47 Test score gain 1.40 4.90 1980 test score 9.64 7.75 % female 0.51 0.50 Information on individual, family and school characteristics come from the base year survey, and data on math achievement scores come from the base year and 1982 first follow-up surveys. Table 1 gives some descriptive statistics of the data. Additional information on the other control variables used in this study can be obtained from the authors. 4. Results In this section we present our quantile regression estimates of the effect of school quality inputs on changes in math scores. We estimate the model first by OLS and then at the 0.05, 0.25, 0.50, 0.75, and 2 0.95 quantiles. The results are presented in Table 2. First note the OLS results. Those variables typically thought to be under the control of policy makers such as the pupil teacher ratio, the length of the school year, per pupil expenditures (at the district level), and the educational attainment of teachers are all insignificantly different from zero, although the signs are usually in the direction one might expect a priori. These findings match those found by most other researchers (Hanushek, 1996). In contrast, the school enrollment variable is positive and statistically significant. The quantile regression results suggest some important differences across different points in the conditional distribution of math score changes. At the lower end of the distribution, the coefficients for year length are negative and insignificant; however, they are positive and significant for the median and the 0.75 and 0.95 quantiles. This suggests that performance at the top of the conditional distribution of math score changes is improved by a lengthened school year while performance at the 2 There are two econometric issues which should be noted. First, there may some selection bias in our sample since it excludes high school dropouts, and second if parents choose where to live based on quality of schools in the community, then some of the school inputs may be endogenous. However, Ehrenberg and Brewer (1994) account for these potential problems using High School and Beyond data, and found their estimates were largely unaffected.
348 E. Eide, M.H. Showalter / Economics Letters 58 (1998) 345 350 Table 2 Comparison of OLS and quantile regression results using the change in test score between the sophomore and senior year as the dependent variable Sample size: 8057 OLS Quantile 0.05 0.25 0.50 0.75 0.95 Pupil teacher ratio (3100) 20.666 23.859 20.936 20.436 0.330 20.355 1.486 3.847 1.743 1.634 1.756 2.894 School year length 0.016-0.009-0.002 0.029* 0.025** 0.027** 0.011 0.012 0.021 0.017 0.012 0.013 % teachers with advanced degrees -0.225-0.840-0.131-0.290 0.062-0.292 ( 3100) 0.255 0.565 0.281 0.290 0.312 0.456 School enrollment ( 31000) 0.250** 0.422** 0.255** 0.215** 0.221** 0.238 0.096 0.171 0.101 0.094 0.108 0.210 Per pupil district expenditures 0.145 0.430* 0.094 0.130-0.057 0.195 ( 31000) 0.096 0.251 0.154 0.093 0.143 0.174 Note: Asymptotic standard errors are given below each parameter estimate (heteroskedasticity robust for OLS; bootstrapped for quantiles). * denotes statistical significance at 10%; ** denotes significance at 5%. Other regressors include: initial math score in 1980, family size, % black population in school, % hispanic population in school, dummy variables for race, gender, educational attainment of father, educational attainment of mother, family income,school location and missing values for variables. bottom of the distribution appears not to benefit from the extra class time, other things being held equal. Another important result is the coefficient on per pupil expenditures. This is a variable that is generally found to be insignificant in most regressions which focus on the mean effect. In the quantile regressions, the coefficient is positive and insignificant at the 0.25 quantile and higher, but it is relatively large and significant for the bottom tail of the distribution suggesting that these expenditures may increase math scores for the lower part of the conditional distribution. Another notable result is the strength of the school enrollment variable in both the OLS regression and in the quantile regressions. The 0.25 estimate in the OLS regression implies that a 500 person increase in enrollment at a school leads to a 0.125 point average gain in individual test scores; interpretation for the quantiles is similar except the gain is for the conditional quantile rather than the mean. The parameter estimates are statistically significant in all cases except for the 0.95 quantile. The results for the pupil teacher ratio and the fraction of teachers with advanced degrees tend to have little effect on test score gains with generally insignificant coefficients of mixed signs across the various regressions. 5. Implications Public education represents one of the largest expenditures undertaken by federal, state, and local governments, and hence the effectiveness of this spending is of vital importance to educational policy makers. Previous results on this issue focused on how increased school spending affects average performance, and the findings suggest almost no positive effects. Our results suggest that some
E. Eide, M.H. Showalter / Economics Letters 58 (1998) 345 350 349 measures of school performance may have positive effects at points in the conditional distribution of test score gains other than the mean. In essence, our findings indicate where resources may matter, not just whether or not they matter on average. For instance, our results suggest that the marginal dollar allocated towards per pupil district expenditures raises test score gains at the bottom of the conditional distribution, and lengthening the school year improves performance in the top half of the conditional distribution, yet neither of these resource measures impact average test score gains. We should note that the way in which the additional per pupil expenditure is spent and how the additional time in school is used will obviously determine how effective these policies are in improving test score performance in the relevant points of the conditional distribution. Our most robust result of the effect of school enrollment on the conditional distribution of test score gains carries some intriguing implications. A simple interpretation of this finding is that there is some kind of increasing returns to scale from school size. Further reflection, however, suggests another plausible explanation. Note that in measuring the effect of enrollment, both the pupil teacher ratio and per pupil expenditures are held constant. This implies that the school enrollment variable is measuring the effect of simply having more classes. This likely allows more flexibility in sorting students among classes; some kind of ability segregation seems the most logical mechanism. This tracking argument seems more tenable to us than a more ambiguous returns to scale explanation since the test score gains are in the specific area of math. If the additional classes were in areas such as music or language, for example, it is unlikely that math scores would increase. Further, if our interpretation is correct, the largest increase in test scores will be at the bottom of the conditional distribution, where the 0.05 quantile estimate (0.422) is greater than the estimates for the other quantiles. 6. Conclusion Previous studies estimating the effect of school quality on student performance focused only on effects on mean performance. Our quantile regression results suggest that there may be differential school quality effects at different points in the test score gain conditional distribution. These findings are useful for educational policy makers since school quality measures which appear to have no effect for the average test score gains may indeed matter at other points of the conditional distribution of test score gains. Acknowledgements We are grateful to Bengte Evenson for research assistance and to an anonymous referee for helpful comments. This research was supported by a grant from the American Educational Research Association which received funds for its AERA Grants Program from the National Science Foundation and the National Center for Education Statistics (U.S. Department of Education) under NSF Grant [RED-9452861. Opinions reflect those of the authors and do not necessarily reflect those of the granting agencies.
350 E. Eide, M.H. Showalter / Economics Letters 58 (1998) 345 350 References Buchinsky, M., 1994. Changes in the U.S. wage structure 1963-1987: Application of quantile regression. Econometrica 62, 405 458. Buchinsky, M., 1995. Quantile regression, Box-Cox transformation model, and the U.S. wage structure, 1963 1987. Journal of Econometrics 65, 109 154. Eide, E., Showalter, M.H., 1997. Factors affecting the transmission of earnings across generations: a quantile regression approach. BYU Working Paper. Ehrenberg, R.G., Brewer, D.J., 1994. Do school and teacher characteristics matter? Evidence from high school and beyond. Economics of Education Review 13, 1 17. Ehrenberg, R.G., Brewer, D.J., 1995. Did teachers verbal ability and race matter in the 1960s? Coleman revisited. Economics of Education Review 14, 1 21. Hanushek, E., 1986. The economics of schooling: production and efficiency in public schools. Journal of Economic Literature 24, 1141 1177. Hanushek, E., 1996. School resources and students performance. In: Burtless, G. (Ed.), Does Money Matter? The Effect of School Resources on Student Achievement and Adult Success. Brookings Institution, Washington, D.C., pp. 43 73. Koenker, R., Bassett, G., 1978. Regression quantiles. Econometrica 46, 33 50. Powell, J.L., 1984. Least absolute deviations for the censored regression model. Journal of Econometrics 25, 303 325. Powell, J.L., 1986. Censored regression quantiles. Journal of Econometrics 32, 143 145.