The Impact of Athletic Performance on Undergraduate College Choice

Transcription

1 University of Pennsylvania ScholarlyCommons Wharton Research Scholars Wharton School The Impact of Athletic Performance on Undergraduate College Choice Roshni Jain University of Pennsylvania Follow this and additional works at: Part of the Business Commons, and the Educational Assessment, Evaluation, and Research Commons Jain, Roshni, "The Impact of Athletic Performance on Undergraduate College Choice" (2004). Wharton Research Scholars This paper is posted at ScholarlyCommons. For more information, please contact

2 The Impact of Athletic Performance on Undergraduate College Choice Abstract The athletic success of men s basketball, football, baseball and soccer teams of National Collegiate Athletic Association (NCAA) Division I and Division III schools may impact the quantity, quality and matriculation rate of applicants. This study analyzes the winning percentages and playoff performance of teams from at 20 Division I colleges and 18 Division III colleges with the corresponding number of applicants, % of SAT scores above 500 (verbal and math separately) and matriculation rate through the construction of a regression model. The model controls for overall school quality and individual school effects through collected national rankings data and individual-level school variables. Results indicate that the effects vary between Division I and Division III schools, with success in baseball and football negatively impacting matriculation rates at Division III schools while an opposite effect is evident for baseball and matriculation rates at Division I schools. Basketball success appears to decrease student quality and the number of applicants at Division III schools, while baseball success improves student quality. Disciplines Business Educational Assessment, Evaluation, and Research This journal article is available at ScholarlyCommons:

3 The Impact of Athletic Performance on Undergraduate College Choice By Roshni Jain Advisor Eric Bradlow Wharton Undergraduate Research Scholars WH April 2004

4 Abstract The athletic success of men s basketball, football, baseball and soccer teams of National Collegiate Athletic Association (NCAA) Division I and Division III schools may impact the quantity, quality and matriculation rate of applicants. This study analyzes the winning percentages and playoff performance of teams from at 20 Division I colleges and 18 Division III colleges with the corresponding number of applicants, % of SAT scores above 500 (verbal and math separately) and matriculation rate through the construction of a regression model. The model controls for overall school quality and individual school effects through collected national rankings data and individual-level school variables. Results indicate that the effects vary between Division I and Division III schools, with success in baseball and football negatively impacting matriculation rates at Division III schools while an opposite effect is evident for baseball and matriculation rates at Division I schools. Basketball success appears to decrease student quality and the number of applicants at Division III schools, while baseball success improves student quality. 2

5 Introduction Gerald Phelan caught the ball! Gerald Phelan caught the ball! And so Doug Flutie s Hail Mary pass led to a Boston College win over Miami in one of the most remembered NCAA athletic events. College athletics can be a leading provider of visibility for an institution. From debates over the equality of college athletic programs and Title IX legislation, to team suspensions due to recruiting and academic violations, college athletics are an important source of income, and negative and positive publicity for many schools. Considering the importance of college athletics in the national spotlight, we began to pose questions about the impact of athletic performance (in particular - successful, or championship-winning athletic programs) on a school from a marketing perspective. While athletics are only one component of a university, if we draw the analogy where: (i) a well-rounded educational experience is the product, (ii) athletics is part of that product, and (iii) undergraduate students are a segment of customers, do successful teams help attract the right customers to your product? What characteristics of a college determine how effective winning athletic teams are in attracting applicants? Playoff success in what sports most impact application and matriculation rates at colleges and universities? What types of applicants do successful athletics teams attract? Is it a lower or higher quality student? The goal of this research will be to answer these questions through the analysis of athletic won-loss records, playoff performance, application statistics, matriculation rates, and test score statistics, while controlling for exogenous school factors, collected for four athletic conferences over a 10 year ( ) period. Essentially, how does performance in college athletics impact the number of applicants a school attracts, the number of students who matriculate once accepted, and 3

6 the overall quality of the matriculated student? In schools where athletics are spectatordriven, i.e. where athletics significantly impact the audience to a degree possibly more than the participants, a relationship between athletic success and the above-mentioned quantity and quality might be expected. These institutions are primarily classified as Division I by the National College Athletics Association, and are sometimes required to meet minimum attendance levels at their sporting events. 1 Our interests span even further, to trying to determine what type of relationships, if any, also exist between athletic success and applicant factors in schools where the athletics center around the experience of the athlete not the audience. According to the NCAA, in those schools classified as Division III, athletics departments place special importance on the impact of athletics on the participants rather than on the spectators. The student-athlete's experience is of paramount concern. 2 Additionally, we are looking to see if the relationships between athletic performance and the aforementioned applicant, matriculation, and quality metrics do exist for Division III schools, and if so, how do they vary from those that impact Division I schools? To address this issue, our research analyzed the effect of both general athletic success, measured by seasonal winning percentages and post-season success, (computed as the number of playoff rounds a team advanced to) in four men s sports football, basketball, baseball and soccer, for both Division I and Division III schools on applicant quantity, quality and matriculation rate. Our findings suggest that athletic performance does have an impact that varies across measures and differentially among Division I and III schools. 1 What's the difference between Divisions I, II and III? Available at About the NCAA, < Accessed on April 26, ibid. 4

7 The remainder of this research is as follows. In the next section we describe the existing literature that relates to our study. We then describe the framing of our study, i.e. the issues and choices that we made to implement the study. We follow that with a summary of our research findings and then a set of conclusions, limitations, and areas for future study. Existing Literature The major existing literature on this topic are studies that link athletic championships with application numbers, yield rates and applicant quality. One such example is a study that has been conducted by Toma and Cross (1996). The focus of their work is on NCAA Division I men s basketball and football championship winning schools (only) from 1979 to Using simple statistical calculations and data from Peterson s Guide to Four-Year Colleges and Universities, they found sizable increases in application numbers primarily in the year immediately following a championship, but up to five years after many of these schools won a national championship. They compared the increased application numbers of the sample schools with those of matched institutions to measure the athletic effect. Their research indicates little impact on the matriculation rate or applicant quality as a result of the successful athletic seasons. The selection criteria of the study makes these results most applicable to those universities where participation in or support of athletics are a prominent aspect of the student experience, especially in the marquee sports of men s basketball and football. Other literature, more broadly related to this topic address studies which consider intercollegiate athletics as a variable in college selection (Hamrick and Hossler 1996). These studies have explored athletics as a factor in the decision-making models of 5

8 college choice. More directly, they attempt to quantify the link between athletic records and application numbers. For example, Murphy and Trandel (1994) measured the increase in applications corresponding to increased winning percentages for over 40 college football teams. They found an effect of a 1.37% increase in applications for a 25% increase in winning percentage. However, there is little existing research studying the impact of less fan popular sports such as soccer and baseball or the impact of successful (participation in playoff tournament) seasons opposed to championship seasons only. Additionally there has been little work done studying these relationships at less athletically-focused Division III institutions and represents a significant part of the contribution here. Framing of the Problem and Methodology To begin constructing a model to uncover these relationships, the choices ( parameters ) of the data set construction needed to be determined, based upon the research questions we considered. These are described next. Which Schools? As our study aims to measure the size of the impact of athletic success on admission variables, we constructed a data set from a broad pool of schools. We looked at two complete Division I conferences and two complete Division III conferences. The Division I conferences selected were, the Big Ten, consisting of 11 schools, and the Atlantic Coast Conference consisting of 9 schools. The Division III conferences selected were the North Coast Athletic Conference, consisting of 10 schools and the University 6

9 Athletic Association consisting of 8 schools. These conferences were chosen as we found that they created a generally balanced set of Division I and III schools, and the majority of them participate in all or three of our targeted sports. Of this set of 38 schools, 33 schools have all four of the sports while 26 schools participate in three of the four. Which Sports? We expanded the focus of our research compared to existing studies and analyzed the impact of four men s sports. The marquee sports of basketball and football are probably the most well-known and would be expected to have the most visible impacts. In addition, we included soccer and baseball, as according to the NCAA these sports involve the most athletes and are played at the most schools after football and basketball. How to Measure Athletic Success? We used two primary methods of determining athletic success. The first was to collect win-loss-tie records for each of the sports for each of the years studied and to calculate winning percentages for each season. For these calculations, ties were considered.5 of a win and.5 of a loss. In the actual regressions run, as described below, the logit of these percentages was run in the model as the dependent variable to provide greater normality of errors. The second method we utilized for describing school athletic performance was measuring advancement in post-season tournaments. If a team did not reach the playoff tournament in a given sport in a given year, it was recorded as a zero, if it made it to the first round it was given a 1, it was given a 2 if it advanced another round forward and so on up until 6, or the championship round. This variable was treated as continuous in our 7

10 regression, although it is a discrete variable, something that should be taken into consideration; yet treating it as continuous provided a parsimonious representation. When running our model, both the winning percentage and playoff advancement were tested separately, as both measures are inherently correlated. What time period? The data we collected spans ten years from Unfortunately, limitations in data availability resulted in an incomplete analysis of all the records from this time period. Approximately six years of data among this set were complete, and certainly constructing this data set represented a unique challenge. The data was collected by hand from NCAA record books, conference record books and from media guides/record books published by each of the schools. What dependent variables? The dependent variable data all came from Peterson s Guides to Four-Year Colleges and Universities for the years (the guides publish data on the matriculated classes of two years prior to the guide date). The dependent variables we used were the number of applications a school received, the matriculation rate (calculated using the number admitted and the freshman class size), the % of incoming freshman class whose SAT V score was above 500 and the % of incoming freshman whose SAT M score was above 500. Although this is not an ideal measure of student quality, data availability limited us from being able to use higher SAT cutoffs or measures like GPA or ranking in high school class and is an area for future consideration. For the matriculation rate and SAT percents, the logit transformation of the numbers was used in the regression model. In certain cases 100% of incoming freshman were above a score of 8

11 500, in these instances a measure of.99 was used instead so a logit value could be calculated. A sensitivity analysis indicated that the exact choice of value near one did not significantly impact our findings. Methodology We created a stacked data set consisting of a row for each school for each year. To measure the difference in impact between Division I and Division III schools we created a division dummy that recorded a 1 for each Division I school and 0 for each Division III school. In addition, we created interaction terms to account for the difference in effect of each sport. The interaction terms are the product of the division dummy and either the winning percentage or playoff success variable for each sport. Thus each regression included an interaction variable for each of the sports included in that model. Additionally, we created control factors in order to reduce error. The control factors included individual school dummies for each school, in order to account for individual school effects. Moreover, US News and World Rankings for the year were used to control for the overall change in quality of the school. Specifically, schools not in the top 50 and in the second or third tier were recorded as being ranked at the beginning rank number of their respective tiers (e.g. a school ranked between 51 and 150 was given a ranking of 51). We note that the individual school dummies and US News Rankings were not redundant as the rankings did demonstrate fluctuation across the sample time period. To align the data for the regression, the athletic records and US News ranking were lagged in comparison to the admissions dependent variables. Spring sport 9

12 (basketball, baseball, soccer) records and US News rankings were matched with the following year s admissions data. For example, if a team had a certain record in the spring of 2003, it would impact the applications in the fall of 2003, which are recorded as statistics relevant to the incoming fall class of For football, a fall sport, the lag is of two years, because the fall season affects the following fall s applicants and the matriculates of two years later. Once the data set was complete we ran three combinations of regressions, schools with all four sports and their winning percentage, schools with three sports and their playoff success and schools with three sports and their winning percentage. This was done partially because the number of data points available with three sports was larger than those available with four sports, and partially to help understand the difference in impact between winning percentage (an overall good season) and playoff performance (a more public measure of a good season). The combination of all four sports and playoff performance was not run because of inadequate available data. Certainly more comprehensive methods to handle missing data were available, but were not applied here to simplify the analysis. Research Findings The regression models yielded some interesting relationships, some more expected than others. We will discuss the effects of winning percentage and playoffs on each of the four dependent variables. 10

13 Matriculation Rate The models (both with and without soccer included) testing for the impact of athletic performance on a school s matriculation rate suggested a significant impact for the football winning percentage (WP), that varied by division, and an overall regression model that is highly predictive (Rsq = 0.97, p<0.0001). The results are shown graphically and with summary output in Figure 1 below. In this case, the estimate for the football WP is negative, indicating that the more success a team has in a season, the more their matriculation rate will fall. This effect holds primarily for Division III schools. This is because the football interaction variable is a positive coefficient. As mentioned before, the interaction variables capture the difference in effect for Division I and Division III schools. As the dummy division variable is equal to 1 for Division I schools, the interaction terms only apply to Division I schools. The cumulative effect for Division I schools therefore is the sum of both the actual WP variables and their respective interactions. For matriculation, the effect of football cancels out and becomes slightly positive for these schools. Specifically, the Logit(Football WP) coefficient is and the Football interaction coefficient is These coefficients sum to a positive Given the small size of this coefficient relative to the other WP coefficients, the effect of football WP on Division I schools is insignificant from a matriculation standpoint. 11

14 Figure 1 Winning Percentage and Matriculation Rate Logit(Matriculation) Actual 0-1 F Ratio Prob > F < Logit(Matriculation) Predicted P<.0001 RSq=0.97 RMSE= Term Estimate Std.Error T Ratio Prob > ltl US News Rankings Logit (Football WP) Logit (Basketball WP) Logit (Baseball WP) Logit(soccer WP) Football Interaction Basketball Interaction Baseball Interaction Soccer Interaction The negative effects on matriculation for Division III do persist, however. The relationship is somewhat unexpected. Of course, it may also be that for Division III schools, they receive more and better applicants when athletic performance is better; yet these people in the end choose not to come. These results certainly are robust, yet to be more causal in interpretation requires further research beyond this study. Descriptive examples, i.e. actual percentages and effect sizes, of the impact of Division I and Division III athletics on matriculation rate, on a more common scale, are presented via examples in Appendix A. Applications The only significant impact we found on applications is that of basketball playoff performance. For Division III teams the impact was negative, with a coefficient of

15 applications and a t-stat of (p<0.10). The basketball playoff interaction coefficient was 1734 with a t-stat of 2.68 (p<0.01). Thus, when both estimates are summed, the net gain in applicants for every round that a Division I basketball team advances is 704 applicants. Given the large standard errors for the basketball playoff and basketball playoff interaction coefficients respectively, this impact of 704 is marginally significant. The coefficients and t-stats for the impact of WP on applications were insignificant in both the soccer and non-soccer conditions. A descriptive example effect size of this effect can be found in Appendix A. Figure 2 Playoff Performance and Applications Applications Actual Applications Predicted P<.0001 RSq=0.73 RMSE= F Ratio Prob > F <.0001 Term Estimate Std. Error T Ratio Prob>ltl Basketball Playoffs Football Playoffs Baseball Playoffs Basketball Playoff Interaction Football Playoff Interaction Baseball Playoff Interaction US News Rankings % of SAT M above 500 and % of SAT V above 500 The effects of athletic performance on these measures of student quality indicate that superior performance in baseball results in higher quality matriculated students in both SAT V and SAT M, while superior performance in basketball playoffs results in a 13

16 decrease in student quality as measured by the SAT M. In the SAT V, and models using winning percentage (WP), Division III schools saw an increase in scores with higher baseball WP in both the soccer and non-soccer conditions. The effect was in fact, stronger in the soccer case compared to the non-soccer case. In the non-soccer case the coefficient for the Logit of the Baseball WP was (p<0.01) while it was (p<0.01) in the soccer condition. Additionally, the positive effect nearly disappears in both cases for Division I schools, with the inclusion of the baseball interaction variable. In the non-soccer condition the sum of the Logit Baseball WP term and baseball interaction term results in a coefficient of and a coefficient of in the soccer condition, both insignificant. Figure 3 Winning Percentage and SAT V with Soccer 5 Logit(500 V) Actual F Ratio Prob > F < Logit(500 V) Predicted P<.0001 RSq=0.85 RMSE= Term Estimate Std. Error t-stat Prob<ltl US News Rankings Logit (Football WP) Logit (Basketball WP) Logit (Baseball WP) Football Interaction Basketball Interaction Baseball Interaction Logit(soccer WP) Soccer Interaction

17 Figure 4 Winning Percentage and SAT V without Soccer 5 Logit(500 V) Actual F Ratio Prob > F < Logit(500 V) Predicted P<.0001 RSq=0.81 RMSE= Term Estimate Std. Error T Stat Prob> ltl US News Rankings Logit (Football WP) Logit (Basketball WP) Logit (Baseball WP) Football Interaction Basketball Interaction Baseball Interaction For the SAT M and WP model, the coefficient for the Logit Baseball WP term is with a t-stat of 2.0, resulting in positive impact on student quality for Division III schools. For Division I schools, however, as in the SAT V case, the effect disappears. Adding the baseball interaction coefficient with a t-stat of results in a coefficient of , or a slightly negative effect for Division I schools. All of these results are in the soccer condition. Figure 5 Winning Percentage and SAT M Logit(500 M) Actual F Ratio Prob > F < Logit(500 M) Predicted P<.0001 RSq=0.88 RMSE=

18 Term Estimate Std. Error T Ratio Prob> ltl Logit (Football WP) Logit (Basketball WP) Logit (Baseball WP) Football Interaction Basketball Interaction Baseball Interaction Logit(soccer WP) Soccer Interaction The SAT M regression also shows evidence of a negative impact on student quality through basketball playoff success, most significantly in Division III schools. In this case, the basketball playoff coefficient is with a t-stat of Once again, the negative effect mostly disappears for Division I schools with the addition of the basketball playoff interaction term. The net result for Division I schools is a coefficient of with a t-stat of Descriptive examples of the quality effects can be found in Appendix A. Figure 6 Playoffs and SAT M Logit(500 M) Actual Logit(500 M) Predicted P<.0001 RSq=0.89 RMSE= Term Estimate Std. Error T-Stat Prob>ltl US News Rankings Basketball Playoffs Football Playoffs Baseball Playoffs Basketball Playoff Interaction Football Playoff Interaction Baseball Playoff Interaction F Ratio Prob > F <

19 Conclusions and Areas for Future Research Our analysis suggests that athletic performance in college men s basketball, football, baseball and men s soccer do have an impact on the applicant quantity, quality and matriculation rate for both Division I and Division III schools. The effects vary both in magnitude and in some cases direction of impact by division and by sport, with more impact seen in Division III schools. The matriculation rate in Division III schools appears to fall with increased WPs in football, while the opposite effect, albeit not as significant, is slightly positive for Division I schools. The reasons for this effect are not entirely clear and require further inquiry. Athletic playoff success also appears to have a detrimental effect on application quantity in Division III schools. There is negative impact on application numbers as basketball playoff success increases in Division III schools, while there is no significant impact on applications for Division I schools. This is somewhat contrary to prior studies and perhaps requires further testing with different sample sets or a more complex model. Overall success in baseball (both in WP and playoffs) leads to an increase in student quality of matriculated classes at Division III schools, while it has no impact on Division I schools. Conversely, basketball playoff success has a slightly negative impact on student quality in Division III schools. Why opposite impacts are present for baseball and basketball we cannot hypothesize without further investigation. These results may be more revealing through the use of a stricter measure of student quality. 17

20 Our study did not reveal any significant impact of soccer performance in Division I or Division III schools. This may be a function of the sport being low-profile and having no significant impact or because the sample size of schools with all four sports was not quite large enough. The next step for future research would be to run the same model with more or different Division I and Division III conferences to test for similar results. Challenges in data collection prevented us from testing a complete data set. Additionally, through the inclusion of multiple conferences, it would be interesting to explore the influence of geography on these athletics and admissions relationships. Further areas of investigation could also include a measurement of effects in Division II schools, men s swimming as it is the sport with next largest number of participants, and women s athletics. Athletic performance does appear to have an impact on aspects of student behavior critical to colleges and further research into the extent and explanation of these effects is warranted. 18

21 References Hamrick, Florence A. and Hossler, Don. Diverse Information Gathering Methods in the Postsecondary Decision making Process. Review of Higher Education, Vol. 19, Issue 2, p Murphy, R.G. and Trandel, G.A. The Relation Between a University s Football Record and the Size of its Applicant Pool. Economics of Education Review, Vol. 13, Issue 3 (September), p Toma, J. Douglas and Cross, Michael. Intercollegiate Athletics and Student College Choice: Understanding the Impact of Championship Seasons on the Quantity and Quality of Undergraduate Applicants. ASHE Annual Meeting Paper, November What's the difference between Divisions I, II and III? Available at About the NCAA, < Accessed on April 12,

22 Appendix A Example 1: Winning Percentage on Matriculation Rate In Division III schools an increase in football WP results in a lower matriculation rate. In Division I schools the effect cancels out. What does this mean in practice? Let s say that Wittenberg University has the following profile in 2003: Division Dummy: 0 Basketball WP:.4 Football WP:.67 Baseball WP:.70 Soccer WP:.5 US Ranking: 81 Using the coefficients in the model, the matriculation rate: If in 2004, the Wittenberg University teams do as follows: Basketball WP:.4 Football WP:.90 Baseball WP:.70 Soccer WP:.5 US Ranking: 81 The 13% increase in Football WP results in the matriculation rate: A % change, or about 1 less student who matriculates (Wittenberg freshman class size 650). Example 2: Baseball Winning Percentage on SAT V In Division III schools, higher baseball WPs result in higher SAT V scores. In Division I schools there is still a positive, but lesser effect. What does this mean in practice? Let s say that Wittenberg University has the following profile in 2003: Division Dummy: 0 Basketball WP:.4 Football WP:.67 Baseball WP:.70 Soccer WP:.5 US Ranking: 81 Using the coefficients in the model, the % SAT V scores above 500: If in 2004, the Wittenberg University teams do as follows: Basketball WP:.4 Football WP:.67 20

23 Baseball WP:.80 Soccer WP:.5 US Ranking: 81 The 10% increase in Baseball WP results in the % SAT V scores above 500: A.804 % change, or about 5 students with higher scores (Wittenberg freshman class size 650). Example 3: Baseball Winning Percentage on SAT M In Division III schools higher baseball WPs result in higher SAT M scores. For Division I schools there is still a positive, but minimal effect. What does this mean in practice? Let s say that Wittenberg University has the following in 2003: Division Dummy: 0 Basketball WP:.4 Football WP:.67 Baseball WP:.70 Soccer WP:.5 US Ranking: 81 Using the coefficients in the model, the % SAT M scores above 500: If in 2004, the Wittenberg University teams do as follows: Basketball WP:.4 Football WP:.67 Baseball WP:.80 Soccer WP:.5 US Ranking: 81 The 10% increase in Baseball WP results in the % SAT M scores above 500: A.533 % change, or about 4 students with higher scores (Wittenberg freshman class size 650). Example 4: Basketball Playoff success on SAT M A somewhat contrasting result than baseball. For Division III schools more success in basketball playoffs results in lower SAT M scores. In Division I schools there is still a negative, but lesser effect. What does this mean in practice? Let s say that Wittenberg University has the following in 2003: Division Dummy: 0 Basketball Playoff: 1 Football Playoff: 1 21

24 Baseball Playoff: 0 US Ranking: 81 Using the coefficients, the % SAT M scores above 500: If in 2004, the Wittenberg University teams do as follows: Basketball Playoff: 2 Football Playoff: 1 Baseball Playoff: 0 US Ranking: 81 If the basketball team advances two rounds further the % SAT M scores above 500: A % change, or about 17 students with lower scores (Wittenberg freshman class size 650). Example 5: How interaction makes the effect disappear for Division I schools Let s say that the University of Minnesota has the following in 2003: Division Dummy: 1 Basketball Playoff: 1 Football Playoff: 1 Baseball Playoff: 0 US Ranking: 51 Using the coefficients, the % SAT M scores above 500: If in 2004, the University of Minnesota teams do as follows: Basketball Playoff: 3 Football Playoff: 1 Baseball Playoff: 0 US Ranking: 30 If the basketball team advances two rounds further the % SAT M scores above 500: A -.35 % change, or about 17 students with lower scores (Minnesota Freshman class size 5000). Example 6: Playoff Success and Applications Let s say that the University of Minnesota has the following in 2003: Division Dummy: 1 Basketball Playoff: 1 Football Playoff: 1 Baseball Playoff: 0 US Ranking: 51 22

25 Number of applications predicted: 16,831 If in 2004, the University of Minnesota teams do as follows: Basketball Playoff: 2 Football Playoff: 1 Baseball Playoff: 0 US Ranking: 51 If the basketball team advances two rounds further the number of applications predicted is: 17,540 This is an additional: 709 or a 4.30% increase in the normal size of the applicant pool. 23