The Effect of Attending Peer Tutoring on Course Grades in Calculus I Brian Rickard University of Arkansas Melissa Mills Oklahoma State University Tutoring centers are common in universities in the United States, but the effects of tutoring on student success are often not examined statistically. This study utilizes multiple regression analysis to model the effect of tutoring attendance on final course grades in Calculus I. Our model predicted that every three visits to the tutoring center would increase a students course grade by one percent, after controlling for prior academic ability. We also found that for lower achieving students attending tutoring had a greater impact on final grades. Keywords: Calculus I, multiple regression, peer tutoring, undergraduate mathematics Introduction and Literature Review Although many institutions across the nation offer free tutoring to students (Johnson & Hansen, 2015) with the goal of improving lower division instruction, there have been few published studies investigating the impact of attending tutoring on performance (Xu, Hartman, Uribe & Mencke, 2001). A common approach is to simply look at success rates for students who attended peer tutoring versus success rates for students that did not attend tutoring (e.g. Garcia, Morales & Rivera, 2014; Jimenez, Acuna, Quiero, Lopez & Zahn, 2015). While this may provide some evidence that tutoring had a positive impact, we argue that this type of analysis is oversimplified. Unfortunately, reporting on student performance and retention in an in-depth manner requires resources in terms of staff time and collaboration with other disciplines that many tutoring centers simply do not have (MacGillivray, 2009). Quantitative measures of the impact of tutoring on grades can be difficult because students tend to self-select and students of different mathematical abilities may attend tutoring for different reasons (Topping, 1996). One way to account for these factors is to use multiple regression. Regression models have been used to show that attending Peer Assisted Learning sessions can improve the grades of mathematics majors (Duah, Croft & Inglis, 2013) and that attending optional tutoring can improve the grades of college algebra students while controlling for students mathematical abilities (Xu, et. al, 2001). This study will add to the literature by using multiple regression to measure the impact of attending optional tutoring sessions at the Mathematics Learning Success Center on course grades for Calculus I students at Oklahoma State University. The research questions for the study are: 1. What is the effect of attending optional drop-in tutoring offered by the MLSC at Oklahoma State University on Calculus I students course grades, after controlling for their high school math GPA and ACT math sub-score? 2. Do we see that attending optional drop-in tutoring at the MLSC benefits students of lower mathematical ability more than stronger students?
Method and Data To evaluate the effectiveness of mathematics tutoring in calculus, student academic and tutoring data were collected from a public 4-year research university in the Midwest with an enrollment of approximately 25,000 students. Study participants include all 640 students enrolled in Calculus I for the fall 2015 semester. Since attending tutoring sessions is voluntary and students receive no credit for participation in tutoring, self-selection bias is acknowledged as a limitation of this study. Data collected for the study includes: student course grade (percentage) in Calculus I, high school math grade point average, ACT math score, number of visits to the tutoring center, and duration of visits at the tutoring center. For visits with a missing duration, the average of the student s other visit durations is used as an estimate. Of the 640 students enrolled in Calculus I, there were 390 students who visited the tutoring center a total of 5193 times. Table 1. Descriptive Statistics Mean Standard Deviation Minimum Maximum Final Course Grade (Percent) 73.93 22.49 3.44 101.46 High School Math GPA 3.51 0.58 1.63 4.75 ACT Math 26.55 3.60 14 36 Visits 8.11 13.75 0 102 Estimated Total Time (Minutes) 738.45 1435.43 0.00 11549.03 Of the 640 students in the study, 390 (60.9%) visited the tutoring center at least one time. For the students who did visit the tutoring center, the average number of visits was 13.3 per student with an average visit length of 78 minutes. These students had slightly higher prior academic achievement scores with an average high school math GPA of 3.58 and math ACT of 26.8 compared to 3.38 and 26.2 respectively for those who did not visit the tutoring center. The average course grade earned for students who attended tutoring was a B (80.4%) while the average for those who did not attend tutoring was a D (62.0%). There were 534 students who completed the final exam and 78 (12.2%) withdrew from the course. Overall, as the number of times a student visits the tutoring center increases, so does the course grade. However, on average, students with a higher frequency of visits also had a higher high school math GPA, so it is unclear at this point whether the increase in exam score is a result of increased tutoring visits or prior math ability (see Table 2). Table 2. Scores by Tutoring Visit Category Number of Students Average Final Exam Average High School Score Math GPA 0 250 62.0 3.38 1-5 169 78.2 3.52 6-10 64 80.3 3.54 11-20 72 82.3 3.65 21+ 85 83.3 3.68 Overall 640 73.9 3.51
Results To investigate the relationship between the variables, Pearson r correlations were computed (see Table 3). The correlation between visits and estimated total time is close to 1 which indicates possible multicollinearity and that likely only one of these variables will be needed in the final model. Table 3. Correlation Matrix High School Math GPA 0.50* Final High School Math GPA ACT Math Visits ACT Math 0.42* 0.42* Visits 0.26* 0.15* -0.03 Estimated Total Time 0.23* 0.12* -0.05 0.94* Note. * significant at p < 0.05 Simple linear regressions using each independent variable separately as a predictor of the dependent variable indicate that each independent variable individually is a significant predictor of course grades. The initial multiple regression model will therefore include all four independent variables. Analysis of this model indicates significant overall statistical predictive ability, F(4, 455)=60.63, p<0.0001. The R 2 of this model is 0.348, which indicates that approximately 34.8% of the variance in course grades can be explained by the predictor variables. In this model, Estimated Total Time has a large p-value of 0.9016 indicating it is unlikely to be a meaningful predictor. Removing this variable and analyzing the subsequent model with predictor variables of high school math GPA, math ACT, and visits results in a significant overall model, F(3, 485) =85.27, p<0.0001, and an R 2 of 0.345. Each independent variable was found to have a p-value of less than 0.0001 indicating a they are all likely meaningful predictors. The R 2 values indicate the proportion of variance in course grades that can be uniquely accounted for by that predictor variable. Coefficients, correlations and collinearity statistics are found in Table 4. Table 4. Multiple Linear Regression Results Parameter Estimate Constant -21.677 P-Value Partial R 2 Tolerance High School Math GPA 12.993 <0.0001 0.13 0.81 ACT Math 1.782 <0.0001 0.10 0.83 Visits 0.333 <0.0001 0.06 0.97 The parameter estimates in Table 4 indicate the change in predicted course grade for each unit change in the predictor variable. As such, the parameter estimate of 0.333 for visits indicates that a student s course grade is predicted to be approximately one percentage point higher for every three visits to the tutoring center. The prediction equation for this regression is: Final Course Grade = -21.677 + 12.993(High School Math GPA) + 1.782(ACT Math)+0.333(Visits).
Interaction It is also of interest to determine if students of lower mathematical ability, determined by high school math GPA, benefit more from tutoring than students with higher mathematical ability. In Table 5, mean course grades are compared between those who did attend tutoring and those who did not at different categories of high school GPA. From this table, it does appear that lower mathematical ability students benefit more from tutoring with a decreasing difference in mean course grades with increasing mathematical ability. Table 5. Mean Final Exam Score by HS GPA and Tutoring Participation HS GPA Category No Tutoring 1+ Tutoring Visits Mean Difference 2.00-2.49 25.9 66.6 40.7 2.50-2.99 56.4 72.7 16.3 3.00-3.49 58.1 74.2 16.1 3.50-3.99 71.6 86.4 14.8 4.00+ 78.5 87.2 8.7 To investigate this analytically, a regression with an interaction between high school GPA and number of visits is analyzed. The results from this analysis are found below in Table 6. It should be noted that the independent variables in this analysis have been centered to mitigate multicollinearity due to the inclusion of both GPA and Visits and the interaction term between those variables. Table 6. Multiple Linear Regression Results Parameter Estimate P-Value Constant 74.105 <0.0001 Partial R 2 Tolerance High School Math GPA 12.929 <0.0001 0.13 0.83 ACT Math 1.798 <0.0001 0.10 0.81 Visits 0.365 <0.0001 0.07 0.89 High School Math GPA/Visits Interaction -0.203 0.0555 0.01 0.93 In this analysis the predictors of high school math GPA, ACT math, Visits and the interaction term all have very low p-values indicating they are likely meaningful predictors of the final course grade. However, it is worth noting that the partial R 2 of the interaction term of 0.01 indicates that the interaction is only able to account for an additional 1% of the variation in course grades over that of the other dependent variables. The overall R 2 of this model is.350 compared to.345 for the model without interaction which indicates that the predictor variables are able to account for 35.0% of the variance in course grades. The sign of the coefficient of the interaction term suggests that visits for students with lower high school math GPA result in a larger increase in course grade than for students with a higher
high school math GPA. The predicted increase in course grade per visit to the tutoring center of a student with a high school math GPA one standard deviation above the mean is 0.25 points. This increase is 0.37 points for students with an average high school math GPA and 0.48 points for students with a high school math GPA one standard deviation below the mean. In other words, students with lower prior math academic achievement see a larger increase in course grade per tutoring visit than students of higher prior math academic achievement. The prediction equation for this regression is: Final Course Grade = -27.752+ 14.576(High School Math GPA) + 1.798(ACT Math)+1.077(Visits)-0.203(Visits*High School Math GPA). The coefficients of this equation have been simplified from the centered form of the equation and therefore differ from those in Table 6. Discussion We attempted to control for students prior mathematical ability by using high school GPA and ACT math sub-score as variables in the multiple regression model. We found that high school math GPA, ACT math sub-score, and the number of tutoring visits were all significant factors in predicting course grades. The model predicts that each visit to the tutoring center raises the student s grade by 0.33%. A student with the mean high school math GPA and the mean ACT math sub-score who does not attend tutoring is predicted to make a 60% in the course. If that same student attends tutoring twice a week for the whole semester (30 visits), the predicted course grade is raised to 70%. To determine if tutoring attendance benefits low achieving students more than high achieving students, we developed a new regression model that includes the interaction between high school math GPA and tutoring visits. This interaction model has slightly better predictive power than the previous multiple regression model. The model predicts that a low achieving student (with high school math GPA and ACT math score each one standard deviation below the mean) would score 56% without tutoring, and would need to attend tutoring 28 times to raise their score to a passing grade (70%). In contrast, a high achieving student (with high school math GPA and ACT math score each one standard deviation above the mean) would score 86% without tutoring, and 28 visits to the tutoring center would only raise the student s grade 7 percentage points. Thus, for the lower achieving student, each visit to the tutoring center has more of an impact on his or her course grade. There are several limitations to this study. We acknowledge that the students had the option to attend tutoring, so there is a self-selection bias that we attempted to control by using prior mathematical ability scores in the multiple regression model. We also have no data about whether or not students made use of other support services, such as office hours or independent study groups.
References Duah, F., Croft, T., & Inglis, M. (2013). Can peer assisted learning be effective in undergraduate mathematics? International Journal of Mathematics Education in Science and Technology. Available from: http://dx.doi.org/10.1080/0020739x.2013.855329 Garcia, R., Morales, J., & Rivera, G. (2014). The use of peer tutoring to improve the passing rates in mathematics placement exams of engineering students: a success story. American Journal of Engineering Education, 5, 61-72. Jimenez, V., Acuna, F., Quiero, F., Lopez, M. & Zahn, C. (2015). Evaluation of a voluntary tutoring program in chemistry, physics, and mathematics for first-year undergraduates at Universidad Andres Bello, Chile. International Journal of Higher Education, 4, 21-31. Johnson, E. & Hanson, K. (2015) Academic and social supports. In Bressoud, D., Mesa, V., & Rasmussen, C., Eds. Insights and Recommendations from the MAA National Study of College Calculus. Washington, DC: MAA Press. p. 69-82. MacGillivray, H. (2009). Learning support and students studying mathematics and statistics. International Journal of Mathematics Education in Science and Technology. 40, 455-472. Topping, K. J. (1996). The effectiveness of peer tutoring in further and higher education: A typology and review of the literature. Higher Education, 32, 321-345. Xu, Y., Hartman, S., Uribe, G., & Mencke, R. (2001). The effects of peer tutoring on undergraduate students final examination scores in mathematics. Journal of College Reading and Learning, 32, 22-31.