Collegiate Assessment of Academic Proficiency: Mathematics Test Spring 2011 Administration

Transcription

1 Collegiate Assessment of Academic Proficiency: Mathematics Test Administration Page 1 of 36

2 Table of Contents Executive Summary... 3 Introduction... 5 Sample... 6 Results... 9 CAAP Scores Overall... 9 CAAP Scores by Sex, Major, GPA, and Effort Additional Questions Scores Overall Additional Questions Scores by Sex, Major, GPA, and Effort CAAP and Additional Questions Scores by Core Mathematics Course Relationship between CAAP Scores and Additional Questions Scores Conclusion Appendix Appendix A: Information about the CAAP Mathematics Test Appendix B: Copy of the Additional Questions Appendix C: Analysis of CAAP Scores Appendix D: Analysis of Additional Questions Appendix E: Analysis of CAAP and Additional Questions Scores by Core Curriculum Questions Appendix F: Effort by Student Classification, Sex, College, Major, and GPA Page 2 of 36

3 Executive Summary The Collegiate Assessment of Academic Proficiency (CAAP) Mathematics Test is designed to measure skills in mathematics. In coordination with the various Texas Tech University (TTU) Colleges, the Office of Planning and Assessment administered the Mathematics Test in select courses across campus at the beginning of the spring 2011 semester. Courses were chosen to achieve a representative number from each College based on enrollment of juniors and seniors. ACT allows the administering institution to add up to nine additional questions to the chosen CAAP module. TTU chose to add the five Mathematics questions from the Online Senior Assessment (OSA) plus two questions about participants completion of their Mathematics Core Curriculum Requirement. A total of 518 TTU students participated in the spring 2011 administration of the CAAP Mathematics Test. The goal was to score tests from 200 juniors and 200 seniors, so 118 answer sheets were excluded due to participants answer sheet irregularities, student classification, or college. A final sample of 175 juniors and 225 seniors is included in this analysis. The final sample seems to be a fairly good representation of the population of all TTU juniors and seniors in terms of sex, ethnicity, and college. In comparing scores from the CAAP Mathematics Test, it appears that TTU juniors (mean = 60.7, SD = 4.4) score higher on average than juniors nationally (mean = 58.0, SD = 4.3) and that TTU seniors (mean = 59.6, SD = 4.0) score higher than seniors nationally (mean = 56.8, SD = 4.1). These differences are statistically significant at the 0.05 level. The TTU sample shows statistically significant patterns in CAAP scores by sex, major, GPA, and. Male participants score higher on average than female participants. Math majors score higher on average than other majors. Participants with higher GPA s and participants reporting more also score higher on average. These same patterns apply to the additional questions scores by sex, major, and. Male participants, Math majors, and participants reporting more score higher on average. For the seniors, higher GPA s also predict higher additional questions scores, but this pattern by GPA was not statistically significant for the juniors. Both participants self report of whether they have completed their mathematics core courses and participants self report of whether they have completed their mathematics core courses at TTU do not have a statistically significant impact on CAAP scores or additional questions scores. The lack of significant comparisons by mathematics core completion may be due to the small sample sizes of juniors and seniors reporting that they have not completed their mathematics core courses. It might also be due to mistakes in students self report of completion. It could also be that the not completed group represents mostly students who have completed one of the two required math courses and that having the extra math course does not improve students scores on the CAAP or the additional questions. Page 3 of 36

4 The analysis shows a correlation between the CAAP scores and the additional questions scores of 0.55 (p value < ). This correlation suggests that there is a significant relationship between the additional questions scores and CAAP scores. This relationship may suggest that the CAAP and the additional questions are measuring similar domains of knowledge or that they are in some other way related. Page 4 of 36

5 Introduction The Collegiate Assessment of Academic Proficiency (CAAP) is the standardized, nationally normed assessment program from ACT that enables postsecondary institutions to assess, evaluate, and enhance student learning outcomes and general education program outcomes. 1 The CAAP offers six different modules: Reading, Writing Skills, Writing Essay, Mathematics, Science, and Critical Thinking. 2 As per decision of the University Assessment Committee in fall 2010, the Office of Planning and Assessment administered the Mathematics Test of the CAAP during spring The Mathematics Test is a 35-item, 40-minute test designed to measure students' proficiency in mathematical reasoning. The test assesses students proficiency in solving mathematical problems encountered in many postsecondary curricula. It emphasizes quantitative reasoning rather than the memorization of formulas. 3 Please find more information on how the test is constructed and scored in Appendix A. ACT allows the administering institution to add up to nine additional questions to the chosen module. The Core Curriculum Committee (CCC) at Texas Tech University (TTU) has been working on its own instrument to measure student abilities in various core areas, the Online Senior Assessment (OSA). The OSA was designed in 2008 and has been administered to graduating seniors from Texas Tech during the spring semester of 2008, 2009, and During the summer of 2010 the same assessment was administered to incoming transfer students at Texas Tech as the Transfer Student Assessment (TSA). The instrument has one section for each of the following areas: Humanities, Multicultural, Mathematics, Natural Sciences, Technology and Applied Science, and Social and Behavioral Sciences. The Mathematics section of the OSA is designed to measure abilities that are also measured by the CAAP Mathematics Test. Since the CAAP Mathematics Test is an established instrument it was decided to use the five Mathematics questions from the OSA as additional questions and compare the results. Another two questions were added as additional questions to ask participants if they have completed their Mathematics core requirement courses and if they completed all of these core requirement courses at TTU (see Appendix B for a copy of the additional questions). In coordination with the various Colleges, the Office of Planning and Assessment administered the Mathematics Test in select courses across campus. Courses were chosen to achieve a representative number from each College based on enrollment of juniors and seniors. and seniors were the target group, so courses with high junior and senior enrollment were selected to participate. Larger courses were chosen from Colleges with higher junior and senior enrollment and smaller courses were chosen from Colleges with lower junior and senior enrollment. Included in the sample were courses from the following Colleges: College of Agricultural Sciences and Natural Resources (AG), College of Architecture (AR), College of Arts 1 accessed 3/2/ accessed 3/2/ accessed 3/2/2011 Page 5 of 36

6 and Sciences (AS), College of Education (ED), College of Human Sciences (HS), College of Mass Communications (MC), College of Visual and Performing Arts (VP), Edward E. Whitacre Jr. College of Engineering (EN), and Jerry S. Rawls College of Business Education (BA). The CAAP Mathematics Test and the additional questions were administered during the regularly scheduled class time. This allowed most students from each class to participate and helped reduce the bias that may have been created if students had been asked to volunteer to take the survey on their own time instead. Sample A total of 518 Texas Tech University (TTU) students participated in the spring 2011 administration of the CAAP Mathematics Test. The goal was to score tests from 200 juniors and 200 seniors. The Office of Planning and Assessment identified 4 irregularities (e.g., pattern scoring) that were exempt from scoring. Also exempt from scoring were the 97 participants of student classifications other than juniors and seniors. Since there were only 175 juniors in the remaining sample, it was decided to score the results of 225 seniors instead of just 200 to achieve a good campus-wide representation. The remaining 400 answer sheets were scored by ACT. Since the students R numbers were used as the student ID on the Mathematics Test, the Office of Planning and Assessment was able to obtain demographic information for the participants from Institutional Research (IR). The demographic information obtained includes sex, ethnicity, student classification, and college. While participants were asked similar questions about their background on the CAAP Mathematics Test, it was decided to use official registration data provided by IR. Using the IR data facilitates comparison to the population numbers and has a greater chance of being accurate. The following graphs compare the final junior and senior samples to the population of all TTU juniors and seniors, respectively, by sex, ethnicity, and college. 60.0% 50.0% 40.0% 30.0% 20.0% 10.0% 0.0% Sample and Population by Sex: Female Male Sample Population 60.0% 50.0% 40.0% 30.0% 20.0% 10.0% 0.0% Sample and Population by Sex: Female Male Sample Population Page 6 of 36

7 The junior and senior samples both appear to be good representations of their respective populations in terms of sex. 80.0% 70.0% 60.0% 50.0% 40.0% 30.0% 20.0% 10.0% 0.0% Sample and Population by Ethnicity: Sample Population 80.0% 70.0% 60.0% 50.0% 40.0% 30.0% 20.0% 10.0% 0.0% Sample and Population by Ethnicity: Sample Population The junior and senior samples both appear to be good representations of their respective populations in terms of ethnicity. Page 7 of 36

8 40.0% 35.0% 30.0% 25.0% 20.0% 15.0% 10.0% 5.0% 0.0% Sample and Population by College: AG AR AS BA ED EN HS MC UC VP Sample Population 35.0% 30.0% 25.0% 20.0% 15.0% 10.0% 5.0% 0.0% Sample and Population by College: AG AR AS BA ED EN HS MC UC VP Sample Population It appears that the junior sample includes more participants from the Whitacre College of Engineering and fewer participants from the College of Human Sciences than would be expected from the population. The senior sample appears to be a fairly good representation of the population in terms of College. Page 8 of 36

9 Results CAAP Scores Overall Three scores are reported for the CAAP Mathematics Test: total test score (35 items), basic algebra subscore (17 items), and college algebra subscore (18 items). Potential total test scores range from 40 (low) to 80 (high) and potential subscores range from 5 (low) to 25 (high). The following table gives a summary of these three scores for the TTU junior and senior samples. Summary of CAAP Scores Total Test Basic Algebra College Algebra n Mean Standard Deviation Minimum Median Maximum It appears that the TTU juniors scored slightly better on average than the TTU seniors in total test scores (60.7 vs. 59.6), basic algebra scores (16.9 vs. 16.5), and college algebra scores (16.9 vs. 15.9). The differences in total test scores and college algebra scores are statistically significant at the 0.05 level and the difference in basic algebra scores is statistically significant at the 0.10 level (see Appendix C for details). This suggests that TTU juniors score higher on average than TTU seniors on the CAAP Mathematics Test. This pattern of juniors performing better on average than seniors is similar to CAAP participants nationally accessed 3/2/2011 Page 9 of 36

10 Percent of Sample Percent of Sample Collegiate Assessment of Academic Proficiency: Mathematics Test The following graphs show the distributions of total test scores, basic algebra scores, and college algebra scores for our junior and senior samples. CAAP Total Test Scores 60.0% 50.0% 52.0% 44.0% 40.0% 30.0% 20.0% 10.0% 0.0% 29.1% 28.0% 14.9% 9.1% 10.2% 8.4% 1.1% 0.9% 1.7% 0.4% Total Test Score The total test score distributions appear similar with a few more seniors scoring between 55 and 60 and a few more juniors scoring between 65 and 70. CAAP Basic Algebra Scores 60.0% 50.0% 40.0% 30.0% 20.0% 10.0% 0.0% 51.4% 41.3% 43.6% 28.0% 9.1% 8.9% 9.7% 5.3% 1.7% 0.9% 0.0% 0.0% Basic Algebra Score The basic algebra score distributions also appear similar with a few more seniors scoring between 13 and 16 and a few more juniors scoring between 16 and 22. Page 10 of 36

11 Percent of Sample Collegiate Assessment of Academic Proficiency: Mathematics Test CAAP College Algebra Scores 35.0% 30.0% 25.0% 20.0% 15.0% 10.0% 5.0% 0.0% 31.1% 29.1% 27.6% 26.9% 24.9% 18.9% 19.4% 13.3% 5.7% 0.0% 0.9% 2.2% College Algebra Score The college algebra score distributions include a few more seniors in each category of scores below 16 and a few more juniors in each category of scores above 16. It appears that the college algebra items contribute more than the basic algebra items to the difference in total test scores for TTU juniors and seniors. The ACT website has a summary for the scores of 12,744 juniors from 122 institutions and 9,402 seniors from 115 institutions who participated in the Mathematics Test in the fall of On average the TTU juniors scored higher than the national sample of juniors in total test scores (60.7 vs. 58.0), basic algebra scores (16.9 vs. 15.3), and college algebra scores (16.9 vs. 15.1). 6 On average the TTU seniors scored higher than the national sample of seniors in total test scores (59.6 vs. 56.8), basic algebra scores (16.5 vs. 14.6), and college algebra scores (15.9 vs. 14.3). 7 Each of these comparisons is statistically significant at the 0.05 level (see Appendix C for details). This suggests that TTU juniors and seniors score higher on average than juniors and seniors nationally on the CAAP Mathematics Test. 5 accessed 3/2/ accessed 3/2/ accessed 3/2/2011 Page 11 of 36

12 The following table shows the number of TTU juniors and seniors who received each total test score and the national percentile for each of these scores. 8 For each score, the national percentile shows the percentage of junior or senior participants, respectively, from the national sample that scored at or below that score. The highlighted cells show those TTU participants who scored higher than the 50 th national percentile. A majority of the TTU juniors (139 participants, 79%) and a large majority of the TTU seniors (185 participants, 82.2%) scored above the 50 th percentile nationally. National Percentiles for CAAP Scores Score TTU Natl. % TTU Natl. % accessed 3/2/2011 Page 12 of 36

13 CAAP Scores by Sex, Major, GPA, and Effort The following graphs show the average CAAP total test scores by sex, major, GPA, and participants self report of how much they gave in taking the assessment (See Appendix C for sample sizes) Average CAAP Scores by Sex Female Male For both the juniors and the seniors the male participants scored higher on average than the female participants. These differences by sex are statistically significant at the 0.05 level (see Appendix C for details). This suggests that on average male TTU juniors and seniors score higher than female TTU juniors and seniors on the CAAP Mathematics Test Average CAAP Scores by Major Math Major Other Major For both the juniors and the seniors the Math majors scored higher on average than the other majors. The Math majors group includes participants majoring in mathematics or other mathematics intense majors (e.g., Physics or Engineering). These differences by major are statistically significant at the 0.05 level (see Appendix C for details). This suggests that on average TTU juniors and seniors with a Math major score higher than TTU juniors and seniors with another major on the CAAP Mathematics Test. Page 13 of 36

14 Average CAAP Scores by GPA below and above It appears that on average the juniors and seniors with higher GPA s scored higher than the juniors and seniors with lower GPA s. These differences in scores by GPA are statistically significant at the 0.05 level (see Appendix C for details). More specifically, the 3.51 and above group scored significantly higher than the group and the group for the juniors, and the 3.51 and above group scored significantly higher than the group for the seniors. The results suggest that on average TTU juniors and seniors with higher GPA s score higher on the CAAP Mathematics Test. Average CAAP Scores by Effort tried my best gave moderate gave little gave no For both the juniors and seniors it appears that on average participants who reported more on the assessment scored higher. These differences between the groups are statistically significant at the 0.05 level (see Appendix C for details). More specifically, the tried my best group scored significantly higher than the other three groups for the juniors and seniors, and the gave moderate and gave little groups scored significantly higher than the gave no group for the seniors. The results suggest that on average TTU juniors and seniors who put forth more on the CAAP Mathematics Test score higher. Considering the impact of participants on CAAP scores, this section of the report will consider if participants self report of can help explain some of the other patterns found Page 14 of 36

15 in CAAP scores. The following table gives a summary of participants self report of by other demographic variables considered in this report. For ease of comparison, the percent of participants selecting tried my best and gave moderate are combined in one percentage and the percent of participants selecting gave little or gave no are combined in one percentage (see appendix F for more details). Effort by Student Classification (combined groups) best or moderate Little or no 92.1% 7.9% 85.1% 14.9% Effort by Sex (combined groups) best or moderate little or no best or moderate little or no Female 95.5% 4.5% 87.5% 12.5% Male 89.5% 10.5% 82.9% 17.1% Effort by Major (combined groups) best or moderate little or no best or moderate little or no Math Major 97.1% 2.9% 87.9% 12.1% Other Major 90.7% 9.3% 84.5% 15.5% Effort by GPA (combined groups) best or moderate little or no best or moderate little or no below % 20.0% 100.0% 0.0% % 20.0% 90.9% 9.1% % 17.0% 82.4% 17.6% % 0.0% 87.5% 12.5% 3.51 and above 100.0% 0.0% 81.8% 18.2% The junior participants were more likely to report more than the senior participants. This might help explain the higher average score for junior participants. However, the female juniors and seniors were more likely to report more than the male juniors and seniors. Since male participants performed better on average than female participants, this would suggest that does not help explain the differences in CAAP scores by sex. The junior and senior math majors were more likely to report more than the juniors and seniors with other majors. This might help explain the higher average scores for math majors. The patterns in reported by GPA seem to be consistent with the patterns in CAAP scores by GPA for the juniors, but not for the seniors. This inconsistency might be due to the small sample sizes included in some of the GPA groups. Page 15 of 36

16 Percent of Sample Collegiate Assessment of Academic Proficiency: Mathematics Test Additional Questions Scores Overall The following table and graph give a summary of the sample s scores on the additional questions (i.e., the five questions from the mathematics section of the Online Senior Assessment (OSA)). The 19 participants who did not answer any of the additional questions are excluded from this part of the analysis. Summary of Additional Questions Scores n Mean 55.3% 55.3% Standard Deviation 27.4% 27.2% Minimum 0% 0% Median 60% 60% Maximum 100% 100% Additional Questions Scores 30.0% 28.3% 25.0% 20.0% 15.0% 10.0% 5.0% 6.0% 3.7% 16.3% 13.9% 24.2% 23.3% 19.9% 21.7% 20.9% 11.6% 10.2% 0.0% 0% 20% 40% 60% 80% 100% Additional Questions Score It appears that on average the juniors and seniors performed very similarly on the additional questions. Both the juniors and seniors answered, on average, between two and three of the five additional questions correctly, with a low score of zero questions answered correctly and a high score of all 5 questions answered correctly. When considering the scores for the additional questions it is important to note that these questions were answered at the end of the testing period (i.e., after the participants had already taken the 40-minute-long Mathematics Test. This might explain the fact that the overall average score of these questions from this administration (55.3%) is lower than the average score from the last administration of these questions with the OSA (62.3%) accessed 3/2/2011 Page 16 of 36

17 Additional Questions Scores by Sex, Major, GPA, and Effort The following graphs show the average additional questions (AQ) scores by sex, major, GPA, and participants self report of how much they gave in taking the assessment (See Appendix D for sample sizes). 80.0% 70.0% 60.0% 50.0% 40.0% 30.0% 20.0% 10.0% 0.0% Average AQ Scores by Sex 63.8% 60.0% 49.5% 45.4% Female Male For both the juniors and the seniors the male participants scored higher on average than the female participants. These differences are statistically significant at the 0.05 level (see Appendix D for details). This suggests that on average male TTU juniors and seniors score higher than female TTU juniors and seniors on these additional questions from the mathematics section of the OSA. This is the same pattern that was found with the CAAP scores. 80.0% 70.0% 60.0% 50.0% 40.0% 30.0% 20.0% 10.0% 0.0% Average AQ Scores by Major 72.0% 70.6% 50.8% 52.2% Math Major Other Major For both the juniors and the seniors the Math majors scored higher on average than the other majors. These differences by major are statistically significant at the 0.05 level (see Appendix D for details). This suggests that on average TTU juniors and seniors with a Math major score higher than TTU juniors and seniors with another major on these additional questions from the mathematics section of the OSA. This is the same pattern that was found with the CAAP scores. Page 17 of 36

18 Average AQ Scores by GPA 80.0% 60.0% 60.0% 61.7% 53.3% 51.4% 52.6% 54.0% 52.3% 68.6% 63.3% 40.0% 37.8% 20.0% 0.0% below and above It appears that on average the seniors with higher GPA s scored higher than the seniors with lower GPA s. These differences by GPA group for the seniors are statistically significant at the 0.05 level (see Appendix D for details). More specifically, the 3.51 and above group scored significantly higher than the group, the group, and the below 2.00 group for the seniors. The results suggest that on average TTU seniors with higher GPA s score higher on the additional questions from the mathematics section of the OSA. This is the same pattern that was found for the seniors with the CAAP scores. It appears that on average the juniors from the highest GPA group and the lowest GPA group scored higher than the juniors from the middle GPA groups. This abnormal pattern is likely due in part to the small sample size of juniors in the lowest GPA group (n = 5). These differences by GPA group for the juniors are not statistically significant at the 0.05 level (see Appendix D for details). 80.0% 60.0% 40.0% 20.0% 0.0% 60.5% 54.9% 59.7% 53.8% tried my best gave moderate Average AQ Scores by Effort 27.5% gave little 45.2% 44.0% 6.7% gave no The question about applied directly to the CAAP Mathematics Test and not the additional questions, but it appears that the responses reflect in answering the additional questions as well, especially for the junior participants. It appears that on average the participants who reported more on the CAAP scored higher on the additional questions. These differences between the groups are statistically significant at the 0.05 level for the juniors and statistically significant at the 0.10 level for the seniors (see Appendix D for details). More specifically, the tried my best and gave moderate groups scored significantly higher than the gave little and gave no groups for the juniors. The results suggests that on average Page 18 of 36

19 Average CAAP Score Average CAAP Score Average AQ Score Collegiate Assessment of Academic Proficiency: Mathematics Test TTU juniors and seniors who put forth more on the CAAP Mathematics Test also score higher on the additional questions from the OSA. This is the same pattern that was found for the CAAP scores. CAAP and Additional Questions Scores by Core Mathematics Course Also included with the five additional questions from the mathematics section of the OSA were two additional questions asking participant if they had completed their mathematics core courses and if they completed all of their mathematics core courses at TTU. These questions allow us to see if having taken mathematics core courses and if having taken these core courses at TTU seems to make a difference in participants CAAP scores and Additional Questions scores. The following graphs show these comparisons Have you completed your mathematics core courses? 80% Have you completed your mathematics core courses? % 40% 55% 56% 56% 48% % 40.0 Yes No 0% Yes No Did you complete all of your mathematics core courses at TTU? % 60% 40% 20% Did you complete all of your mathematics core courses at TTU? 54% 57% 54% 57% 40 Yes No 0% Yes No It appears that having completed mathematics core courses does not have a large impact on CAAP or AQ scores. This is true for both the junior and senior participants (see Appendix E for details). It also appears that having completed mathematics core courses at TTU does not have a large impact on participants CAAP or AQ scores. This is also true for both the junior and senior participants (see Appendix E for details). The lack of significant findings in CAAP and AQ Page 19 of 36

20 scores by mathematics core completion may be partially due to the small number of juniors (n=28) and seniors (n=20) reporting that they have not completed their mathematics core courses. There may also be problems with the participants self report of their core requirement completion. For example, the sample of students reporting that they have not completed their mathematics core courses includes 8 students with math intense majors. It is not likely that students with math intense majors have not completed their mathematics core courses by their junior or senior year. Another possibility is that the not completed group represents mostly students who have completed one of the two required math courses, and that having the extra math course does not create a large difference in CAAP or additional questions scores. Relationship between CAAP Scores and Additional Questions Scores The following table shows how participants CAAP scores correlate with their overall additional questions (AQ) scores and with each separate additional question. Correlations with CAAP Scores AQ Score AQ1 AQ2 AQ3 AQ4 AQ5 Correlation p - value < < < < < < n The correlation between participants CAAP scores and participants additional questions scores (0.5479) is statistically significant at the 0.05 level. This means that there is a relationship between participants CAAP scores and participants additional questions scores. This relationship may suggest that the CAAP and the additional questions are measuring similar domains of knowledge or it might suggest that participants performance is in some other way related (e.g., participants who do well on tests do well on both the CAAP and the additional questions). Looking at each additional question separately, it appears that correct responses for all of the additional questions were significantly correlated with CAAP scores at the 0.05 level, but that these correlations are not as strong as the correlation with the additional questions score overall. Page 20 of 36

21 CAAP Score Collegiate Assessment of Academic Proficiency: Mathematics Test The following graph shows how participants additional questions scores relate to their CAAP scores. 75 CAAP Scores by AQ Scores % 20% 40% 60% 80% 100% 120% Additional Questions Score It appears that on average, participants CAAP scores increase as participants AQ scores increase. The red trend line shows this relationship. The following table shows a summary of the linear regression model that this trend line represents. Regression Model of CAAP Scores by Additional Questions Scores n F value p value R < Variable Coefficient Stand Error p value Intercept < AQ Score < The table confirms that participants additional questions scores are significantly related to participants CAAP scores. The R 2 value of reported in the table means that approximately 30% of the variance in CAAP scores can be explained by the variance in additional questions scores. This again suggests that participants additional questions scores and CAAP scores are related. Page 21 of 36

22 Conclusion Overall it appears that TTU juniors and seniors score higher on average than juniors and seniors nationally on the Mathematics Test from the CAAP. At TTU, it appears that the juniors score higher on average than the seniors. This pattern is consistent with the national samples of juniors and seniors. The results also suggest that at TTU male juniors and seniors score higher on average than female juniors and seniors. TTU juniors and seniors with a Math major score higher on average than juniors and seniors with another major. and seniors with higher GPA s score higher on average than TTU juniors and seniors with lower GPA s. TTU juniors and seniors who report that they put forth more on the assessment score higher. Patterns in participants self report of may help to explain the patterns in CAAP scores by student classification and major. The results of the additional questions from the mathematics section of the OSA, suggest that male juniors and seniors score higher than female juniors and seniors. and seniors with a Math major score higher than juniors and seniors with another major. with higher GPA s score higher on average than seniors with lower GPA s. There was not a statistically significant pattern in juniors scores by GPA, potentially due to the small number of participants in the lowest GPA group. Although not directly related to the additional questions, juniors and seniors who report that they put forth more on the CAAP also score higher on the additional questions. These patterns are consistent with the patterns found in CAAP scores. Participants self report of whether they have completed their mathematics core courses and whether they have completed their mathematics core courses at TTU did not have a significant impact on CAAP scores or additional questions scores. The lack of significant comparisons by mathematics core completion may be due to the small sample sizes of juniors and seniors reporting that they have not completed their mathematics core courses, or it might be due to mistakes in students self report of completion. It also might be that the not completed group represents mostly students who have completed one of the two required math courses and that having the extra math course does not improve students scores on the CAAP or the additional questions. The correlation between additional questions scores and CAAP scores suggests that there is a significant relationship between the two. This relationship may suggest that the CAAP and the additional questions are measuring similar domains of knowledge or that they are in some other way related (e.g., students who do well on tests do well on both the CAAP and the additional questions). The regression model using additional questions scores as the predictor for CAAP scores confirms this relationship. Page 22 of 36

23 Appendix Appendix A: Information about the CAAP Mathematics Test Screenshots from (accessed on 3/2/2011) Page 23 of 36

24 Page 24 of 36

25 Appendix B: Copy of the Additional Questions Student ID: R INSTRUCTIONS: Please turn to page 3 in your answer document and find block P, Additional Questions. Each question below corresponds to a column in block P. Read the question and then mark only one circle in each column that best describes your response. For example, if for question A you think answer option 7 is correct, you would bubble in the 7 in column A, etc. QUESTIONS Column A The core curriculum at Texas Tech requires you to have courses in mathematics. For example you could take a course in logic, college level algebra, finite math, statistics, calculus, etc. Have you completed your core curriculum requirement in Mathematics? (For example, this might include such classes at TTU as MATH 1300, MATH 1320, MATH 1330, or PHIL 2310) 0. Yes 1. No 2. I don t know Column B Did you complete all of your courses for your core curriculum requirement in Mathematics at Texas Tech? (Answering No would mean that you took a dual credit class, an advanced placement class, a CLEP exam, or transferred credits from another institution for your Mathematics core curriculum requirement) 0. Yes 1. No 2. I don t know Column C Page 25 of 36

26 According to the Cable News Network, the number of participants in roller-blade (in-line skating) competitions was 84% larger in 2003 as compared to The number of rollerblade competition participants from 1995 to decreased by about 16%. 1. increased by about 16%. 2. almost doubled. 3. almost tripled. Column D Which of the following numbers is largest? 0. 7/ /50 Column E The population of a western county is 100,000. The local development planners predict that the population will increase by 7% each year for each of the next 10 years. On the other hand, the county electoral board predicts that the population will increase by a constant factor of 7,500 each year for each of the next 10 years. Whose prediction method predicts a larger population for the county at the end of the 10-year cycle? 0. The local development planners method. 1. The county electoral board s method. 2. Both methods predict the same final population at the end of the 10-year cycle. 3. There is not enough information provided to answer the question. Column F Alice is looking to rent an art studio. A realtor has shown her two potential locations. Each location is suitable in terms of lighting, available space and access to local amenities. However, the rental contracts are substantially different. The contract for the first studio, Studio A, stipulates a non-refundable deposit of $1500 with a monthly rent of $450. The contract for the second studio, Studio B, stipulates a non-refundable deposit of only $750 but a monthly rent of $525. If Alice anticipates renting a studio for one year and wants the studio whose total cost for one year is less expensive, which studio contract should she accept? 0. The contract from studio A, because the total contract is less expensive. 1. The contract from studio B, because the total contract is less expensive. 2. Either contract, since the total contract is the same for either studio. 3. There is not enough information provided to answer the question. Column G Page 26 of 36

27 A group of students are thinking of producing a college magazine. They produce a prototype of the magazine and conduct a small survey to compare male and female opinions of it. The following question is asked of a random sample of students: Would you pay a dollar for this magazine? Forty females and thirty males replied. The results of the survey are shown in the following graph: Select the answer that most accurately describes the results shown in the graph above. 0. Females are less likely than males to buy the magazine. 1. Males are less likely than females to buy the magazine. 2. Males are more likely to buy the magazine. 3. Females and males are equally likely to buy the magazine. Page 27 of 36

28 Appendix C: Analysis of CAAP Scores CAAP Scores by Student Classification n Mean St. Dev. t value p value Basic Algebra Scores by Student Classification n Mean St. Dev. t value p value College Algebra Scores by Student Classification n Mean St. Dev. t value p value Page 28 of 36

29 CAAP Scores: TTU vs. National Sample () n Mean St. Dev. t value p value TTU < National Sample 12, CAAP Scores: TTU vs. National Sample () n Mean St. Dev. t value p value TTU < National Sample 9, Basic Algebra Scores: TTU vs. National Sample () n Mean St. Dev. t value p value TTU < National Sample 12, Basic Algebra Scores: TTU vs. National Sample () n Mean St. Dev. t value p value TTU < National Sample 9, College Algebra Scores: TTU vs. National Sample () n Mean St. Dev. t value p value TTU < National Sample 12, College Algebra Scores: TTU vs. National Sample () n Mean St. Dev. t value p value TTU < National Sample 9, CAAP Scores by Sex n Mean St. Dev. t value p value Page 29 of 36

30 () Female Male CAAP Scores by Sex () n Mean St. Dev. t value p value Female < Male CAAP Scores by Major () n Mean St. Dev. t value p value Math Major < Other Major CAAP Scores by Major () n Mean St. Dev. t value p value Math Major < Other Major Page 30 of 36

31 CAAP Scores by GPA () n Mean St. Dev. F value p value significant comparisons A) below < E > D B) E > C C) D) E) 3.51 and above CAAP Scores by GPA () n Mean St. Dev. F value p value significant comparisons A) below E > C B) C) D) E) 3.51 and above CAAP Scores by Effort () n Mean St. Dev. F value p value significant comparisons A) tried my best < A > B B) gave moderate A > C C) gave little A > D D) gave no CAAP Scores by Effort () n Mean St. Dev. F value p value significant comparisons A) tried my best < A > B B) gave moderate A > C C) gave little A > D D) gave no B > D C > D Page 31 of 36

32 Appendix D: Analysis of Additional Questions Additional Questions Scores by Student Classification n Mean St. Dev. t value p value % 27.4% % 27.2% Additional Questions Scores by Sex () n Mean St. Dev. t value p value Female % 25.3% Male % 28.3% Additional Questions Scores by Sex () n Mean St. Dev. t value p value Female % 24.8% 5.3 < Male % 26.5% Additional Questions Scores by Major () n Mean St. Dev. t value p value Math Major % 27.1% 4.26 < Other Major % 27.1% Additional Questions Scores by Major () n Mean St. Dev. t value p value Math Major % 27.3% Other Major % 26.3% Page 32 of 36

33 Additional Questions Scores by GPA () n Mean St. Dev. F value p value A) below % 14.1% B) % 32.7% C) % 27.8% D) % 26.0% E) 3.51 and above % 27.6% significant comparisons Additional Questions Scores by GPA () n Mean St. Dev. F value p value significant comparisons A) below % 29.1% E > D B) % 27.6% E > C C) % 25.9% E > A D) % 26.0% E) 3.51 and above % 28.1% Additional Questions Scores by Effort () n Mean St. Dev. F value p value significant comparisons A) tried my best % 28.0% A > C B) gave moderate % 24.1% A > D C) gave little % 30.1% B > C D) gave no 3 6.7% 11.5% B > D Additional Questions Scores by Effort () n Mean St. Dev. F value p value A) tried my best % 28.6% B) gave moderate % 26.6% C) gave little % 27.1% D) gave no % 32.9% significant comparisons Page 33 of 36

34 Appendix E: Analysis of CAAP and Additional Questions Scores by Core Curriculum Questions Have you completed your Mathematics Core (CAAP) n Mean St. Dev. t value p value Yes No Have you completed your Mathematics Core (CAAP) n Mean St. Dev. t value p value Yes No Did you complete your Core Courses at TTU (CAAP) n Mean St. Dev. t value p value Yes No Did you complete your Core Courses at TTU (CAAP) n Mean St. Dev. t value p value Yes No Page 34 of 36

35 Have you completed your Mathematics Core (AQ) n Mean St. Dev. t value p value Yes % 27.8% No % 25.7% Have you completed your Mathematics Core (AQ) n Mean St. Dev. t value p value Yes % 27.5% No % 21.9% Did you complete your Core Courses at TTU (AQ) n Mean St. Dev. t value p value Yes % 25.7% No % 29.4% Did you complete your Core Courses at TTU (AQ) n Mean St. Dev. t value p value Yes % 27.6% No % 26.9% Page 35 of 36

36 Appendix F: Effort by Student Classification, Sex, College, Major, and GPA Effort by Student Classification tried my best moderate little no 46.1% 46.1% 5.9% 2.0% 38.8% 46.3% 12.4% 2.5% Effort by Sex tried my moderate little no tried moderate little no best my best Female 42.4% 53.0% 3.0% 1.5% 37.5% 50.0% 10.4% 2.1% Male 48.8% 40.7% 8.1% 2.3% 40.0% 42.9% 14.3% 2.9% Effort by College tried my best moderate little no tried my best moderate little no Agriculture 23.1% 69.2% 7.7% 0.0% 21.4% 50.0% 21.4% 7.1% Architecture 40.0% 60.0% 0.0% 0.0% 50.0% 37.5% 12.5% 0.0% Arts and Sciences 47.8% 41.3% 8.7% 2.2% 42.1% 43.9% 10.5% 3.5% Business Admin. 38.1% 61.9% 0.0% 0.0% 44.7% 42.1% 10.5% 2.6% Education 71.4% 28.6% 0.0% 0.0% 66.7% 33.3% 0.0% 0.0% Engineering 67.6% 29.4% 2.9% 0.0% 40.6% 46.9% 12.5% 0.0% Human Sciences 15.4% 69.2% 7.7% 7.7% 23.5% 47.1% 23.5% 5.9% Mass Comm. 33.3% 33.3% 22.2% 11.1% 15.4% 84.6% 0.0% 0.0% University College 100.0% 0.0% 0.0% 0.0% 33.3% 33.3% 33.3% 0.0% Visual/Perform. Arts 33.3% 66.7% 0.0% 0.0% 40.0% 40.0% 20.0% 0.0% Effort by Major tried my best moderate little no tried my best moderate little no Math Major 67.6% 29.4% 2.9% 0.0% 39.4% 48.5% 12.1% 0.0% Other Major 39.8% 50.8% 6.8% 2.5% 38.7% 45.8% 12.5% 3.0% Effort by GPA tried my best moderate little no tried my best moderate little no below % 40.0% 20.0% 0.0% 42.9% 57.1% 0.0% 0.0% % 20.0% 20.0% 0.0% 36.4% 54.5% 9.1% 0.0% % 54.7% 11.3% 5.7% 37.6% 44.7% 15.3% 2.4% % 53.5% 0.0% 0.0% 34.4% 53.1% 9.4% 3.1% 3.51 and above 70.0% 30.0% 0.0% 0.0% 66.7% 33.3% 0.0% 0.0% Page 36 of 36