(Includes a Detailed Analysis of Responses to Overall Satisfaction and Quality of Academic Advising Items) By Steve Chatman

Report #202-1/01 Using Item Correlation With Global Satisfaction Within Academic Division to Reduce Questionnaire Length and to Raise the Value of Results An Analysis of Results from the 1996 UC Survey of Undergraduate Students (Includes a Detailed Analysis of Responses to Overall Satisfaction and Quality of Academic Advising Items) By Steve Chatman This paper describes an application of inter-item correlation analysis to undergraduates responses to a general survey of student opinion and experience to satisfy two objectives: (1) to identify where ratings of satisfaction correlate with overall satisfaction in order to better understand the student experience and (2) to reduce survey length by eliminating items that are apparently of little consequence in producing satisfied students. The analysis is based on results from a UC undergraduate survey in which UC Davis participated in the spring of 1996. The item reduction strategy is as follows. If responses to an item are highly correlated with responses to the global satisfaction item and if those correlations are significant for more than one academic division, then the item is operationally defined as important and is retained. In this instance, visual examination of a plot showing average strength of association across divisions by frequency significant instances is used to eliminate nearly one-half the initial pool. The analysis could have been performed at the institutional level and, if analyzed at the level of the institution would have been straightforward and would have naturally reflected the opinions of the student body. However, item correlation with global satisfaction varies by academic division and that variance would be lost in an institutional analysis. The special importance of divisional variance is that faculty members are best able to affect student satisfaction and a report that presents data closer to their level of experience and interest has a better chance of being successful. To summarize, the divisional analysis of inter-item correlation with a single measure of global satisfaction can be used to (1) construct reports that are more directly meaningful to faculty and to (2) eliminate less important items from future administrations. The first application, constructing more meaningful reports, is valid on its face, as each academic division should receive information tailored to fit when item responses clearly vary by division. The second application extends this reasoning more generally to eliminate from future item pools those items that are infrequently associated with global satisfaction and are weakly associated when they reach statistical significance. This paper emphasizes the second application, survey item reduction. Survey Davis is one of four UC campuses conducting a survey of enrolled students in 1995-96 as part of a System effort to create a common survey form or at least a survey with a common core of items. Questions on the survey are gleaned and tailored from a variety of sources (e.g., ACT College Outcomes Survey) with

supplemental sections on student time allocation and information technology. The survey was mailed to 2,500 randomly selected UC Davis undergraduate students along with a transmittal letter signed by Vice Chancellor Wall and an enclosed postage-paid envelope. To be included in the sample pool undergraduate students had to be enrolled during 1995-96 for 3 quarters and could be foreign or domestic students. The first mailing was on April 21 and 3rd class discounting was used. The first mailing was followed by a post card on April 29 then a second mailing of survey, postage-paid envelope and follow-up letter signed by Celeste Hunziker on May 13. Replies were taken until June 8. In sum, 1,412 responses were received for a response rate of 56%. The UCD survey form was produced in house. It was of standard letter size, yellow in color, and six pages in length. Responses were indicated on customized scan forms that could be optically scanned. Each survey carried an identifying number to protect respondent anonymity while minimizing the number of follow-up mailed items. Summary of Analysis Respondents are first sorted into one of ten academic divisions based on degree program: Agriculture/Interdivisional; Agricultural Sciences; Agriculture Environmental Sciences; Agriculture Human Sciences; Biological Sciences; Engineering; Letters & Science (L&S) Interdivisional; L&S Humanities, Arts, and Cultural Studies; L&S Mathematics and Physical Sciences; or L&S Social Sciences. Correlation of responses to each of 79 questionnaire items with global satisfaction is then determined within division. The measure of association used is Kendall s Tau; a nonparametric measure of association appropriate for ordered variables, and the significance level of each correlation is recorded as displayed (Table 1). At this step all correlations of less than 0.20 are eliminated. A Kendall s Tau correlation of 0.20 reflects a moderately strong association and there is little reason to include weak correlations whose significance is more a function of division size (hypothesis test power) than meaningful psychometric agreement. A brief review of table elements shows both agreement and disagreement across divisions in the strength with which responses correlate. It is also clear that responses to ecosystem items, items eliciting ratings of both importance of topic and progress made in the topical area, favor progress made when looking for significant correlation with global satisfaction. The relative unimportance of importance of ratings for the ecosystem items is an interesting first result. This set of moderately strong correlations becomes the database for subsequent analysis. Next, mean correlation across divisions and number of times the correlation is significant are determined for each item and a ranked listing is produced where items are first sorted first by frequency of significant occurrences then by strength of mean correlation. The sorted item list appears as Table 2 and in graphic form as Figure 1. Based on the scatter-plot, items that are significant in only one or two divisions and are, on average, correlated with global satisfaction more modestly, <0.25, are excluded. These are arbitrary restrictions that serve to eliminate nearly half the initial list of respondent ratings. The criteria reflect the position that items associated with global satisfaction more rarely and more weakly should be eliminated first. Of course in practice, other issues should be considered when selecting or rejecting individual items. The resulting list of 45 items are flagged in Table 2 and appear as Table 3 where attention is called to the university division principally responsible for affecting the outcome measured by each item. The preponderance of items subject to academic purview is immediately apparent and illustrates the importance of reporting at the level of academic division where faculty impact and interests lie. For example, availability of faculty outside of class is clearly both an important characteristic and an Academic Affairs issue but is not correlated with global satisfaction in all divisions. Some characteristics, like overall social and cultural experience can be more affected by Student Services efforts, and others are the purview of both Academic and Student Affairs or of other institutional entities.

What are the 10 most important factors to undergraduate student satisfaction? 10 th off-campus atmosphere of ethnic, political and religious understanding 9 th library facilities 8 th quality of academic advising overall 7 th academic calendar for this campus 6 th availability of faculty outside of class 5 th faculty attitude toward students 4 th opportunities for involvement in campus activities 3 rd concern for student as an individual 2 nd overall social and cultural experience 1 st overall academic experience To summarize, the importance of University services, experiences, and programs, as measured by correlation with global satisfaction, can be used to form a shorter, more meaningful questionnaire. Limitations The analysis focused strictly on association There is no mention of whether or not students are satisfied overall or by division. There is no mention of level of satisfaction with the topic of any survey item. It does happen to be the case that UC Davis students do tend to be satisfied with their experience overall and with most aspects of that experience (see Appendix Table A), but the analysis reported here does not demand that responses to two items be similar in magnitude, only in rank order. In other words, if those who are most satisfied with their experience overall are also most satisfied with progress in learning to write effectively, the correlation would be high whether or not the most satisfied respondents are only neutral in their rating. Conversely, the absence of a significant correlation with global satisfaction does not mean that ratings are low on the second measure. The mean rating could be equally high with respondents ordered differently for some unknown reason. Either of these patterns would be exceptional cases as it is typically true that ratings are high on both items when the correlation is high (see Appendix A). Detailed Analysis of Responses to Overall Satisfaction and Quality of Academic Advising Items In recognition of the importance of academic satisfaction and in support of a discussion by the Undergraduate Advising Council, responses to two items, global satisfaction and satisfaction with academic advising will be discussed in detail. Overall Satisfaction Overall the mean rating of satisfaction by UC Davis undergraduates is 4.0 (Appendix B) on a scale from 1 being very dissatisfied to 5 being very satisfied. Expressed as percentages, 26% are very satisfied, 54% are satisfied, and 15% are neutral. Only one in 20 (4%) is dissatisfied and less than ½% are very dissatisfied. Collectively, these are high marks and are equivalent to those for overall academic experience reported for students at UC Berkeley, UC Santa Barbara, UC Santa Cruz, and UC San Diego (Table 9, The 1996 Undergraduate Experience Survey, UC Santa Cruz).

There is also little variation in overall satisfaction by academic division. All 10 divisions received ratings from 3.8 to 4.2 and the majority of divisional averages are 4.0. The lowest mark, 3.8 is by undeclared Letters and Science students. There is no significant difference in satisfaction by sex overall or within divisions (<.01). Only satisfaction by race/ethnicity shows a significant difference and that is attributed to Asian students being less inclined to rate overall satisfaction very satisfied and more likely to use neutral to indicate level of satisfaction. Asian students are no more likely to report being dissatisfied than any other group (Appendix Table B). In sum, undergraduates are uniformly well satisfied with the University. Quality of Academic Advising Quality of academic advising is not as highly rated as overall experience. The mean response is 3.4 and the distribution of responses found 2% very dissatisfied and 10% dissatisfied. Students are less likely to be very satisfied (9%) or satisfied (41%). When examined by academic division, the amount of variation found is similar to that for global satisfaction. Mean rating by academic division varies from 3.2 in Engineering to 3.8 in Agricultural Sciences, but most are 3.3 to 3.5. As is true for overall satisfaction with UC Davis, there are no difference by sex overall. However, there is a significant difference by sex within division with female Engineering students being more satisfied than males. Ratings of satisfaction with academic advising are similar to satisfaction with UC Davis overall in that Asian students are less likely to be very satisfied. There is one case where satisfaction with advising varied by race/ethnicity within an academic division, Social Sciences. Asian Social Science students are less likely to be very satisfied but are no more likely to be dissatisfied. Comparative data might be helpful. As stated earlier, this survey is part of a cooperative effort by five UC campuses where each campus used some items that were in common but supplemented the questionnaire with institutionally specific items. In the area of advising, one item, quality of academic advising by faculty was asked by the five participating schools and the rating by UC Davis students is significantly higher than the mean ratings for Berkeley, Santa Barbara, Santa Cruz and San Diego (see Appendix Table C). There is some evidence that sampling strategies and survey methodology varied by campus, and that this result should be noted with extreme caution, but UC Davis s high marks are at the very least encouraging.

Table 1: Significance Level of Kendall Tau (Item Correlation with Global Satisfaction) Academic Division* AGCW AGSC ENVR HHAD BISC ENGI LSCW HARC M&PS SOSC Importance Reading with greater speed & comprehension (I) 0.804 0.209 0.196 0.116 0.330 0.882 0.673 0.785 0.119 0.714 Developing writing skills (I) 0.916 0.305 0.369 0.285 0.055 0.278 0.105 0.557 0.475 0.500 Developing quantitative reasoning skills (I) 0.188 0.794 0.609 0.876 0.895 0.001 0.492 0.217 0.505 0.833 Developing up-to-date computer skills (I) 0.300 0.646 0.708 0.910 0.574 0.146 0.077 0.851 0.315 0.875 Developing research skills (I) 0.338 0.363 0.001 0.740 0.272 0.456 0.276 0.295 0.285 0.042 Developing effective speaking skills (I) 0.658 0.058 0.257 0.435 0.565 0.904 0.430 0.146 0.115 0.454 Developing problem-solving skills (I) 0.347 0.889 0.586 0.120 0.826 0.004 0.222 0.463 0.704 0.107 Becoming competent in field of study (I) 0.617 0.115 0.009 0.021 0.443 0.132 0.424 0.328 0.428 0.922 Developing an appreciation for art, music & literature (I) 0.307 0.173 0.016 0.840 0.104 0.572 0.353 0.429 0.661 0.515 Developing scientific knowledge and skills (I) 0.399 0.329 0.168 0.445 0.641 0.041 0.490 0.210 0.056 0.825 Acquiring knowledge and skills for employment (I) 0.024 0.241 0.157 0.862 0.055 0.473 0.578 0.064 0.869 0.050 Acquiring knowledge and skills for graduate school (I) 0.597 0.111 0.031 0.319 0.819 0.114 0.047 0.273 0.178 0.920 Experience through internship (I) 0.314 0.010 0.064 0.687 0.665 0.196 0.292 0.525 0.624 0.214 Developing more effective interpersonal skills (I) 0.765 0.640 0.055 0.089 0.643 0.655 0.377 0.563 0.114 0.872 Acquiring a well-rounded general education (I) 0.888 0.032 0.481 0.003 0.640 0.138 0.515 0.412 0.729 0.153 Generating original ideas and projects (I) 0.893 0.762 0.151 0.551 0.515 0.008 0.071 0.438 0.889 0.306 Reasoning objectively about attitudes and values (I) 0.900 0.350 0.634 0.265 0.530 0.019 0.042 0.737 0.853 0.272 Developing leadership skills (I) 0.451 0.202 0.029 0.653 0.321 0.987 0.158 0.348 0.473 0.390 Exercising rights and responsibilities of citizenship (I) 0.357 0.053 0.122 0.020 0.197 0.021 0.035 0.977 0.295 0.105 Increasing awareness of governmental issues (I) 0.143 0.015 0.056 0.056 0.373 0.039 0.019 0.493 0.875 0.028 Developing uses of leisure time (I) 0.784 0.054 0.770 0.354 0.295 0.039 0.420 0.129 0.338 0.191 Applying principles for improved health (I) 0.356 0.029 0.598 0.069 0.076 0.073 0.094 0.378 0.452 0.096 Participating in community service activities (I) 0.670 0.137 0.130 0.006 0.583 0.049 0.456 0.212 0.818 0.347 Progress Reading with greater speed & comprehension (P) 0.473 0.815 0.010 0.899 0.016 0.006 0.030 0.070 0.212 0.004 Developing writing skills (P) 0.029 0.000 0.045 0.133 0.009 0.009 0.196 0.947 0.018 0.000 Developing quantitative reasoning skills (P) 0.084 0.096 0.009 0.607 0.002 0.001 0.075 0.878 0.211 0.001 Developing up-to-date computer skills (P) 0.252 0.350 0.631 0.086 0.000 0.015 0.560 0.769 0.000 0.617 Developing research skills (P) 0.466 0.016 0.003 0.260 0.003 0.190 0.157 0.452 0.008 0.000 Developing effective speaking skills (P) 0.807 0.001 0.015 0.715 0.003 0.029 0.696 0.441 0.185 0.070 Developing problem-solving skills (P) 0.299 0.000 0.035 0.385 0.000 0.000 0.788 0.367 0.001 0.089 Becoming competent in field of study (P) 0.570 0.057 0.063 0.147 0.000 0.000 0.714 0.612 0.000 0.000 Developing an appreciation for art, music & literature (P) 0.544 0.011 0.005 0.716 0.015 0.001 0.249 0.004 0.192 0.001 Developing scientific knowledge and skills (P) 0.069 0.009 0.099 0.192 0.000 0.000 0.093 0.146 0.000 0.147 Acquiring knowledge and skills for employment (P) 0.977 0.000 0.000 0.023 0.002 0.000 0.018 0.150 0.000 0.000 Acquiring knowledge and skills for graduate school (P) 0.475 0.000 0.001 0.002 0.000 0.000 0.062 0.749 0.000 0.000 Experience through internship (P) 0.278 0.005 0.371 0.531 0.004 0.003 0.160 0.166 0.445 0.001 Developing more effective interpersonal skills (P) 0.017 0.002 0.040 0.031 0.000 0.000 0.890 0.078 0.165 0.000 Acquiring a well-rounded general education (P) 0.005 0.000 0.468 0.002 0.011 0.132 0.897 0.000 0.000 0.000 Generating original ideas and projects (P) 0.185 0.000 0.154 0.183 0.004 0.005 0.598 0.061 0.014 0.000 Reasoning objectively about attitudes and values (P) 0.588 0.012 0.378 0.060 0.020 0.003 0.349 0.031 0.002 0.000 Developing leadership skills (P) 0.080 0.001 0.001 0.055 0.006 0.002 0.382 0.314 0.000 0.000 Exercising rights and responsibilities of citizenship (P) 0.051 0.002 0.061 0.000 0.000 0.003 0.096 0.055 0.046 0.000 Increasing awareness of governmental issues (P) 0.768 0.226 0.534 0.199 0.005 0.030 0.047 0.245 0.134 0.000 Developing uses of leisure time (P) 0.052 0.000 0.004 0.010 0.000 0.000 0.492 0.079 0.080 0.000 Applying principles for improved health (P) 0.020 0.003 0.006 0.015 0.000 0.003 0.962 0.037 0.198 0.000 Participating in community service activities (P) 0.006 0.020 0.051 0.030 0.031 0.028 0.739 0.424 0.113 0.007 Faculty instruction in major field 0.000 0.000 0.014 0.000 0.000 0.000 1.000 0.006 0.000 0.000

Table 1: Significance Level of Kendall Tau (Item Correlation with Global Satisfaction) Academic Division* AGCW AGSC ENVR HHAD BISC ENGI LSCW HARC M&PS SOSC TA/graduate student instruction in major field 0.022 0.014 0.345 0.001 0.000 0.038 0.807 0.076 0.000 0.000 Availability of faculty outside of class 0.002 0.000 0.009 0.000 0.000 0.000 0.919 0.003 0.000 0.000 Faculty attitude toward students 0.000 0.001 0.000 0.000 0.000 0.000 0.189 0.003 0.000 0.000 Course content in major 0.199 0.066 0.000 0.000 0.000 0.000 0.132 0.003 0.005 0.000 Testing/grading procedures 0.025 0.002 0.463 0.000 0.000 0.000 0.049 0.000 0.001 0.000 Variety of courses offered 0.000 0.014 0.012 0.000 0.000 0.000 0.000 0.000 0.000 0.000 Class size 0.012 0.000 0.039 0.301 0.000 0.000 0.003 0.000 0.004 0.000 Availability of required courses 0.210 0.016 0.360 0.001 0.000 0.001 0.077 0.000 0.003 0.000 Availability of courses in general 0.061 0.065 0.547 0.000 0.000 0.000 0.051 0.000 0.001 0.000 Flexibility of degree requirements 0.693 0.003 0.074 0.000 0.000 0.001 0.176 0.238 0.001 0.000 Opportunities for independent study and research 0.002 0.015 0.006 0.003 0.000 0.009 0.173 0.343 0.034 0.000 Practical work experiences in your field of study 0.088 0.000 0.028 0.000 0.000 0.033 0.373 0.849 0.100 0.000 Quality of academic advising by faculty 0.134 0.000 0.000 0.000 0.000 0.000 0.234 0.002 0.110 0.002 Quality of academic advising by staff 0.182 0.001 0.000 0.004 0.000 0.000 0.309 0.009 0.014 0.000 Quality of academic advising by academic peer advisors 0.175 0.155 0.531 0.022 0.000 0.002 0.498 0.033 0.239 0.068 Quality of academic advising by dean's office 0.112 0.000 0.003 0.256 0.000 0.001 0.166 0.130 0.330 0.010 Quality of academic advising overall 0.024 0.000 0.000 0.000 0.000 0.000 0.064 0.001 0.006 0.000 Classroom facilities 0.004 0.000 0.094 0.100 0.000 0.000 0.059 0.003 0.000 0.000 Laboratory facilities 0.001 0.000 0.279 0.014 0.000 0.000 0.022 0.000 0.009 0.001 Computer facilities 0.012 0.017 0.931 0.004 0.000 0.000 0.259 0.096 0.011 0.000 Computer services 0.074 0.091 0.343 0.039 0.000 0.001 0.351 0.079 0.021 0.000 Library facilities 0.000 0.006 0.017 0.000 0.000 0.000 0.006 0.001 0.001 0.000 Library services 0.002 0.043 0.001 0.009 0.000 0.032 0.027 0.000 0.007 0.000 Campus catalog 0.143 0.000 0.032 0.000 0.000 0.001 0.003 0.000 0.000 0.000 Academic calendar for this campus 0.212 0.004 0.552 0.000 0.000 0.000 0.004 0.000 0.000 0.000 General registration procedures 0.168 0.000 0.034 0.017 0.000 0.008 0.013 0.002 0.000 0.000 On-campus atmosphere of ethnic, political and religious understan 0.075 0.000 0.036 0.003 0.000 0.000 0.000 0.000 0.000 0.000 Off-campus atmosphere of ethnic, political and religious understan 0.671 0.000 0.042 0.000 0.000 0.008 0.000 0.000 0.000 0.000 Concern for student as an individual 0.000 0.000 0.001 0.000 0.000 0.000 0.000 0.000 0.000 0.000 Opportunities for involvement in campus activities 0.005 0.000 0.000 0.000 0.000 0.000 0.345 0.000 0.000 0.000 Overall academic experience 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 Overall social and cultural experience 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 * Academic Division: Agriculture/Interdivisional; Agricultural Sciences; Agriculture Environmental Sciences; Agriculture Human Sciences; Biological Sciences; Engineering; Letters & Science (L&S) Interdivisional; L&S Humanities, Arts, and Cultural Studies; L&S Mathematics and Physical Sciences; or L&S Social Sciences

Table 2: Kendall Tau of Items Correlating with Global Satisfaction Within Division Academic Division* Top AGCW AGSC ENVR HHAD BISC ENGI LSCW HARC M&PS SOSC Count Mean 45 1 Overall academic experience 0.488 0.585 0.499 0.530 0.489 0.528 0.401 0.521 0.583 0.590 10 0.521 X 2 Overall social and cultural experience 0.453 0.572 0.478 0.394 0.417 0.350 0.609 0.435 0.449 0.436 10 0.459 X 3 Concern for student as an individual 0.412 0.419 0.292 0.227 0.268 0.303 0.386 0.410 0.263 0.325 10 0.331 X 4 Opportunities for involvement in campus activities 0.269 0.439 0.307 0.275 0.306 0.384 0.372 0.333 0.291 9 0.331 X 5 Faculty attitude toward students 0.387 0.325 0.330 0.287 0.237 0.378 0.205 0.247 0.275 9 0.297 X 6 Availability of faculty outside of class 0.300 0.436 0.225 0.227 0.242 0.360 0.310 0.236 8 0.292 X 7 Academic calendar for this campus 0.265 0.315 0.267 0.245 0.383 0.310 0.332 0.218 8 0.292 X 8 Quality of academic advising overall 0.404 0.352 0.253 0.236 0.294 0.228 0.233 0.248 8 0.281 X 9 Library facilities 0.357 0.268 0.248 0.214 0.367 0.204 0.267 0.216 8 0.268 X 10 Off-campus atmosphere of ethnic, political and religious understanding 0.378 0.236 0.237 0.528 0.352 0.346 0.329 7 0.344 X 11 Variety of courses offered 0.325 0.295 0.311 0.477 0.274 0.369 0.235 7 0.327 X 12 On-campus atmosphere of ethnic, political and religious understanding 0.409 0.263 0.249 0.459 0.292 0.263 0.329 7 0.323 X 13 Faculty instruction in major field 0.434 0.300 0.330 0.245 0.206 0.353 0.273 7 0.306 X 14 Course content in major 0.270 0.348 0.235 0.275 0.228 0.219 0.265 7 0.263 X 15 Classroom facilities 0.242 0.369 0.224 0.230 0.211 0.260 0.245 7 0.254 X 16 Campus catalog 0.290 0.286 0.200 0.384 0.281 0.353 6 0.299 X 17 Laboratory facilities 0.317 0.388 0.257 0.330 0.229 0.200 6 0.287 X 18 Quality of academic advising by faculty 0.375 0.353 0.249 0.215 0.243 0.255 6 0.282 X 19 Testing/grading procedures 0.286 0.295 0.241 0.317 0.257 0.210 6 0.268 X 20 Acquiring knowledge and skills for graduate school (P) 0.300 0.241 0.201 0.221 0.318 0.231 6 0.252 X 21 Availability of required courses 0.223 0.211 0.213 0.335 0.228 0.230 6 0.240 X 22 Class size 0.405 0.303 0.339 0.279 0.206 5 0.306 X 23 Opportunities for independent study and research 0.410 0.234 0.246 0.215 0.236 5 0.268 X 24 Quality of academic advising by staff 0.322 0.367 0.201 0.206 0.229 5 0.265 X 25 Availability of courses in general 0.257 0.256 0.325 0.234 0.233 5 0.261 X 26 Flexibility of degree requirements 0.278 0.240 0.263 0.212 0.229 5 0.244 X 27 Quality of academic advising by dean's office 0.411 0.337 0.215 0.271 4 0.309 X 28 Acquiring a well-rounded general education (P) 0.309 0.255 0.301 0.254 4 0.280 X 29 Acquiring knowledge and skills for employment (P) 0.300 0.259 0.228 0.313 4 0.275 X 30 Library services 0.297 0.274 0.276 0.225 4 0.268 X 31 Practical work experiences in your field of study 0.390 0.320 0.204 3 0.305 X 32 Developing scientific knowledge and skills (P) 0.227 0.234 0.371 3 0.277 X 33 General registration procedures 0.356 0.221 0.245 3 0.274 X 34 Developing leadership skills (P) 0.281 0.228 0.285 3 0.265 X 35 Developing uses of leisure time (P) 0.290 0.243 0.209 3 0.247 X 36 Becoming competent in field of study (P) 0.228 0.261 0.202 3 0.230 X 37 Developing problem-solving skills (P) 0.256 0.215 0.211 3 0.227 X 38 Developing research skills (P) 0.247 0.202 0.205 3 0.218 X 39 Applying principles for improved health (P) 0.286 0.228 2 0.257 X 40 TA/graduate student instruction in major field 0.209 0.287 2 0.248 X 41 Developing more effective interpersonal skills (P) 0.272 0.206 2 0.239 42 Developing an appreciation for art, music & literature (P) 0.227 0.216 2 0.222 43 Computer facilities 0.211 0.230 2 0.221 44 Quality of academic advising by academic peer advisors 0.205 0.217 2 0.211 45 Developing quantitative reasoning skills (P) 0.207 0.201 2 0.204 46 Generating original ideas and projects (P) 0.363 1 0.363 X 47 Developing writing skills (P) 0.337 1 0.337 X 48 Developing up-to-date computer skills (P) 0.277 1 0.277 X 49 Developing effective speaking skills (P) 0.274 1 0.274 X 50 Exercising rights and responsibilities of citizenship (P) 0.250 1 0.250 X 51 Developing research skills (I) 0.243 1 0.243

Table 2: Kendall Tau of Items Correlating with Global Satisfaction Within Division Academic Division* Top AGCW AGSC ENVR HHAD BISC ENGI LSCW HARC M&PS SOSC Count Mean 45 52 Participating in community service activities (P) 0.238 1 0.238 53 Experience through internship (P) 0.231 1 0.231 54 Reasoning objectively about attitudes and values (P) 0.228 1 0.228 55 Computer services 0.214 1 0.214 56 Experience through internship (I) 0.201 1 0.201 57 Becoming competent in field of study (I) 0.200 1 0.200 58 Increasing awareness of governmental issues (P) 59 Reading with greater speed & comprehension (I) 60 Developing writing skills (I) 61 Developing quantitative reasoning skills (I) 62 Developing up-to-date computer skills (I) 63 Developing effective speaking skills (I) 64 Developing problem-solving skills (I) 65 Developing an appreciation for art, music & literature (I) 66 Developing scientific knowledge and skills (I) 67 Acquiring knowledge and skills for employment (I) 68 Acquiring knowledge and skills for graduate school (I) 69 Developing more effective interpersonal skills (I) 70 Acquiring a well-rounded general education (I) 71 Generating original ideas and projects (I) 72 Reasoning objectively about attitudes and values (I) 73 Developing leadership skills (I) 74 Exercising rights and responsibilities of citizenship (I) 75 Increasing awareness of governmental issues (I) 76 Developing uses of leisure time (I) 77 Applying principles for improved health (I) 78 Participating in community service activities (I) 79 Reading with greater speed & comprehension (P) Mean 0.352 0.338 0.289 0.276 0.256 0.268 0.433 0.294 0.289 0.266 0.275 Standard Deviation 0.080 0.085 0.080 0.072 0.072 0.070 0.084 0.080 0.078 0.083 0.057 Only significant correlations of 0.20 or higher are included. A 0.20 Kendall Tau correlation is moderately strong. * Academic Division: Agriculture/Interdivisional; Agricultural Sciences; Agriculture Environmental Sciences; Agriculture Human Sciences; Biological Sciences; Engineering; Letters & Science (L&S) Interdivisional; L&S Humanities, Arts, and Cultural Studies; L&S Mathematics and Physical Sciences; or L&S Social Sciences

Figure 1: Relationship of Item Correlation with Global Satisfaction to Number of Times Item was Significant Across Disciplines 10 9 Number of Times Significant 8 7 6 5 4 3 2 Items Retained Items Eliminated 1 0.20 0.25 0.30 0.35 0.40 Average Correlation Across Divisions

Table 3: Reduced Item Set with Campus Responsibility Identified -- Items Most Clearly Associated with Global Satisfaction UC Davis Responsibility Academic S.A. Clearly Administrative Questionnaire Items Purview Purview Both Support Other 1 Overall academic experience x 2 Overall social and cultural experience x 3 Concern for student as an individual x 4 Opportunities for involvement in campus activities x 5 Faculty attitude toward students x 6 Availability of faculty outside of class x 7 Academic calendar for this campus x 8 Quality of academic advising overall x 9 Library facilities x 10 Off-campus atmosphere of ethnic, political and religious understanding x 11 Variety of courses offered x 12 On-campus atmosphere of ethnic, political and religious understanding x 13 Faculty instruction in major field x 14 Course content in major x 15 Classroom facilities x 16 Campus catalog x 17 Laboratory facilities x 18 Quality of academic advising by faculty x 19 Testing/grading procedures x 20 Acquiring knowledge and skills for graduate school (P) x 21 Availability of required courses x 22 Class size x 23 Opportunities for independent study and research x 24 Quality of academic advising by staff x 25 Availability of courses in general x 26 Flexibility of degree requirements x 27 Quality of academic advising by dean's office x 28 Acquiring a well-rounded general education (P) x 29 Acquiring knowledge and skills for employment (P) x 30 Library services x 31 Practical work experiences in your field of study x 32 Developing scientific knowledge and skills (P) x 33 General registration procedures x 34 Developing leadership skills (P) x 35 Developing uses of leisure time (P) x 36 Becoming competent in field of study (P) x 37 Developing problem-solving skills (P) x 38 Developing research skills (P) x 39 Applying principles for improved health (P) x 40 TA/graduate student instruction in major field x 41 Generating original ideas and projects (P) x 42 Developing writing skills (P) x 43 Developing up-to-date computer skills (P) x 44 Developing effective speaking skills (P) x 45 Exercising rights and responsibilities of citizenship (P) x (P) identifies items reflecting self-reported progress made toward achieving goal.

Appendix Table A: UC Davis Undergraduate Experiences Survey -- Goals, Progress, and Satisfaction Frequencies Percentages Very Very None Little Moderate Great Great n = None Little Moderate Great Great Mean Importance of Goal Reading with greater speed & comprehension (I) 35 81 285 457 513 1371 3% 6% 21% 33% 37% 4.0 Developing writing skills (I) 8 25 155 428 755 1371 1% 2% 11% 31% 55% 4.4 Developing quantitative reasoning skills (I) 12 38 200 431 678 1359 1% 3% 15% 32% 50% 4.3 Developing up-to-date computer skills (I) 15 58 242 402 656 1373 1% 4% 18% 29% 48% 4.2 Developing research skills (I) 17 75 298 454 526 1370 1% 5% 22% 33% 38% 4.0 Developing effective speaking skills (I) 17 61 206 421 658 1363 1% 4% 15% 31% 48% 4.2 Developing problem-solving skills (I) 9 27 185 435 710 1366 1% 2% 14% 32% 52% 4.3 Becoming competent in field of study (I) 6 10 79 276 999 1370 0% 1% 6% 20% 73% 4.6 Developing an appreciation for art, music & literature (I) 80 181 378 352 371 1362 6% 13% 28% 26% 27% 3.6 Developing scientific knowledge and skills (I) 43 92 244 383 607 1369 3% 7% 18% 28% 44% 4.0 Acquiring knowledge and skills for employment (I) 8 16 77 210 1057 1368 1% 1% 6% 15% 77% 4.7 Acquiring knowledge and skills for graduate school (I) 50 66 150 310 791 1367 4% 5% 11% 23% 58% 4.3 Experience through internship (I) 40 47 151 413 711 1362 3% 3% 11% 30% 52% 4.3 Developing more effective interpersonal skills (I) 20 44 237 448 611 1360 1% 3% 17% 33% 45% 4.2 Acquiring a well-rounded general education (I) 45 95 285 439 515 1379 3% 7% 21% 32% 37% 3.9 Generating original ideas and projects (I) 11 53 297 489 518 1368 1% 4% 22% 36% 38% 4.1 Reasoning objectively about attitudes and values (I) 17 41 276 496 529 1359 1% 3% 20% 36% 39% 4.1 Developing leadership skills (I) 26 71 238 458 571 1364 2% 5% 17% 34% 42% 4.1 Exercising rights and responsibilities of citizenship (I) 48 118 338 418 441 1363 4% 9% 25% 31% 32% 3.8 Increasing awareness of governmental issues (I) 55 135 364 425 382 1361 4% 10% 27% 31% 28% 3.7 Developing uses of leisure time (I) 45 76 279 431 529 1360 3% 6% 21% 32% 39% 4.0 Applying principles for improved health (I) 31 60 237 447 591 1366 2% 4% 17% 33% 43% 4.1 Participating in community service activities (I) 92 194 425 362 296 1369 7% 14% 31% 26% 22% 3.4 Progress Reading with greater speed & comprehension (P) 111 309 651 272 57 1400 8% 22% 47% 19% 4% 2.9 Developing writing skills (P) 37 199 591 453 124 1404 3% 14% 42% 32% 9% 3.3 Developing quantitative reasoning skills (P) 37 182 549 479 140 1387 3% 13% 40% 35% 10% 3.4 Developing up-to-date computer skills (P) 134 317 452 342 157 1402 10% 23% 32% 24% 11% 3.1 Developing research skills (P) 110 281 476 352 179 1398 8% 20% 34% 25% 13% 3.1 Developing effective speaking skills (P) 184 344 506 259 98 1391 13% 25% 36% 19% 7% 2.8 Developing problem-solving skills (P) 39 181 571 438 164 1393 3% 13% 41% 31% 12% 3.4 Becoming competent in field of study (P) 55 164 487 477 210 1393 4% 12% 35% 34% 15% 3.4 Developing an appreciation for art, music & literature (P) 198 357 432 273 128 1388 14% 26% 31% 20% 9% 2.8 Developing scientific knowledge and skills (P) 103 213 463 410 208 1397 7% 15% 33% 29% 15% 3.3 Acquiring knowledge and skills for employment (P) 94 311 546 330 112 1393 7% 22% 39% 24% 8% 3.0 Acquiring knowledge and skills for graduate school (P) 135 262 516 354 123 1390 10% 19% 37% 25% 9% 3.0 Experience through internship (P) 573 187 253 202 175 1390 41% 13% 18% 15% 13% 2.4 Developing more effective interpersonal skills (P) 99 255 501 388 147 1390 7% 18% 36% 28% 11% 3.2 Acquiring a well-rounded general education (P) 56 215 532 396 198 1397 4% 15% 38% 28% 14% 3.3 Very Very None Little Moderate Great Great None Little Moderate Great Great Mean Generating original ideas and projects (P) 88 350 589 283 87 1397 6% 25% 42% 20% 6% 3.0 Reasoning objectively about attitudes and values (P) 71 159 542 449 167 1388 5% 11% 39% 32% 12% 3.3 Developing leadership skills (P) 174 317 479 283 137 1390 13% 23% 34% 20% 10% 2.9 Exercising rights and responsibilities of citizenship (P) 136 250 525 307 169 1387 10% 18% 38% 22% 12% 3.1 Increasing awareness of governmental issues (P) 200 421 437 220 113 1391 14% 30% 31% 16% 8% 2.7

Appendix Table A: UC Davis Undergraduate Experiences Survey -- Goals, Progress, and Satisfaction Frequencies Percentages Developing uses of leisure time (P) 160 311 430 304 183 1388 12% 22% 31% 22% 13% 3.0 Applying principles for improved health (P) 124 279 526 318 151 1398 9% 20% 38% 23% 11% 3.1 Participating in community service activities (P) 363 400 362 174 101 1400 26% 29% 26% 12% 7% 2.5 Frequencies Percentages Very Very Very Very DissatisfiedDissatisfied Neutral Satisfied Satisfied DissatisfiedDissatisfied Neutral Satisfied Satisfied Mean Faculty instruction in major field 16 77 246 694 295 1328 1% 6% 19% 52% 22% 3.9 TA/graduate student instruction in major field 34 126 386 585 175 1306 3% 10% 30% 45% 13% 3.6 Availability of faculty outside of class 14 94 301 654 290 1353 1% 7% 22% 48% 21% 3.8 Faculty attitude toward students 27 87 335 671 277 1397 2% 6% 24% 48% 20% 3.8 Course content in major 19 83 262 661 323 1348 1% 6% 19% 49% 24% 3.9 Testing/grading procedures 35 117 446 661 145 1404 2% 8% 32% 47% 10% 3.5 Variety of courses offered 26 138 255 598 388 1405 2% 10% 18% 43% 28% 3.8 Class size 89 330 429 460 97 1405 6% 23% 31% 33% 7% 3.1 Availability of required courses 88 300 397 443 136 1364 6% 22% 29% 32% 10% 3.2 Availability of courses in general 71 254 464 489 122 1400 5% 18% 33% 35% 9% 3.2 Flexibility of degree requirements 53 156 501 436 113 1259 4% 12% 40% 35% 9% 3.3 Opportunities for independent study and research 26 133 342 305 148 954 3% 14% 36% 32% 16% 3.4 Practical work experiences in your field of study 55 193 323 293 146 1010 5% 19% 32% 29% 14% 3.3 Quality of academic advising by faculty 46 129 375 410 183 1143 4% 11% 33% 36% 16% 3.5 Quality of academic advising by staff 38 123 369 442 166 1138 3% 11% 32% 39% 15% 3.5 Quality of academic advising by academic peer advisors 29 92 299 296 138 854 3% 11% 35% 35% 16% 3.5 Quality of academic advising by dean's office 55 104 264 182 91 696 8% 15% 38% 26% 13% 3.2 Quality of academic advising overall 29 118 469 499 115 1230 2% 10% 38% 41% 9% 3.4 Classroom facilities 22 130 475 655 111 1393 2% 9% 34% 47% 8% 3.5 Laboratory facilities 22 113 307 530 134 1106 2% 10% 28% 48% 12% 3.6 Computer facilities 44 145 248 569 265 1271 3% 11% 20% 45% 21% 3.7 Computer services 30 100 295 604 256 1285 2% 8% 23% 47% 20% 3.7 Library facilities 4 38 149 686 507 1384 0% 3% 11% 50% 37% 4.2 Library services 16 52 218 646 428 1360 1% 4% 16% 48% 31% 4.0 Campus catalog 7 35 212 757 382 1393 1% 3% 15% 54% 27% 4.1 Academic calendar for this campus 40 94 321 635 270 1360 3% 7% 24% 47% 20% 3.7 General registration procedures 48 147 353 634 222 1404 3% 10% 25% 45% 16% 3.6 Very Very Very Very DissatisfiedDissatisfied Neutral Satisfied Satisfied DissatisfiedDissatisfied Neutral Satisfied Satisfied Mean On-campus atmosphere of ethnic, political and religious underst 48 119 380 553 222 1322 4% 9% 29% 42% 17% 3.6 Off-campus atmosphere of ethnic, political and religious underst 50 135 444 440 147 1216 4% 11% 37% 36% 12% 3.4 Concern for student as an individual 96 245 553 380 88 1362 7% 18% 41% 28% 6% 3.1 Opportunities for involvement in campus activities 14 89 374 551 301 1329 1% 7% 28% 41% 23% 3.8 Overall academic experience 10 77 302 749 266 1404 1% 5% 22% 53% 19% 3.8 Overall social and cultural experience 21 110 396 592 264 1383 2% 8% 29% 43% 19% 3.7 UC Davis in general 7 53 208 764 372 1404 0% 4% 15% 54% 26% 4.0

Appendix Table B: Satisfaction with UC Davis Overall and with Academic Advising Overall Frequencies Percentages Very Very Very Very Dissatisfied Dissatisfied Neutral Satisfied Satisfied n = Dissatisfied Dissatisfied Neutral Satisfied Satisfied Mean Quality of academic advising overall 29 118 469 499 115 1230 2% 10% 38% 41% 9% 3.4 Agriculture/Interdivisional 1 8 19 22 5 55 2% 15% 35% 40% 9% 3.4 Agricultural Sciences 1 4 13 31 10 59 2% 7% 22% 53% 17% 3.8 Agriculture Environmental Sciences 0 10 31 28 3 72 0% 14% 43% 39% 4% 3.3 Agriculture Human Sciences 1 8 46 57 10 122 1% 7% 38% 47% 8% 3.5 Biological Sciences 3 20 122 116 34 295 1% 7% 41% 39% 12% 3.5 Engineering 8 19 62 45 12 146 5% 13% 42% 31% 8% 3.2 Letters & Science (L&S) Interdivisional 0 1 9 13 1 24 0% 4% 38% 54% 4% 3.6 L&S Humanities, Arts, and Cultural Studies 1 17 38 38 11 105 1% 16% 36% 36% 10% 3.4 L&S Mathematics and Physical Sciences 2 5 33 37 0 77 3% 6% 43% 48% 0% 3.4 L&S Social Sciences 12 26 96 108 28 270 4% 10% 36% 40% 10% 3.4 Differences by sex? No Differences by sex within division? Yes, Engineering. Female 1 10 17 5 33 3% 0% 30% 52% 15% 3.8 Male 7 19 52 28 7 113 6% 17% 46% 25% 6% 3.1 Differences by race/ethnicity? Yes. Unknown 3 12 14 24 6 59 5% 20% 24% 41% 10% 3.3 Asian 9 36 181 163 25 414 2% 9% 44% 39% 6% 3.4 Underrepresented Minority 6 12 67 72 19 176 3% 7% 38% 41% 11% 3.5 White 11 58 207 240 65 581 2% 10% 36% 41% 11% 3.5 Differences by race/ethnicity with division? Yes, Social Sciences. Unknown too few Asian 2 6 25 32 3 68 3% 9% 37% 47% 4% 3.4 Underrepresented Minority 4 3 23 20 7 57 7% 5% 40% 35% 12% 3.4 White 3 15 48 54 18 138 2% 11% 35% 39% 13% 3.5 UC Davis in general 7 53 208 764 372 1404 0% 4% 15% 54% 26% 4.0 Agriculture/Interdivisional 0 2 15 34 17 68 0% 3% 22% 50% 25% 4.0 Agricultural Sciences 2 2 9 31 21 65 3% 3% 14% 48% 32% 4.0 Agriculture Environmental Sciences 0 2 6 45 23 76 0% 3% 8% 59% 30% 4.2 Agriculture Human Sciences 1 1 13 74 39 128 1% 1% 10% 58% 30% 4.2 Biological Sciences 1 11 46 193 71 322 0% 3% 14% 60% 22% 4.0 Engineering 1 11 27 89 42 170 1% 6% 16% 52% 25% 3.9 Letters & Science (L&S) Interdivisional 0 1 10 14 7 32 0% 3% 31% 44% 22% 3.8 L&S Humanities, Arts, and Cultural Studies 0 4 21 62 32 119 0% 3% 18% 52% 27% 4.0 L&S Mathematics and Physical Sciences 1 3 18 49 28 99 1% 3% 18% 49% 28% 4.0 L&S Social Sciences 1 16 42 172 89 320 0% 5% 13% 54% 28% 4.0 Differences by sex? No Differences by sex within division? No. Differences by race/ethnicity? Yes

Appendix Table C: External Comparison of Satisfaction with Faculty Advising Frequencies Percentages Very Very Very Very Dissatisfied Dissatisfied Neutral Satisfied Satisfied n = Dissatisfied Dissatisfied Neutral Satisfied Satisfied Mean Quality of academic advising by faculty 46 129 375 410 183 1143 4% 11% 33% 36% 16% 3.5 UCB 3.2 UCSC 3.2 UCSD 2.8 UCSB 3.2 Agriculture/Interdivisional 1 7 19 20 5 52 2% 13% 37% 38% 10% 3.4 Agricultural Sciences 3 3 11 26 15 58 5% 5% 19% 45% 26% 3.8 Agriculture Environmental Sciences 0 9 26 19 12 66 0% 14% 39% 29% 18% 3.5 Agriculture Human Sciences 4 12 39 43 17 115 3% 10% 34% 37% 15% 3.5 Biological Sciences 8 23 91 107 44 273 3% 8% 33% 39% 16% 3.6 Engineering 6 25 40 44 15 130 5% 19% 31% 34% 12% 3.3 Letters & Science (L&S) Interdivisional 0 2 9 9 3 23 0% 9% 39% 39% 13% 3.6 L&S Humanities, Arts, and Cultural Studies 9 15 30 29 19 102 9% 15% 29% 28% 19% 3.3 Simple Average of UCSC Arts and Humanities 3.4 L&S Mathematics and Physical Sciences 3 9 28 28 6 74 4% 12% 38% 38% 8% 3.3 UCSC Natural Sciences 2.9 L&S Social Sciences 12 24 82 81 46 245 5% 10% 33% 33% 19% 3.5 UCSC Social Sciences 3.2