A Brief Analysis of Graduate Student Exit Interviews, California State University, Chico December 1995-December 2004 William M. Loker Interim Associate Dean School of Graduate, International and Interdisciplinary Studies California State University, Chico June 2005
Introduction The School of Graduate, International and Interdisciplinary Studies routinely collects exit interview data from graduating masters students in the form of a brief survey (see attached). Students receive this survey when applying for graduation and many take the time to fill it out and return it with their graduation application. The same survey has been administered since 1990. While attempts have been made to tally responses to the survey, there has been no systematic attempt at further analysis of results. Unfortunately, it appears that the surveys from 1990-94 were tallied and discarded. Therefore, the current analysis covers the period December 1995-December 2004. There are a total of 885 surveys from this time period. The procedure for analyzing the surveys has been as follows. The surveys were reviewed and responses coded by a student assistant under the supervision of the author during Fall 04. This process included reducing the open-ended responses (to questions 20-22) to a manageable number of responses and coding these for data input. The responses were then entered into an Excel spreadsheet by the student assistant. Data were reviewed and cleaned up and prepared for analysis. For the purposes of analysis, the Excel data was exported to SPSS and subsequent analysis has been carried using that program. What follows is a preliminary attempt to summarize the response to the survey, looking at the graduates as a whole, then comparing the responses of individual programs to the entire sample. It should always be borne in mind, when reviewing these results, that we are examining self-reported data from students. Hence, we are examining perceptions of programs, not the actual conditions prevailing in these programs. Having said that, clearly the perceptions of our graduates regarding their experiences is important and should be considered, among other data, in assessing the strengths and weaknesses of the graduate experience at CSU, Chico. Overall Summary Table 1 lists the number of masters graduates by program for the period under consideration. It is notable that more than half of our masters graduates received degrees in five programs: Computer Science, various Business degrees, various Psychology degrees, Speech Pathology and PE/Kinesiology. An additional five masters programs provides another 25% of our graduates (various Education masters programs, various Interdisciplinary Studies programs, various Communications graduate degrees, various Political Science degrees and Social Work). It is notable that Social Work appears among the top-degree granting graduate programs, as it only came into existence in AY 1999-2000, graduating its first cohort of students in????. Clearly it is a large and growing program, much in demand. The remaining 15 degree-granting programs account for less than 25% of our graduate degrees. Turning to results of the exit surveys, there are a series of Tables (2-6) that summarize the primary questions directed at the quality of the graduate experience. Several of these questions are ordinal questions, using a 4-point ranked scale, where 1 is excellent and 4 is poor. The one exception is Question 6: Is the focus of your masters program: 1. Too Specialized, 2. About Right, 3. Too Broad. Questions 20-22 are open-ended questions and will be discussed in more detail below.
Table 1. Masters Programs, Number of Exit Surveys, 12/95-12/04 Frequency Percent Valid Percent Cumulative Percent Valid Computer Science 109 12.3 12.4 12.4 Business (All MA/Ss) a 97 11.0 11.0 23.4 Psychology (All MA/Ss) b 93 10.5 10.6 34.0 Speech Pathology 82 9.3 9.3 43.3 PE/Kinesiology 81 9.2 9.2 52.5 Education (all MAs) c 55 6.2 6.3 58.8 All Interdisciplinary MA/Ss d 46 5.2 5.2 64.0 Communications (All MAs) e 38 4.3 4.3 68.3 Political Sciences (All f MAs) 35 4.0 4.0 72.3 Social Work 30 3.4 3.4 75.7 English (All MAs) g 29 3.3 3.3 79.0 Biology/Botany 24 2.7 2.7 81.7 Anthropology 21 2.4 2.4 84.1 Nutrition 21 2.4 2.4 86.5 Teach Int'l Languages 18 2.0 2.0 88.5 Recreation 17 1.9 1.9 90.5 Instructional Technology 17 1.9 1.9 92.4 Geography (Geo & P) 13 1.5 1.5 93.9 Music 11 1.2 1.3 95.1 Social Science 11 1.2 1.3 96.4 History 9 1.0 1.0 97.4 Electrical Engineering 8.9.9 98.3 Art (All MAs) h 7.8.8 99.1 Geosciences 5.6.6 99.7 Nursing 3.3.3 100.0 Total 880 99.4 100.0 Missing System 5.6 Total 885 100.0 a Includes MBA, MS Accounting, MIS, Redding program. b Includes Psychology, MFT, PPS c Includes Education, Ed Admin, Special Education, Curriculum d Includes Interdisciplinary MA, MS, ISMA Math, ISMS Math Ed e Includes Communications, Information and Communications Studies f Includes Political Science, MPA, MPA with Health Admin option g Includes English, MFA Creative Writing, English Literature h Includes Art, Art History Table 2 summarizes the responses to what are arguably the most important quality measures of the graduate programs at CSU, Chico: rate overall quality, variety of courses, availability of courses, overall quality of faculty and the answer to the question: Were you intellectually challenged by your program? Recall that in all cases, the closer the response is to 1 the better the program s performance. In these pooled responses the mean response is around 1.6 for all measures, except variety and availability of courses, where responses average closer to 2. In all cases, the median and modal responses are 2, or good. Absent external benchmarks, it is difficult to assess how we are doing on these measures of overall quality. But it
is clear that variety and availability of graduate-level courses is viewed as somewhat problematic by our students. Table 2. Graduate Student Exit Surveys, All Graduate Programs, CSU, Chico Rate overall quality of graduate program Rate variety of course offerings in program Rate availability of course offerings Rate overall quality of department faculty N 881 880 881 878 876 Missing 4 5 4 7 9 Mean 1.6657 2.0057 2.0023 1.6093 1.6261 Median 2.0000 2.0000 2.0000 2.0000 2.0000 Mode 2.00 2.00 2.00 1.00 2.00 StdDev.62323.73393.81464.66300.59504 Were you intellectually challenged by program? The second set of quality indicators in Table 3 focuses more on what might be termed services provided by the programs and university: advising, assistance with financial aid, and career counseling. Faculty score high on availability for advice but slightly lower on satisfaction with advice received, and Quality of encouragement to persist toward goals. Scores are notably lower on Career counseling, and especially on Satisfaction with financial aid assistance. On this last question, the modal response was Not applicable. While this can be interpreted in a variety of ways, it appears to indicate that students received little or no assistance in obtaining financial aid. Why this is so is impossible to determine from the survey responses alone and bears further investigation. But clearly, our graduate students view the situation of financial aid as problematic. Table 3. Graduate Student Exit Surveys, All Graduate Programs, CSU, Chico Rate faculty availability for advice. Rate degree of satisfaction with advice received. Rate degree of satisfaction with assistance obtaining financial aid. Rate quality of encouragement received from department/committee to persist toward goals. Rate quality of career information received. N 877 873 874 870 792 Missing 8 12 11 15 93 Mean 1.5753 1.7520 3.2946 1.7109 2.3068 Median 1 2.0000 3.0000 2.0000 2.0000 Mode 1 2.00 5.00 1.00 2.00 StdDev.71196.77634 1.5776.80207.92715 The third set of quality indicators in Table 4 focuses on a variety of topics: student morale in the program, quality of thesis direction (for those completing a thesis), focus of programs, quality of resources in programs, and, finally, the quality of services offered by the Graduate School. The mean of most response hovers around 2.0. (Remember that 2.0 is the optimal response for the focus of program question). Students ranked quality of direction on the thesis highest among all these indicators. This is somewhat surprising, given that many students mention issues with the thesis as one of the things they would do differently (see below). One possible interpretation is that students underestimate the challenge of the thesis, but
feel that, ultimately, faculty are doing a very good job at guiding them through the thesis writing process. Table 4. Graduate Student Exit Surveys, All Graduate Programs, CSU, Chico Rate student morale in your program. Rate thesis direction received from committee and chair. Rate the focus of your masters program (2 = About Right ) Rate quality of resources and facilities in your program. Rate quality of assistance received from Graduate School. N 858 477 874 843 838 Missing 27 408 11 42 47 Mean 1.9790 1.6887 2.0561 2.0991 1.9511 Mode 2.00 1.00 2.00 2.00 2.00 Std. Deviation.67167.74051.32683.81223.75134 Next we will briefly review the responses to the open-ended questions: Question 20, Given what you know now, what would you have done differently in the pursuit of your master s degree? Question 21, What specific suggestions do you have for improving the quality of your program? and Question 22, What specific suggestions do you have for improving the quality of the service offered by the Graduate School? In reviewing the responses to these questions, we were faced with the difficulty of taking hundreds of responses and distilling these down to a manageable number, without sacrificing too much of the sense and meaning of each response. In some cases this was not too difficult: many people responded variations of none or nothing to these questions. Other cases were more difficult to resolve. Also, there tends to be many more missing data here, as graduates did not take the time to write out responses to these questions. With these caveats in mind, we will proceed to examine the responses. Tables 5,6 and 7 which summarize these responses are accompanied by bar charts that present a graphic depiction of the responses.
Table 5. In retrospect, what would you do differently? F % Valid % None (everything OK) 129 14.6 21.5 Personal planning (start earlier, finish more quickly, etc) 103 11.6 17.2 Thesis related (start thesis sooner, etc) 93 10.5 15.5 Advising (sought more, better, rel w/ committee) 76 8.6 12.7 Pursued degree elsewhere 42 4.7 7.0 Taken more classes 39 4.4 6.5 Miscellaneous/other 25 2.8 4.2 Program planning (course sequence, not write thesis) 19 2.1 3.2 Program specific comments 16 1.8 2.7 Different program (soc wrk to psych, vice versa, other) 16 1.8 2.7 Not worked full-time 14 1.6 2.3 Academic preparation (better, different) 10 1.1 1.7 More Ind study, internship, field, job experience 9 1.0 1.5 Financial Aid 5.6.8 No degree 3.3.5 Total 599 67.7 100.0 Missing 286 32.3 Total 885 100.0
Table 5 presents the results from Question 20, what would you do differently? It is gratifying that the modal response (21.5%) is None, everything ok. indicating student satisfaction with their experience. At the other end of the spectrum is the response, pursued degree elsewhere which was the response of about 7% of those who answered this question. In other words, more than three times as many respondents would change nothing, compared to those who would not choose CSU, Chico as the university to pursue their degree. On the other hand, it is a little discouraging that nearly 1 out of 12 graduate students would go elsewhere if they had to do it over again. Other notable categories of response had to do with planning around the thesis (15.5%) and seeking more or better quality advice about the program (12.7). Table 6 summarizes responses to the question regarding suggested improvements to the quality of the specific degree program. Here the modal response (28.8%) focuses on the variety and availability of classes. As we saw in Table 2 (above) variety and availability of courses is the indicator on which our graduate programs score lowest. Doubtless, this is symptomatic of the overall focus of CSU, Chico (and the CSUs in general) on undergraduate education, to the neglect of graduate programs. Variety and availability of classes is followed by improved advising and improved facilities, which, together with variety/availability of classes, accounts for about 50% of the responses to this question. Again, these responses reflect the secondary role of graduate education, where faculty time is focused mostly on undergraduate education and the research facilities necessary to support graduate student research is not a priority at our campus. As faculty and administrators involved in graduate education, this will not come as a surprise to most of us. What is interesting is that students notice this, too. In fact, this pattern of responses enhances our confidence in the validity of student perceptions in general, giving the overall results of the exit surveys more credibility.
Table 6. Suggested Improvements to Your Degree Program More classes, better availability F % Valid % Cumulative % 161 18.2 28.8 28.8 Improved advising 61 6.9 10.9 39.7 Improved facilities 52 5.9 9.3 49.0 None (no changes needed) 52 5.9 9.3 58.3 Miscellaneous, other (More selective admissions) 48 5.4 8.6 66.9 Better faculty 42 4.7 7.5 74.4 More professors 33 3.7 5.9 80.3 Program specific (specific courses, sequences) 33 3.7 5.9 86.2 More rigor 16 1.8 2.9 89.1 More practical training, internships 14 1.6 2.5 91.6 Thesis related 14 1.6 2.5 94.1 More job information 12 1.4 2.1 96.2 Financial Aid 10 1.1 1.8 98.0 More specialized 8.9 1.4 99.5 More opportunity to teach 2.2.4 99.8 Less specialized 1.1.2 100.0 Total 559 63.2 100.0 Missing 326 36.8 Total 885 100
Table 7. Suggested Improvements for Graduate School F % Valid % Cumulative % None (OK as is) 112 12.7 30.5 30.5 Improve communication (send more, web, newsletter) 85 9.6 23.2 53.7 Don't Know, little/no contact 74 8.4 20.2 73.8 Improve efficiency (availability, processing of 28 3.2 7.6 81.5 forms) Personnel complaints 21 2.4 5.7 87.2 Program specific (more classes) 15 1.7 4.1 91.3 Thesis related (prep workshop each semester) 11 1.2 3.0 94.3 Misc, other 9 1.0 2.5 96.7 More job information 7.8 1.9 98.6 Commencement (more seating) 5.6 1.4 100.0 Total 367 41.5 100.0 Missing 518 58.5 Total 885 100 The final set of open-ended responses focuses on improving services provided by the Graduate School. The Graduate School is an administrative arm of the university responsible for admissions, general information for students on procedures throughout their graduate education, review of thesis, processing applications for graduation and the organization of commencement, among other duties. The Graduate School does not deliver educational content to students, but provides a context for programs to operate and for students to pursue their degrees. About half of the respondents to this question replied either none (no improvements needed) 30% or Don t know, little contact with Graduate School 20%. This is not a bad outcome as both responses basically indicate no complaint with the Graduate School. The don t know responses are a little troubling, as students seem to be unaware of what the Graduate School does, but, it could be worse! The most common suggestion for improvement was more and better communication. Complaints focused on perceived bureaucratic delays and inefficiencies (7.6%) with only 5.7% having specific complaints about treatment received from Graduate School personnel. Given the volume of graduate students and limited staff resources, this is a very positive result. Biology: Biology graduate students rated their program as below the mean on student morale and both variety and availability of course offerings (see Table 9). There were no time trends evident in the responses, but the number of cases is small. Also, the number of respondents to the openended questions is small. However it is notable that for Question 20, the response, pursued degree elsewhere was at 15.8% (three responses) twice as high as the university average (7%). Also, Biology graduate students seemed to have more complaints with Grad School personnel (4 = 40%) compared to average (6%). BIOL students are < 0.5% of all CSU, Chico graduates but make up nearly 20% of those who complained about Grad School personnel.
Table 9. BIOL vs. All Other Graduate Programs, T-test results of significant differences N Mean StdDev T Sig Rate variety All = 851 1.9935.72818 3.348 0.001 of course offerings BIOL = 24 2.5000.82092 Rate 4.199 0.000 availability of All = 852 1.9853.80507 course offerings BIOL = 24 2.6875.90665 Rate student morale in your program. All = 829 1.9710.66875 BIOL = 24 2.2500.79400 2.003 0.045