National Survey of Student Engagement at Highlights for Students Sue Erickson Carmen Williams Office of Institutional Research April 19, 2012 April 19, 2012
Table of Contents NSSE At... 1 NSSE Benchmarks... 2 Time Preparing for Class... 8 Advising... 10 Overall Satisfaction... 12 NSSE In Action... 14 About Institutional Research... 15 Prepared by s Office of Institutional Research (701) 777 4358 www.und.nodak.edu/dept/datacol
National Survey of Student Engagement At The National Survey of Student Engagement (NSSE) is one of the most commonly used assessment tools aimed at gauging levels of student activity on campuses across the United States. NSSE asks first year and senior students to report on various aspects of their college experience, both inside and outside the classroom. Nearly 1,500 colleges and universities across the United States and Canada have used NSSE, so comparisons can easily be made between and similar institutions. has participated in NSSE during the spring semester of odd numbered years since 2001. While various NSSE survey questions have been changed, fairly consistent data is available since 2003. This report will focus on aspects of NSSE that might be of interest to student groups. It will look at trends over the past decade, as well as focus on ways is using NSSE results to impact academic and support programming. What is student engagement? While there doesn t exist a clear definition for student engagement, the meaning is generally understood to reflect active student learning practices. The more students study a subject and the more they interact with faculty regarding their coursework, the more likely they are to understand that subject matter and enhanced learning outcomes will be evident. Likewise, the more involved a student is on campus, both within and outside the academic realm, the more connected they become to campus and the greater the likelihood they have a positive experience. This will also lead to enhanced learning and increased retention and graduation rates. High levels of engagement leads to greater numbers of successful graduating students. NSSE has two companion surveys: The Faculty Survey of Student Engagement (FSSE) asks faculty members to report on their impressions of student engagement. This tool allows institutions to compare faculty and student responses to determine similarities and/or difference between what faculty report they perceive and students report what they are experiencing. The Beginning College Survey of Student Engagement (BCSSE) asks incoming first year students to reflect on the aspects of engagement they experienced in high school and their expectations for the first year of college. These responses can be combined with first year NSSE responses to provide a picture of student transition to college, and how their experiences are meeting their expectation. This report will look at various components of both the FSSE and BCSSE alongside NSSE results. Office of Institutional Research - NSSE For Students Page 1
National Survey of Student Engagement Benchmarks The National Survey of Student Engagement results are combined into five benchmark areas. These benchmark areas include: Level of Academic Challenge (LAC): Including items tied to hours spent preparing for class, number of assigned readings and written papers or reports, and perceived coursework emphasis on analyzing elements, synthesizing information, making judgments about the value of information and applying theories to new situations. Active and Collaborative Learning (ACL): Asking students about learning in different settings, including asking questions in class, making class presentations, working with classmates, tutoring, and discussing course topics outside of class. Student Faculty Interaction (SFI): Interactions with faculty inside and outside the classroom, such as discussing grades or career plans, working with faculty outside of coursework or on research, and receiving prompt feedback from faculty. Enriching Educational Experiences (EEE): Complementary learning opportunities, including hours spent participating in co curricular activities, participation in internships, community service, foreign language coursework, and senior capstone projects, and interactions with a diverse mix of students. Supportive Campus Environment (SCE): the satisfaction students report on the campus environment providing support to succeed both academically and non academically, and student rating of relationships with other students, faculty, and administrative personnel. Benchmarks are calculated on a 100 point scale. To achieve a benchmark mean of 100, however, each survey taker would have to record the highest response for each of the benchmark component survey items. Each benchmark is calculated for both firstyear and senior respondents. Due to the large population of students taking NSSE, small differences in benchmark mean score will calculate to be statistically significantly different. Institutions, therefore, are encouraged to look at populations within their campus to determine which groups of students might score higher or lower on a particular benchmark. The following sections of this report look at these various benchmark areas. Figure 1 shows performance on NSSE benchmarks versus select national comparison groups. scores indicate the best relative performance in the area of having a Supportive Campus Environment. shows the worst (when compared to peer groups) in the area of Enriching Educational Experiences. This stems from s relative lack of diversity among the student population, and the tendency of seniors to not be as likely to indicate they have completed internships and study abroad. Office of Institutional Research - NSSE For Students Page 2
Level of Academic Challenge (LAC) Active and Collaborative Learning (ACL) Class Figure 1. University of North Dakota Benchmark Results from the 2011 National Survey of Student Engagement (NSSE) Five benchmarks are created from clusters of NSSE questions. The below table summarizes 's results compared to peer groups. When 2011 scores are compared to 2009, first-year scores show improvement while scores for seniors are on the decline. 's worst performance is in the area of enriching educational experiences. Comparison Groups* How challenging is your institution's intellectual and creative work? Are your students intensely involved in their education? Student-Faculty Interaction (SFI) Do your students work with faculty members inside and outside the classroom? Enriching Educational Experiences (EEE) Do your students tak e advantage of complementary learning opportunities? Supportive Campus Environment (SCE) Do your students feel the college is committed to their success? 2009 Scores Institutional Peers Carnegie Peers (RU/H) Plains Public First-Year 51 53 ------ ------ ------ Senior 56 55 ------ First-Year 39 41 ------ ------ ------ Senior 50 47 ------ First-Year 31 33 ------ ------ ------ Senior 41 40 ------ ------ ------ First-Year 23 25 Senior 37 36 ** * First-Year 60 62 ------ ------ Senior 56 57 ------ ------ ** ** ** * Indicates 's score is statistically lower than the comparison group. Indicates 's score is statistically higher than the comparison group. 7 of 's 15 Institutionally defined peers participated in NSSE 2011; 49 RU H institutions participated ('s 2010 Carnegie Class); 27 Plains Public institutions participated (including public institutions from IA, KS, MN, MO, NE, ND, & SD ** Indicates an Effect Size greater than 0.20, implying more than a small significance. (Effect size equals mean difference divided by the pooled standard deviation.) Office of Institutional Research - NSSE For Students Page 3
Figure 2. First Year NSSE Benchmarks At Over Time 61.5 NSSE First Year Benchmarks 2003 2005 2007 2009 2011 Supportive Campus Environment Level Academic Challenge Active and Collaborative Learning Student Faculty Interaction Enriching Educational Experiences (not calculated in 2003) 52.6 41.2 38.7 24.9 While relatively flat, all FY benchmark scores show an increase in the 2011 administration over the 2009 benchmark scores. This follows the trend where all 2009 benchmarks increased over 2007 benchmark mean scores. The benchmark rankings (SCE highest to EEE lowest) is a trend that is consistently seen at all institutions and is driven by the nature of the component survey items. Office of Institutional Research - NSSE For Students Page 4
Figure 3. Senior NSSE Benchmarks At Over Time NSSE Senior Benchmarks 2003 2005 2007 2009 2011 Supportive Campus Environment Level Academic Challenge Active and Collaborative Learning Student Faculty Interaction Enriching Educational Experiences (not calculated in 2003) 57.2 55.0 46.8 43.9 35.6 For seniors, benchmark means trended downward in 2011 when compared to 2009 benchmark scores, with SCE being the only item to show an increase. This downward trend may be a one year anomaly in the data points, as the mean scores for seniors have tended to alternate up and down over time. As a general rule, senior benchmark scores tend to be higher than that of FY respondents, however comparisons between the two groups are generally discouraged, as the perspectives and expectations of FY and SR students are quite different. Office of Institutional Research - NSSE For Students Page 5
Figure 4. First Year Benchmarks By Where Student Lives Figure 5. Senior Benchmarks By Where Student Lives Benchmarks by Type of Residence 52.5 56.0 50.4 53.7 Level of Academic Challenge 41.6 45.8 39.0 29.6 Active & Collaborative Learning Dormitory/Campus Housing 32.9 36.0 32.3 Off campus w/in driving distance 23.3 Student Faculty Interaction 25.0 27.1 22.0 21.6 Enriching Educational Experiences 61.6 64.5 64.1 49.0 Supportive Campus Environment Off campus w/in walking distance Other 83% of FY respondents report living in dormitory / campus housing, so those responses tend to drive s overall FY survey responses. While the population is small, it is interesting to note those FY respondents living off campus within walking distance of tend to score higher on the NSSE benchmarks than on campus students. 52.36 54.38 55.56 56.79 53.1 47.54 50.23 50.11 58.7 28.4 35.73 42.62 41.66 45.88 31.3 36.65 36.63 37.39 48.78 26.9 57.56 58.17 57.73 56.25 54.5 Level of Academic Challenge Active & Collaborative Learning Dormitory/Campus Housing Off campus w/in driving distance Other Student Faculty Interaction Enriching Educational Experiences Supportive Campus Environment Off campus w/in walking distance Fraternity/Sorority house SR respondents living in fraternity or sorority houses tend to score the highest benchmark mean scores, with the exception being SCE. The Other category is primarily made up of those students taking online courses. Fig. 6 presents additional information on these online student responses Office of Institutional Research - NSSE For Students Page 6
Benchmarks for Online Learners When evaluating the differences between respondents taking primarily online courses versus those on campus, some interesting revelations occur. It is worth noting that online respondents report little difference with regard to the level of academic challenge benchmark and the supportive campus environment benchmark. This would indicate they feel they are experiencing similar quality of coursework, while at the same time receiving adequate support even though they do not reside on campus. It is not so surprising to see the online respondents reporting lower benchmark scores for those items requiring personal interaction: ACL, SFI, and EEE. Not being on campus provides few opportunities for students to work together, fewer opportunities to have discussions with faculty, and minimal opportunity to interact with those different from themselves. Few FY online respondents completed NSSE, therefore their results are not presented. With regard to the SR respondents, 72% of them attend parttime, which is quite different from s student population where just one fourth attend part time. 54.8 53.8 Level of Academic Challenge Figure 6. Senior Benchmark Scores On campus Versus Online Student Respondents 50.2 30.5 Active & Collaborative Learning 41.5 31.9 Student Faculty Interaction On Campus 37.5 Online 27.9 Enriching Educational Experiences 57.7 55.2 Supportive Campus Environment Office of Institutional Research - NSSE For Students Page 7
Time Spent Preparing For Class One positive revelation from the 2011 NSSE is that respondents, both first year and seniors, are reporting increasing amounts of time being spent preparing for class (Fig. 7). The averages indicate most students spend roughly thirteen to fifteen hours per week on academic preparation on things such as studying, reading, writing, doing homework or lab work, analyzing data, or rehearsing. While positive things are happening on the academic front, 2011 NSSE results may show some caution with regard to student involvement outside the classroom. Both FY and SR respondents reported spending less time on co curricular activities, which are defined by NSSE as organizations, publications, student government, fraternities or sororities, and intercollegiate or intramural sports (Fig. 8). Fig. 9 shows student time spent studying comes close to what faculty expect, yet faculty think students actually spend significantly less time on academic work. 3.98 4.60 4.52 Expect Figure 9. Faculty Impression of Time Spent Preparing For Class 2.64 2.55 2.62 3.61 3.81 4.18 Actually Think 4.06 4.28 4.18 2.94 3.01 2.84 3.94 4.00 4.46 Actual Expect Actually Think Actual LD Faculty FY UD Faculty SR 3.5 3 2.5 2 1.5 3.94 3.61 Figure 7. Reported Hours Spent Preparing for Class Mean on 8 point scale (1= None 2= 1 5 hours per week 3= 6 10 hrs. per week 4= 11 15 hrs. per week 5= 16 20 hrs. per week 6= 21 25 hrs. per week 7= 26 30 hrs. per week 8= more tha 4.13 3.71 4.00 3.81 4.27 4.46 4.10 4.18 2003 2005 2007 2009 2011 Figure 8. Reported Hours Spent articipating in Co Curricular Activities Mean on 8 point scale (1= None 2= 1 5 hours per week 3= 6 10 hrs. per week 4= 11 15 hrs. per week 5= 16 20 hrs. per week 6= 21 25 hrs. per week 7= 26 30 hrs. per 2.13 2.15 2.05 2.06 2.28 2.43 2.26 2.29 2.18 2.00 2003 2005 2007 2009 2011 SR FY SR FY 2003 2007 2011 Office of Institutional Research - NSSE For Students Page 8
Figure 10. Hours Spent Preparing For Class BCSSE Vs. NSSE 5% 24% 69% 31% 55% 19% 42% 39% 2% 13% 0% 0% BCSSE High School BCSSE Expected FY NSSE FY None 1 10 hours 11 20 hours More than 20 hours Figure 11. Hours Spent Participating in Co curricular Activities BCSSE Vs. NSSE 14% 43% 5% 24% 3% 7% 57% 66% 36% 32% 6% 5% BCSSE High School BCSSE Expected FY NSSE FY None 1 10 hours 11 20 hours More than 20 hours Figures 10 and 11 look at reported time spent on these activities as reported on both BCSSE and NSSE. Recall BCSSE is given to incoming students and asks them to report on their high school experience and their college expectation while NSSE asks these same students during spring of their freshman year a similar question. When asked about their high school experience, most students (71%) report they spend ten or fewer hours preparing for class during high school. When asked what they expect in college, most report an increase, with nearly one third indicating they plan on spending more than 20 hours per week on academic work, When these students are asked as FY students, the amount they actually study is somewhere between more than what they reported they did in high school, but significantly less than what they expected as incoming students. When looking at participation in activities outside the classroom, incoming students report relatively high levels of participation during high school, and they expect to spend less time on these activities once they get to college. However when they are here on campus, the amount of time reported spent on these co curricular activities is even less than what the incoming student expect. This indicates that should not only encourage incoming students to seek opportunities to participate in student organizations but seek ways to make this participation easy. Office of Institutional Research - NSSE For Students Page 9
NSSE respondents generally report being happy with the academic advising they receive. Figure 12. Overall Evaluation of Quality of Advising (1 = Poor, 2= Fair 3= Good, 4= Excellent) Figure 12 shows slight increases in reported quality of advising over time, while Figs. 13 and 14 compare reported advising at being quite similar to that at research universities. 2.90 2.82 2.89 2.92 3.05 2.80 2.75 2.85 2.90 2.91 Advising Figure 13. First Year Student Reported Quality of Academic Advising First Year 2003 2005 2007 2009 2011 Senior Figure 14. Senior Student Reported Quality of Academic Advising Research Universities Research Universities Fair, 16% Poor, 6% Excell ent, 33% Poor, 5% Fair, 17% Excell ent, 34% Fair, 17% Poor, 10% Excell ent, 29% Fair, 22% Poor, 11% Excell ent, 28% Good, 45% Good, 45% Good, 44% Good, 39% Office of Institutional Research - NSSE For Students Page 10
When evaluating the quality of advising, the emphasis placed on faculty advising by each college may result in differing student expectations and student opinions. Figures 15 and 16 below show varying student respondent satisfaction with the advising received. Some positive highlights from first year students would include Arts and Sciences, which is showing a nice upward trend in the number of students responding the advising they have received is good or excellent. While the College of Education shows high quality advising in the first year, this represents relatively few respondents and may not be representative of the true atmosphere. For senior respondents, those from the School of Medicine and Aerospace report the highest quality advising, while Business seniors responded with the lowest level of quality. As a general rule, most colleges are seeing an increase in the level of reported advising quality. Figure 15. First Year Quality of Academic Advising By College (Percent Reporting Excellent or Good) Figure 16. Senior Quality of Academic Advising By College (Percent Reporting Excellent or Good) 73% 77% 72% 73% 73% 80% 68% 66% 77% 74% 76% 78% 67% 73% 78% 78% 87% 79% 77% 75% 79% 77% 84% 85% 70% 92% 90% 63% 72% 69% 65% 69% 70% 66% 70% 93% 57% 69% 70% 69% 70% 71% 65% 63% 72% 79% 73% 82% 65% 86% 100% Undec JDO SEM Institution A&S NUR BPA SMHS EHD 2007 2009 2011 BPA A&S SEM NUR Institution 2007 2009 2011 EHD JDO SMHS Office of Institutional Research - NSSE For Students Page 11
Overall Satisfaction With In summarizing student attitudes, NSSE asks respondents two questions: how they evaluate their entire educational experience at and if they could start over again, would they go to the same institution they are now attending. When compared to respondents from research universities with high activity (RU/H), respondents show more positive results. Nearly all FY respondents indicate they have a good educational experience at. 85% 84% 2005 Figure 17. Entire Educational Experience (Percent Responding "Excellent" or "Good") 2007 90% 90% 87% 84% 85% 84% 86% 84% 2009 FY 2011 RU/H 2011 2005 2007 2009 Figure 18. Would You Attend Again? (Percent Responding "Definitely Yes" or "Probably Yes") SR 2011 87% 84% 87% 87% 84% 81% 84% 85% 83% 80% RU/H 2011 2005 2007 2009 2011 RU/H 2011 2005 2007 2009 2011 RU/H 2011 FY SR Office of Institutional Research - NSSE For Students Page 12
NSSE results from first year respondents were evaluated with regard to comparing those respondents who returned to for their sophomore year in the fall of 2011 and those not returning. Not surprisingly, more of those returning indicate they had an excellent or good experience at, and many more indicate they would definitely attend again (Fig. 19). Similarly, those FY respondents who did not return are significantly less likely to indicate on NSSE that they would again choose if given the chance to start over (Fig. 20) Certain survey items reveal these students may not be developing close relationships with their fellow students which may be leading to a certain sense of alienation or not belonging. In particular, students not returning report a lower mean score when asked to rate the quality of their relationships with other students; on a 7 point scale, the mean scores are 5.57 for retained FY students compared to 5.07 for those not retained. Students not being retained also score a lower benchmark mean score in the Active and Collaborative learning benchmark, with those returning averaging 42.0 on this scale compared to 36.9 for those not returning. Figure 19. Rating Entire Educational Experience Returning FY Respondents Versus Non Returning Respondents Returning FY Respondents Good, 55% Poor, 2% Fair, 6% Excellent, 67% Non Returning Respondents Fair, 11% Poor, 11% Good, 48% Excellent, 30% Figure 20. Would You Attend Again? Returning FY Respondents Versus Non Returning Respondents Returning FY Respondents Definite No, 3% Probably Yes, 40% Probably No, 8% Definite Yes, 49% Non Returning Respondents Probably No, 24% Definite No, 11% Definite Yes, 22% Probably Yes, 43% Office of Institutional Research - NSSE For Students Page 13
NSSE In Action Students are often asked to complete various surveys to assist in assessing educational outcomes and student satisfaction. One question that may arise is Are these results being used? Two examples of NSSE results driving campus discussions include: In the Spring 2012 issue of On Teaching, published by s Office of Instructional Development, Brian Schill explains the importance of undergraduates participating in research with faculty and provides tips to faculty on how to get students active in their research work. Mr. Schill quotes NSSE to show undergraduates at are less likely to report having participated in research with a faculty when compared to national research universities (Fig. 21) Figure 21. Percent of Senior Respondents Indicating They Have Done Research With Faculty In the Fall of 2009, Provost LeBel requested the formation of a working group on campus to enrich the learning experience for undergraduates at. This Undergraduate Learning Working Group (ULWG), evaluated NSSE, along with other assessment tools, to develop actions that would lead to a fuller undergraduate experience for students. In particular, the ULWG recommended and implemented the formation of First Year Experience(FYE) courses aimed to support a successful transition to college, foster academic engagement, include core academic content, and meet at least one Essential Studies requirement. These FYE courses were developed and piloted during Fall 2011, and NSSE responses were used to measure differences between those students participating in the FYE courses and those not completing the courses. 15% 16% 18% 18% 20% 2005 2007 2009 2011 RU/H 2011 Office of Institutional Research - NSSE For Students Page 14
About Institutional Research The Office of Institutional Research (OIR) serves as a resource to university decision makers by providing official and consistent data, in a timely manner, that can be used to assess the goals and objectives of the University of North Dakota. Please visit our website: http://und.edu/research/institutional research/ In addition to conducting various survey assessments, OIR tracks official enrollments, student retention, and credit hour production. OIR also supports s Assessment Committee, and compiles data from the course evaluations (the University Student Assessment of Teaching (USAT) forms) completed each semester. Office of Institutional Research - NSSE For Students Page 15
Office of Institutional Research - NSSE For Students Page 16