Using NSSE Findings to Examine the Relationship Between Student Engagement and First- Year Retention and Attrition Presenter: Rick Miller, Ph.D. 1
Overview SUNY Potsdam has been using the NSSE since 2003. This workshop will describe how SUNY Potsdam has linked five years of NSSE and institutional data to examine patterns of student retention and attrition from multiple cohorts. Explain how NSSE data were used in this study to compare engagement levels of student persisters and leavers after the first year of study at SUNY Potsdam. 2
Workshop Outcomes Demonstrate how NSSE data can be linked to institutional data sets. Discuss how to effectively use NSSE findings for improving student retention. Demonstrate how longitudinal data can be used to assess retention patterns. 3
Purpose Collecting NSSE data over many years provides institutions with greater ability to monitor trends and patterns. Five years of NSSE and institutional data were merged to examine a large pool of freshmen from multiple student cohorts. 4
Purpose (Cont.) Compare engagement levels of students who returned to SUNY Potsdam in the fall of their second year and students who did not decide to return. Provides a means of narrowing down and identifying areas in which an institution can focus to retain firstyear students. Identify areas that might be problematic for nonpersisters. 5
Methodology SUNY Potsdam contracted with Hanover Research Council (July 2008). Modeled after analyses conducted at Ohio University and UNC Charlotte. Longitudinal student tracking files (by entering cohort) matched with corresponding NSSE files via unique ID match. Frequency analyses and t-tests were conducted to compare differences between persisters and non-persisters. 6
Report Analyses Organized into three levels of analysis: 1. Demographic and academic characteristics. 2. NSSE Student level index scores. Provides a snapshot of student engagement and persistence. 3. Question by question detail. Take a closer look at scale items. Helps to understand differences between persisters and leavers. 7
Demographic and Academic Characteristics 1,061 first-year students included in both data sets. 877 enrolled in fall semester following completion of the first-year. 184 did not enroll. 8
Demographic and Academic Characteristics (cont) Gender: the two groups closely resembled each other. The non-persisting group included 30.5 percent males and 69.5 percent females. Among persisting students, 33 percent were male and 67 percent were female. 9
Demographic Characteristics (cont) % Enrolled % Not Enrolled White 77.7 85.6 Hispanic 2.6 2.5 Black 1.5 Asian/ Pacific Islander 2.0 2.6 American Indian/Alaska Native 1.1 Foreign 2.8 1.7 Unknown 12.3 7.6 10
Academic Performance Characteristics Measure SAT Total Score SAT Math Score SAT Verbal Score Average SAT Scores and GPAs Enrolled Not Enrolled Mean Diff. N Mean N Mean 660 1087.89 142 1067.82 20.08 811 542.16 162 534.51 7.65 811 546.97 162 542.10 4.87 GPA 877 3.11 184 2.37 0.74 Note that SAT scores are taken from institutionally reported data provided in the NSSE surveys and GPAs are taken from the SUNY Potsdam student tracking files.
Key Findings Overall, we found that students who were ultimately retained after one year were more likely to have displayed higher levels of engagement during their first year on nearly all items of the survey. 12
Key Findings (cont.) Persisting students reported: higher levels of participating in class discussions. having serious conversations with students who were very different in terms of their religious beliefs, political opinions, or personal values more often than non-persisting students. that examinations challenged them to do their best work. 13
Key Findings (cont.) Students who were retained evaluated the quality of academic advising more positively. Students who enrolled in the fall after their second year were less likely to work off-campus than non-persisting students. 14
Key Findings (cont.) Students who returned for the fall of their second year reported having higher quality relationships with other students, faculty members, and administrative personnel than students who did not return. Students who persisted were more likely to feel that the institution provided them with the support they needed to thrive socially and to attend campus events. 15
Key Findings (cont.) Persisting students evaluated their entire educational experience at SUNY Potsdam higher than non-persisting students, with an overwhelming majority indicating that they would definitely (47.4 percent) or probably (38.7 percent) choose to attend the institution even if they could start over again. 16
Benchmark s Comparisons 2005-2007 Benchmarks Enrolled Not Enrolled Mean Diff N Mean N Mean Sig Academic Challenge studentlevel index (adjusted for parttime status) Active and Collaborative Learning student-level index score Student-Faculty Interaction student-level index score * p<.05 ** p<.01 ***p<.001 556 52.152 125 51.060 1.092 -- 615 43.335 145 40.413 2.921 -- 561 34.288 127 31.899 2.389 -- 17
Benchmark s Comparisons (cont.) Benchmarks Enrolled Not Enrolled N Mean N Mean Mean Diff Sig. Enriching Educational Experiences student-level index score 537 27.899 119 26.372 1.528 -- Supportive Campus Environment student-level index score 521 63.986 115 57.575 6.411 ** * p<.05 ** p<.01 ***p<.001 18
Q7. Which of the following have you done or do you plan to do before you graduate from your institution? Response Frequencies (%) Item Status Undecided No Yes 7a 7b 7c 7d Practicum, internship, field experience, co-op experience, or clinical assignment Community service or volunteer work Participate in a learning community or some other formal program where groups of students take two or more classes together Work on a research project with a faculty member outside of course or program requirements Enrolled 15.52 2.8 81.68 Not Enrolled 22.15 7.59 70.25 Enrolled 21.09 7.62 71.28 Not Enrolled 22.78 10.13 67.09 Enrolled 32.57 26.08 41.35 Not Enrolled 34.19 26.45 39.35 Enrolled 44.85 23 32.15 Not Enrolled 38.71 31.61 29.68 19
Q7. Which of the following have you done or do you plan to do before you graduate from your institution? Response Frequencies (%) Item Status Undecided No Yes 7e Foreign language coursework Enrolled 14.25 24.3 61.45 Not Enrolled 16.67 22.44 60.9 7f Study abroad Enrolled 33.38 25.86 40.76 Not Enrolled 35.9 22.44 41.67 7g Independent study or self-designed major Enrolled 35.41 50.83 13.76 Not Enrolled 39.1 46.79 14.1 7h Culminating senior experience (capstone course, senior project or thesis, comprehensive exam, etc. Enrolled 48.54 10.06 41.4 Not Enrolled 51.92 13.46 34.62 20
Quality of Relationships Q8. Mark the box that best represents the quality of your relationships with people at your institution. Enrolled Not Enrolled Mean Item N Mean N Mean Diff Sig. 8a 8b Relationships with other students Relationships with faculty members 1=Unfriendly, unsupportive, sense of alienation; 7=Friendly, supportive, sense of belonging (8a) 787 5.644 158 5.070 0.575 *** 1=Unavailable, unhelpful, unsympathetic; 7=Available, helpful, sympathetic (8b) 786 5.566 158 5.139 0.427 *** 8c * p<.05 ** p<.01 ***p<.001 Relationships with administrative personnel and offices 1=Unhelpful, inconsiderate, rigid; 7=Helpful, considerate, flexible (8c) 786 4.963 158 4.456 0.507 *** 21
How Can These Findings be Used for Improving Student Retention? Link with other data (e.g., non-returning student survey, Annual Freshmen Survey) to validate findings and for additional insight and understanding patterns of persistence or departure. Present findings widely, encourage dialogue, and collaborate closely to improve programs and services with the following leadership & policy groups for example: President s Administrative Cabinet General Education Committee Retention Management Council Academic Departments (Monthly NSSE Interest Workshops) First-Year Experiences Program, Student Affairs Units, Undergraduate Admissions 22
References The Hanover Research Council, (August 2008): SUNY Potsdam Multi-Year NSSE Data: Freshman Retention Analysis. Internal report prepared for SUNY Potsdam. National Survey of Student Engagement. (2000). National Benchmarks of Effective Educational Practice. Bloomington, IN: Indiana University, Center for Postsecondary Research. 23