Coupling a Retention-Driven Predictive Analytics Initiative with MSCHE Standards December 8, 2017 Steven Doellefeld, Ph.D., MBA Jack Mahoney, MBA STANDARD IV SUPPORT OF THE STUDENT EXPERIENCE The institution commits to student retention, persistence, completion, and success through a coherent and effective support system sustained by qualified professionals, which enhances the quality of the learning environment, contributes to the educational experience, and fosters student success. 1
STANDARD IV SUPPORT OF THE STUDENT EXPERIENCE Clearly stated, ethical policies and processes to admit, retain, and facilitate the success of students whose interests, abilities, experiences, and goals provide a reasonable expectation for success and are compatible with institutional mission, including: o Orientation, advisement, and counseling programs to enhance retention and guide students throughout their educational experience; o Processes designed to enhance the successful achievement of students educational goals including certificate and degree completion, transfer to other institutions, and postcompletion placement; Vision Improving student retention rates through a robust academic early warning system for academically at risk students, drawing on data from both traditional and non-traditional sources. Improving student retention rates by providing rich analytics for advisors to help them guide students into academic areas that may be better suited for their skills, abilities, and interests. Fostering a richer student relationship with their advisors, faculty, and staff Fostering improvements in admissions practices by predicting the likelihood of student success, potentially opening the door to applicants who might otherwise have been overlooked. Supporting and promoting experiential education activities of students and faculty through monitoring, tracking, and development of said activities. 2
THE UALBANY ADVANTAGE Combining cutting edge data analytics and building student success teams across campus which will empower students to reach new levels of success. Freshmen to Sophomore Retention Rates 3
Freshman Retention Rates High Touch Programs EAB: Decision Making Process Strongest Factors: Predictive engine Intuitive user interface Strong advisement tools Value-add reporting tools 4
Predictive Analytics Overview Visibility uncover hidden risk Prioritize and triage Intervention Organize and act early Measure the impact Predictive Risk SSC uses a risk model to predict individual students likelihood to graduate in a given major The risk model is used to help prioritize advisor interaction with students as well as trigger other analyses Advanced multivariate statistical and machine learning techniques perform variable selection and conduct hypothesis testing to highlight the most impactful trends Data scientists use academic and demographic data of historical students to create the predictive model Current students are then scored or applied against our predictive model on a scale from 0 - unlikely to graduate to 1 - most likely to graduate 5
Data Requirements Phase I Types of Data Required (Rolling 10 years): General Student Information (including HS) Student Registration Audit Student Term Info Including GPAs Student Major Declaration by Term Minor Declaration Student Course Info by Term (including Grades) Student Group Participation Course Information Section Information Section Meeting Patterns and Times Section Instructor Student Section Absences (if available) Student Exam Results (if available) Student Relationships with Advisor, Instructor, Tutor, etc. Data Elements Used in UAlbany Model Cumulative GPA First Term GPA Last Term GPA GPA Ever Below Threshold GPA Trend Term Grade Variance Run Grade Variance D-F Counts D-F Run Total D-F Trend W Counts W Run Total W Trend First Academic Year Ever Graduated Indicator Current Majors Current College Current Degree Current Major Frequency Number of Current Majors Number of Former Majors Completed Terms Attempted Credits Trend Average Credits per Term Current Credits per Term Earned-Attempted Credit Ratio 6
Data Elements Used in UAlbany Model (continued) First Term Transfer Credits Lifetime Accum. Credits Lifetime Attempted Credits Lifetime Transfer Credits Post Matric Transfer Credits Transfer Credit Proportion Major-Skill Alignment Estimated Skills International Student Indicator Readmitted Student Indicator Transfer Student Indicator Veteran Indicator Standardized SAT/ACT Scores Student Admit Code Student Ethnicity Student Gender Data Requirements Phase II Financial Aid Data Required (Rolling 10 years): FAFSA Data (e.g., EFC, parents education, calculated need, Total Aid vs Total Need Gap) Award Detail Federal need-based scholarship (Pell, SEOG) State scholarship (TAP) Federal sub and unsub loans (Perkins, Stafford) Alternative loans Work Study Institutional scholarship Offered, Accepted, Disbursed Amounts 7
Swipe Card Data Potential Phase III Non-academic or financial data Bus Use Meal swipes Library use Athletic events Student activities Post-graduate data (e.g., NSC) Project Organization UAlbany Advantage Leadership Team Project Manager Functional Leader Technical Lead Workflow Development Training & Development Success Marker Development Campaign Coordination Policy Development Influence configuration, functionality, roles & permissions. Early engagement, participation in training & development. Coordination and collection of success marker milestones. Alignment of outreach activities across campus. Development and review of applicable data governance policy. 8
Engagement Teams Early engagement Create in-office experts Support early adopters Empower ownership Cross-functional collaboration 1 st Year Impact Cross-University Collaboration for student success Building relationships & a platform for engagement Focus on delivering a solution, not just technology 9
2 nd Year Impact Development of targeted campaigns for advisement Collaboration with department chairs and professional advisors Developing and subsequent rollout of success markers Leveraging Technology for Impactful Outcomes Advising Tools: Informed student interactions Campaigns: Data-driven pro-active outreach Success Markers: The next level 10
PLATFORM FOR ENGAGEMENT COLLABORATION Advising Tools: Gut Check 11
Advising Tools: Communication Campaigns: Data-Driven Outreach 12
Advising Tools: Individual Trends Success Markers: Definition Beyond identification of predictive courses Focus on level of course success to predict future success in major. 13
Success Markers: Course Analysis Average Credits Earned Average Grade Graduation Rate Percent D/F Percent W Ten Most Attempted Courses Ten Most Predictive Courses Success Markers: Course Data # of Predictive Predictive Avg. Avg. Lifetime Earned Credits Grad. Number Course Students Course Rank Cutoff Grade Grade when Attempted Rate of Grads ACHM120 1334 1 B 2.41 28.36 28.90% 385 ACHM121 1183 2 B 2.15 45.75 34.10% 403 AMAT108 1220 3 A 2.74 42.14 33.60% 410 ABIO212Y 1057 4 B 2.74 61.19 46% 486 ACHM220 1085 5 C 1.99 67.38 50% 543 ACHM124 1294 6 B 2.98 27.32 30.40% 393 APHY105 951 7 B 2.45 68.63 44.90% 427 ABIO110 616 8 B 2.71 24.84 29.90% 184 ABIO402 730 9 C 2.79 102.26 69.60% 508 APSY101 1007 10 B 2.76 38.98 28.80% 290 ACHM125 1110 11 A 2.95 43.58 35.80% 397 ABIO121 535 12 C 2.23 45.58 31.60% 169 ABIO111Z 563 13 B 2.64 39.76 32% 180 14
Success Markers: MAPs Major Data Previous Major 15
Major Data Next Major Success Markers: Advising Insights 16
Combining cutting edge data analytics and building student success teams across campus which will empower students and UAlbany to reach new levels of success. THE UALBANY ADVANTAGE Impact of UAlbany Advantage Unified technology Proactive outreach Fostering collaboration across the University Sharing information to improve student outcomes Leveraging success teams 17
QUESTIONS? Contact Info Steven Doellefeld, Ph.D., MBA steven@albany.edu 518-437-4564 Jack Mahoney, MBA jmahoney@albany.edu 518-437-4928 18