NTU Student Dashboard
28,000 Students > 45% Widening Participation Background > 93% Employability < 5% Drop-out Rate
Our Starting Point Three Drivers: HERE Project (part of What Works? Student Retention & Success, 2008-2012) IS analytics innovation and experimentation Internal audit: good at retention, could use data more to better inform interventions
NTU Dashboard - Context Existing Rich Information: Business Intelligence Tools Refined Data Warehouse One Version of the Truth Dashboard: Simple Tools New Insights Right Technology Partner Excellent Student Support: Welcome Week Robust Tutor System Support One-stop Shop Etc Desired Outcomes: Attainment Belonging Retention Great Data Insight Means to Intervene Impact
Dashboard Outcomes Attainment Students as Agents Quantified Self Feedback Academic Tool Belonging Relationship Building Reduce Barriers Tutors Notes and Planning Student Retention Encourages Virtuous Behaviour Early Warnings Support Information
Timeline Experiments/Proof of Concept (2012-13) 3 months Pilot Programme (2013-14) 4 first year courses, 22 staff, 400 students Understand how staff & students use it Explore reactions to using the dashboard Test methodology does it work? Full Implementation (Start 2014-15) By Spring 2014, it was felt that there was sufficient benefit to consider full implementation, further evaluation conducted in Nov 2014 Jan 2015 Implemented in all 9 Schools Undergraduate & Postgraduate
The Dashboard
What does the dashboard do? Student Biographical Information Tutor Comments Staff View Student View Combined Engagement Measure Door swipes Library Loans VLE Use Dropbox Submissions Attendance NTU Student Dashboard Compares and Rates Engagement across Cohort Raises Alerts!!
Developed by external company www.solutionpath.co.uk
Student Engagement Rating - Cumulative
Student Engagement Rating week-week
Engagement Measured via Student Behaviour Proportionate use of Existing Data
Patterns of Student Withdrawal 1 st Yr Arts & Humanities 1 st Yr Social Sciences 1 st Yr Art & Design
Tutor Notes Arranged to see [student] as attendance was poor. We had an open and honest discussion about this and I explained to him the relationship between attendance and success! We talked about the barriers to his attendance, which related to too many late nights! Will meet again to review his attendance [tutor name] [student] has not attended my lectures for a few weeks. Emailed him to ask him to attend a one to one tutorial [tutor name]
Impact
Pilot Courses Outcomes Student feedback Recognise Benefits 93% Wanted to be Warned of Risk of Failure Feel Strongly Positive about the Dashboard But Have Concerns of Abuse Will this be used against me if I fail? and some unease about the range of people with access Staff feedback Strongly Positive Can see how they can use it in own practice Valuable to Understand what Happened to Students, Plenty of Examples of Driving up Engagement Time Spent Considerably less than 30 Minutes per Week
100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% Students progressing to second year of study by capped pre-entry A Level tariff and engagement rating low satisfactory good high 0-220 points 23.7% 82.9% 95.4% 89.8% 221-240 points 34.7% 87.4% 95.6% 95.7% >240 points 40.9% 92.6% 95.3% 94.1%
Students progressing to next year of study by WP status and engagement rating 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% low satisfactory good high WP (n-2,780) 25.4% 82.5% 92.6% 94.3% Not WP (n=7,643) 39.9% 89.8% 96.0% 96.3%
Overall Impact Engagement is the Strongest Predictor of Progression Very closely tied with Entry Qualifications for Predictor of Final Degree Attainment Groups with Historically Poorer Progression and Attainment have Different Engagement For these Groups Engagement is a Stronger Predictor than Demographics
Other Benefits Along the Way Single Source of Data for Tutors and Students Valuable Research Data for Improving Learning and Teaching Sharing of Good Practice Rich Data for Supporting Students from Priority Backgrounds Management Information and Strategic Planning More joined up Academic and Student Support
Going Forward Dashboard Methodology Works Continual Improvement Attendance and Access to Electronic Resources Further UI Developments Institutional processes Making Time for Interventions and Developing the Personal Tutor Sharing Good Practice Connecting the Right People to Students at the Right Time Not Duplicating Existing Systems Impacting upon Student Behaviours Crucial - need to work on what do I do now? Some evidence that the most engaged students compete against one another, but how do we get less engaged students to engage? Risk of Shifting Blame Does this pathologise student behaviour/ create a new deficit model? What is our responsibility for creating more engaging learning experiences?
Thank You Questions?