WHITE PAPER Instructor Feedback Writ Large: Student Perceptions on Feedback Insights into Understanding Reception of Feedback from 2,000 Students and Educators Prevent Plagiarism. Engage Students. turnitin.com
Table of Contents 1.0 Introduction... 3 2.0 Survey Goal and Background... 5 3.0 Key Findings and Implications... 6 4.0 Recommendations... 9 5.0 Next Steps: Need for an Expanded Study... 10 6.0 Appendix: Full Survey Results... 11 About Turnitin... 17 WHITE PAPER Instructor Feedback Writ Large: Student Perceptions on Feedback 2
1.0 Introduction In September 2014, Turnitin conducted an exploratory survey of over 2,000 students to examine students perceptions on instructor feedback. The survey s primary purpose was to identify what students generally think about the feedback they receive on their assignments. More specifically, the survey was interested in identifying the formats and types of feedback students consider to be the most effective, and in gaining insight into how often students receive the types of feedback they most desire. Alongside the student questionnaire, we conducted a parallel survey with over 100 educators to compare and contrast their perceptions on feedback with those of students. A 2013 Turnitin study 1 of instructor feedback looked less at the content of feedback. Rather, its focus was on the format of delivery, general categories of feedback (i.e. grammar/mechanics, thesis development, structure), and the frequency/ timing of feedback that students receive. Three key findings that emerged in the 2013 study include: Despite high rates of electronic submission, students do not report receiving electronic feedback at nearly the same rate. Students find feedback on thesis/development the most valuable, but prefer receiving feedback that emphasizes grammar/mechanics and composition/structure instead. Students depend on timely feedback, but don t always receive it: 28 percent of respondents reported that their instructors take 13+ days to provide feedback on their papers. 1 Turnitin. Closing the Gap: What Students Say About Instructor Feedback. Turnitin.com. April 2013. Accessed 12.1.2014. WHITE PAPER Instructor Feedback Writ Large: Student Perceptions on Feedback 3
Building on this earlier study, the September 2014 Student Perceptions on Feedback study sought to gain more detailed insight into student perception of the efficacy of the feedback that they receive. Initial findings from the 2014 study include: Students find face-to-face feedback very or extremely effective. 77 percent of students view face-to-face comments as very or extremely effective, but only 30 percent receive face-to-face very or extremely often. A high proportion of students view general-type comments very or extremely effective. But, a smaller percentage of educators feel the same. Even though 68 percent of students report receiving general comments very or extremely often, and 67 percent of students say this feedback type is very or extremely effective, only 33 percent of educators view general comments as very or extremely effective. Students more strongly prefer suggestions for improvement over praise or discouragement. In terms of the content contained in feedback, the greatest percentage of students found suggestions for improvement very or extremely effective, while the fewest percentage of students said the same for praise or discouragement. Students and educators differ on what constitutes effective feedback. Of the types of feedback in question, six of the eight categories exhibited a gap of perceived effectiveness between educators and students that was greater than 15 percent. Most notably, the greatest difference between educator and student responses occurs with general, overall comments about the paper and specific notes written in the margins. WHITE PAPER Instructor Feedback Writ Large: Student Perceptions on Feedback 4
2.0 Survey Goals and Background The purpose of the survey was to identify possible differences between what students deem to be effective forms of feedback, the feedback types they actually receive and the forms of feedback that educators say is effective. The explicit goals were to: Analyze the perceptions of feedback effectiveness by format Distinguish the perceptions of the most effective types of feedback Identify the most frequently received types of feedback The survey was conducted from September 25 29, 2014 using an online survey tool. A total of 2,401 respondents completed the survey in its entirety (with 3,224 respondents starting, but not completing it). The survey consisted of 10 multiple choice questions, most of which used a five-point, Likert scale. Survey Demographics Out of 2,401 respondents, over 58 percent of student respondents identified themselves as attending bachelor programs, 39 percent claimed to attend graduate programs, and three percent said they attend a high school program. Among all respondents, 31 percent reported that most of their classes (76 to 100 percent) are held online. Respondents by Attending Program Bachelor s 39% Program High 3% School Graduate 58% Program WHITE PAPER Instructor Feedback Writ Large: Student Perceptions on Feedback 5
3.0 Key Findings and Implications Key Findings: On Feedback Delivery On a scale from 1-5, how often have you received feedback in the following formats? On a scale from 1-5, how effective has the feedback been in the following formats? Feedback Formats Very or Extremely Often Feedback Formats Very or Extremely Typed comments electronically 60.15% Face-to-face (one on one conversation) 76.45% Written comments on paper 55.58% Typed comments electronically 69.73% Face-to-face (one on one conversation) 29.17% Written comments on paper 69.21% Comments recorded by voice 5.41% Comments recorded by video 34.00% Comments recorded by video 4.58% Comments recorded by voice 31.48% A majority of students consider face-to-face feedback to be very or extremely effective. But only 29 percent of respondents said that they received feedback face-to-face. Face-to-face feedback was viewed as the the most effective mode of feedback with 77 percent of students reporting face-to-face feedback as very or extremely effective. Yet, only 29 percent reported receiving that type of feedback from instructors. Nearly 70 percent of students reported that typed comments (received electronically) and written comments (on paper) were very or extremely effective. And, over 60 percent of respondents report receiving written and typed comments very or extremely often. Though 77 percent of students regard face-to-face feedback as very or extremely effective, more students received electronic, typed feedback (60 percent) and written comments (56 percent) on their papers very or extremely often. By comparison, only 39 percent of the surveyed educators found typed comments to be very or extremely effective. 31 percent of educators consider written comments on paper to be of the same level of effectiveness. WHITE PAPER Instructor Feedback Writ Large: Student Perceptions on Feedback 6
Implications The students preference for face-to-face feedback surfaced in this survey may be a reflection of their desire for more immediate and personalized instructor feedback an implication that is also brought to bear in the reported effectiveness of typed or written comments by students. As for the difference in effectiveness of typed or written comments noted by educator respondents, this difference may in fact be a reflection of the often-heard educator lament that students just don t read comments. It may be that students actually do read comments, but something interferes with their implementation of that feedback. Delving into more detail around how students consume feedback will be one aspect of a broader survey that is planned for launch in the spring of 2015. Key Findings: On Feedback Types This survey asked students to consider the following feedback types: General, overall comments about the paper Specific notes written in the margins Pointing out mistakes Pointing out what I did right Suggestions for improvement How well assignment met criteria and rubrics Praise or discouragement Of the types of feedback surveyed, the greatest percentage of students reported suggestions for improvement as very or extremely effective, while the fewest percentage of students said the same for praise or discouragement. Over 76 percent of students said suggestions for improvement were very or extremely effective, followed closely by 73 percent of students who reported specific notes written in the margins to be of the same level of effectiveness. 67 percent to 69 percent of students reported most of the other feedback types as effective. 57 percent of students said praise or discouragement was very or extremely effective. WHITE PAPER Instructor Feedback Writ Large: Student Perceptions on Feedback 7
On a scale from 1-5, how effective have you found the following types of feedback? (2,401 respondents) Feedback Types Very or Extremely Suggestions for improvement 75.95% How well assignment met criteria and rubrics 73.37% Pointing out what I did right 69.10% Pointing out mistakes 68.83% Specific notes written in the margins 68.55% Use of examples 68.16% General, overall comments about your work 67.02% Praise or discouragement 56.85% Nearly 68 percent of students receive general, overall comments on their paper very or extremely often. Closely following general, overall comments, 62 percent of students claimed to receive feedback on how well their assignments met criteria and rubrics very or extremely often. In order of frequency, students reported receiving the following feedback types very or extremely often. On a scale from 1-5, how often do you receive the following types of feedback? (2,435 respondents) Feedback Types Very or Extremely Often General, overall comments 68.43% How well assignment met criteria and rubrics 62.33% Suggestions for improvement 58.05% Pointing out mistakes 54.73% Specific notes written in the margins 50.99% Pointing out what I did right 48.75% Praise or discouragement 44.37% Use of examples 36.20% WHITE PAPER Instructor Feedback Writ Large: Student Perceptions on Feedback 8
By comparison, the greatest percentage of educators (66 percent) reported the use of examples to be very or extremely effective. The fewest percentage of educators (34 percent) said the same about general, overall comments. Implications: There is a clear disconnect between educators view of effective feedback and the type of feedback students report receiving. While students report receiving general, overall comments most frequently, educators view it as least effective. Furthermore, students report receiving use of examples least frequently even though educators view it as most effective. This disconnect could be a reflection of the time pressure and challenge that educators face in leaving more targeted feedback for every student, such as when using examples. Turnitin plans to further explore this topic in a future survey study. On a scale from 1-5, how effective have you found the following types of feedback? Feedback Types Educators Students Gap General, overall comment 33.66% 66.99% 33.33% Specific notes written in the margins 46.81% 73.40% 26.59% Pointing out mistakes 44.44% 68.96% 24.52% Pointing out what I did right 47.87% 68.16% 20.29% Suggestions for improvement 59.22% 76.04% 16.82% How well assignment met criteria and rubrics 52.43% 68.67% 16.24% Use of examples 65.59% 69.14% 3.55% Praise or discouragement 53.47% 56.92% 3.45% There was a wide gap in perceived effectiveness in six of the eight categories that were surveyed. Educators and students don t always agree in their perceptions of what constitutes effective feedback. In fact, educators and students differ in their responses to the effectiveness in six of the eight types of feedback by a margin of over 15 percent. On average, educators surveyed rated the effectiveness of feedback across types less than how students rated feedback. 50 percent of educators and 69 percent of students found all feedback types, on average, to be very or extremely effective. The difference in rating may underscore the value that students see in merely receiving feedback. Of note are the higher ratings that students reported for the types of feedback that are more specific and detailed. The fact that the largest number of students consider suggestions for improvement to be very or extremely effective lends additional support to this assertion and also strongly suggests that students are looking at the feedback they receive as an extension of course or classroom instruction. WHITE PAPER Instructor Feedback Writ Large: Student Perceptions on Feedback 9
4.0 Recommendations Comments recorded by voice or audio may be a time-saving substitute for face-to-face feedback. Only five percent of student respondents reported receiving voice or video comments at the same frequency as that reported by students who report receiving face-to-face feedback very or extremely often (30 percent). As a way to negotiate time pressures and still be able to provide more personalized feedback, educators might consider leveraging the use of recorded voice or audio comments to provide feedback on student work. Many grading platforms and learning management systems (LMS) offer this feature as part of their services. Expose students to different types of effective feedback, and evaluate what is helpful for them. From the study, there was a clear correlation between exposure to feedback and perceived effectiveness of feedback. Consider taking time to discuss with students the types of feedback that you typically provide and why and also ask students to share with you what types of feedback they have found to be most effective. On that note, the definition of effective feedback for educators and students is always going to be informed by context namely the context of your course or class. Administer a survey in your course to find out what types of feedback students think would improve their work. Use this as an opportunity to drill down to what effective feedback means for your students. Having a view into your students needs in regards to feedback will help you communicate effectively with your students. To that end, there is a quick and easy survey that you can use to analyze feedback effectiveness in your classroom (see appendix). WHITE PAPER Instructor Feedback Writ Large: Student Perceptions on Feedback 10
5.0 Next Steps: Need for an Expanded Study Participate and share your students voices. Turnitin is launching a more detailed study in the spring of 2015 to better understand how students respond to feedback. Some questions we hope to answer include, What makes feedback effective? and What aspects of instructor feedback do students understand? Encourage your students to participate by sharing the following link to the student survey: https://www.surveymonkey.com/s/tii-effective-feedback. WHITE PAPER Instructor Feedback Writ Large: Student Perceptions on Feedback 11
6.0 Appendix Full survey responses: Q1: Are you a student currently enrolled in: Q2: What year of schooling are you? Answer Options Response Count Answer Options Response Count Bachelor's Program 926 Freshman 132 Graduate Program 1390 Sophmore 238 High School 65 Junior 254 Senior 517 Q3: Roughly what percentage of your classes are conducted online? Answer Options Response Count 0-25% Classes Taken Online 1115 26-50% Classes Taken Online 313 51-75% Classes Taken Online 213 76-100% Classes Taken Online 735 Q4: On a scale from 1-5, how often have you received feedback in the following formats? Answer Options 1 Not at All 2 Not Very Often 3 Sometimes 4 Very Often 5 Extremely Often Response Count Face-to-Face 24.71% 20.20% 25.92% 17.44% 11.72% 2465 Written comments on paper 12.44% 11.20% 20.77% 29.27% 26.32% 2508 Typed comments electronically 5.29% 11.28% 23.29% 26.52% 33.63% 2572 Comments recorded by voice 75.67% 11.37% 7.55% 3.27% 2.14% 2384 Comments recorded by video 80.73% 8.90% 5.79% 2.69% 1.89% 2382 WHITE PAPER Instructor Feedback Writ Large: Student Perceptions on Feedback 12
Q5: How would you prefer to access these types of feedback? Answer Options Desktop Tablet Cell Phone Response Count Written/typed comments 1933 457 184 2574 Comments recorded by voice 879 380 523 1782 Comments recorded by video 1047 490 231 1768 Q6: On a scale from 1-5, how effective has the feedback been in the following formats Answer Options 1 Not at All 2 Not Very 3 Somewhat 4 Very 5 Extremely Response Count Face-to-Face 3.93% 3.58% 16.05% 33.86% 42.59% 2038 Written comments on paper 1.46% 4.59% 24.73% 41.87% 27.34% 2264 Typed comments electronically 1.06% 4.23% 24.99% 40.27% 29.46% 2461 Comments recorded by voice 17.73% 19.47% 31.32% 20.63% 10.85% 1207 Comments recorded by video 18.52% 17.83% 29.65% 20.35% 13.65% 1150 Q7: On a scale from 1-5, rate each of these subjects based on the amount of feedback you ve received in the past. Subject 1 None at All 2 Low Amount 3 Some 4 High Amount 5 Extremely High Amount Response Count English 123 91 395 701 646 1956 History 299 266 484 363 213 1625 Mathmatics 272 251 506 429 268 1726 Business/Economics 254 195 422 429 305 1605 Visual/Performance Arts 395 230 312 224 166 1327 Psychology 287 205 425 416 378 1711 Physical Sciences 284 190 426 443 378 1721 Engineering 442 141 225 181 182 1171 Computer Science 361 195 325 273 239 1393 WHITE PAPER Instructor Feedback Writ Large: Student Perceptions on Feedback 13
Q8: On a scale from 1-5, rate each of these subjects based on the effectiveness of the feedback you ve received in the past. Subject 1 Not at All 2 Not Very 3 Somewhat 4 Very 5 Extremely Response Count English 63 64 417 695 669 1908 History 145 185 435 432 302 1499 Mathmatics 141 164 430 518 367 1620 Business/Economics 123 124 433 449 331 1460 Visual/Performance Arts 191 159 318 292 180 1140 Psychology 151 127 438 484 394 1594 Physical Sciences 154 146 404 493 385 1582 Engineering 209 107 241 219 211 987 Computer Science 167 120 335 325 276 1223 Q9: On a scale from 1-5, how often do you receive the following types of feedback? Answer Options 1 Not at All 2 Not Very Often 3 Sometimes 4 Very Often 5 Extremely Often Response Count General, overall comments about your work 1.72% 4.94% 24.92% 41.04% 27.39% 2388 Specific notes written in the margins 6.41% 10.28% 32.31% 33.43% 17.56% 2324 How well assignment met criteria and rubrics 3.31% 7.67% 26.69% 37.29% 25.04% 2360 Praise or discouragement 6.12% 14.63% 34.87% 29.70% 14.68% 2337 Suggestions for improvement 2.49% 9.57% 29.89% 38.11% 19.94% 2372 Use of examples 10.34% 22.78% 30.68% 22.91% 13.29% 2340 Pointing out mistakes 2.40% 11.19% 31.68% 35.38% 19.35% 2377 Pointing out what I did right 6.25% 14.58% 30.43% 31.62% 17.13% 2353 WHITE PAPER Instructor Feedback Writ Large: Student Perceptions on Feedback 14
Q10: On a scale from 1-5, how effective have you found the following types of feedback? 1 Not at All 2 Not Very 3 Somewhat 4 Very 5 Extremely Response Count General, overall comments about your work 1.53% 5.81% 25.64% 37.39% 29.63% 2356 Specific notes written in the margins 1.62% 4.00% 21.01% 40.58% 32.79% 2223 How well assignment met criteria and rubrics 1.52% 5.75% 24.17% 38.81% 29.75% 2296 Praise or discouragement 3.46% 9.49% 30.20% 35.17% 21.69% 2255 Suggestions for improvement 1.09% 4.31% 18.66% 41.24% 34.71% 2299 Use of examples 2.42% 6.20% 22.29% 37.37% 31.72% 2194 Pointing out mistakes 1.82% 6.06% 23.29% 39.83% 29.00% 2310 Pointing out what I did right 2.07% 5.37% 24.39% 38.57% 29.59% 2271 WHITE PAPER Instructor Feedback Writ Large: Student Perceptions on Feedback 15
WHITE PAPER Instructor Feedback Writ Large: Student Perceptions on Feedback 16 Name
About Turnitin Turnitin is the global leader in evaluating and improving student work. The company s cloud-based service for originality checking, online grading and peer review saves instructors time and provides rich feedback to students. One of the most widely distributed educational applications in the world, Turnitin is used by more than 10,000 institutions in 126 countries to manage the submission, tracking and evaluation of student papers online. Turnitin is a service of iparadigms, LLC 1111 Broadway, 3rd Floor Oakland CA, 94607 USA USA/Canada: 866-816-5046 International: 011 44 845 643 0105 sales@turnitin.com turnitin.com WHITE PAPER Instructor Feedback Writ Large: Student Perceptions on Feedback 17