Face-to-Face versus Online Delivery of a Graduate Course in Quality Assurance

Paper ID #7183 Face-to-Face versus Online Delivery of a Graduate Course in Quality Assurance Dr. Julie Dunston, Southern Illinois University, Carbondale Dr. Julie K. Dunston is an Associate Professor in the Department of Technology and is a Certified Six Sigma Black Belt (CSSBB) through the American Society for Quality (ASQ). Her research interests include intelligent manufacturing; process modeling and control; and quality improvement with the integration of AI and statistical methods. She has conducted research with Ford Motor Company in the area of intelligent manufacturing of composites and has worked with John Deere on projects utilizing neural networks for predicting fatigue life and implementing Six Sigma for the development of torque standards. Her teaching interests include robotics, engineering statistics, quality assurance, and Six Sigma. Dr. Garth V Crosby, Southern Illinois University Carbondale Dr. Garth V. Crosby is an assistant professor in the Technology Department at Southern Illinois University Carbondale. He obtained his M.S. and Ph.D. degrees from Florida International University in Computer Engineering and Electrical Engineering, respectively. Dr. Crosby s primary interests of research are wireless networks, wireless sensor networks, network security and active learning strategies for STEM. He has served as a reviewer for several conferences, magazines and journals publications, including IEEE INFOCOM, and IEEE Transactions on Systems, Man and Cybernetics. He is a senior member of the Institute of Electrical & Electronics Engineers (IEEE), a member of the National Society of Black Engineers (NSBE), and Eta Kappa Nu. c American Society for Engineering Education, 2013

Face-to-Face versus Online Delivery of a Graduate Course in Quality Assurance To address the industry requirement for workers with advanced technical knowledge our university has begun to offer its Manufacturing Systems MS degree program online. In this paper, we describe the journey from course development to delivery of an existing course, Quality Assurance, in this new online medium. The material is presented from the perspective of the course developer and instructor. We discuss the necessary skills needed for the course developer to properly create an online course that is pedagogically sound, and how we overcame the various technical obstacles inherent in the transition of the course delivery from face-to-face to online. We present the course content and discuss various options that we think can be utilized to enhance the presentation of the material. The course was delivered in a regular semester time period with two concurrent sections: (i) an on campus face-to-face delivery, and (ii) an online format. The on-campus and online sections each had an enrollment of 15 students. Similar (almost identical) content was delivered to both sections at the same weekly pace. Similar assignments and quizzes/tests were given to the two groups. The main difference between both groups of students was that the off-campus students were all working full-time as they take the course whereas most of the on-campus students were not employed full-time. We conduct a quantitative study of the performance of the two groups. The performance was based on test results. A qualitative study was also conducted to assess perceptions of the two groups as it relates to the effectiveness of delivery of the course material. The basic research questions addressed in this study are: (i) do on-campus students perform better than online? and (ii) are there any differences in learning and perceptions between the two groups? In concluding, we discuss lessons learned and the way this study has impacted future delivery of the course. Introduction According to the U.S. Department of Education, about 0.8 million, or 22 percent, of all postbaccalaureate students took distance education courses in 2007 08. The percentage of postbaccalaureate students who took their entire program through distance education (9 percent) was higher than the percentage at the undergraduate level 1. The 2011 Survey of Online Learning conducted by the Babson Survey Research Group and the College Board identifies that the rate of growth in online enrollments is ten times that of the rate in all higher education. Thirty-one percent of higher education students now take at least one course online. Despite the fact that growth rates in online enrollments have declined slightly from previous years, the survey group sees no evidence of a dramatic slowdown in online enrollments in the near future. A possible contributing factor to this statement is the strong support for online course development among upper administration; 65% of higher education institutions state that online learning is a critical part of their long-term strategy 2. With the increasing popularity of online courses, there have been a multitude of recent studies that examine the effectiveness of this mode of delivery and the impact on student satisfaction and performance. A study by Beqiri, et. al. 3 provided some insights into factors that impact students satisfaction with online courses. The research concluded that online courses might be better received when offered at the graduate level (involving adult populations) than undergraduate

level. Since course familiarity seems to play a significant role in student satisfaction, it was recommended that core and prerequisite courses be offered in a more traditional format. A final conclusion was that schools and universities lean toward a blended course-delivery mode versus 100% online delivery. The shortcoming of this particular study, however, is that it does not address the fact that a major factor in the increasing trend in online course offerings is a result of the need for flexibility for students who have geographical or scheduling constraints. In these instances, it is important to understand how to optimize the product that students are receiving. For example, Kim and Moore 5 determined the factors that affect student satisfaction and learning experiences in online delivery. Results indicated that student interaction with classmates and the instructor are critical in improving students understanding of course material and the broader context of the class as a whole. Strategies for improving interaction could include online office hours and providing timely feedback on assignments. Similar findings were discovered in the results of a survey indicating that student interaction, quality and timely interaction between student and professor, consistent course design, technical support availability and flexibility are important factors in creating online courses that result in high student satisfaction 6,9. Although student performance has been shown to be comparable in several studies 7,8,11, other research has shown that student satisfaction, in general, is less in online courses than in traditional courses 7,8,10. In this paper, we assess the attitude and perception of both online and face-to-face students towards online courses. By examining these differences, it is possible to shed some light on how the needs of online students can be uniquely served and how satisfaction in online courses can be improved. Course Overview Data collected for this study was taken from two class sections of MFGS510, Quality Assurance. One section was strictly online and the other section was offered as a traditional face-to face delivery format in which the class met for one 3-hour session once a week. The enrollment in the online and face-to-face sections was 10 and 13, respectively. Student assignments between the two sections were identical and included case studies, discussions, and problem-solving exercises. For the online section, students had access to asynchronous recorded lectures and step-by-step problem-solving videos. During the course of the semester, 3 examinations were administered through the learning management system (LMS) to both sections of students. Topics were organized according to modules, with a 10-point quiz given at the end of each module to assess student learning and to provide students with reviewable material for the exams. Survey A survey was developed for both class sections to evaluate students perceptions and attitudes regarding the course and to compare the results between the online and face-to-face sections. There were 18 questions total, with 16 assessed using a Likert scale (ranging from strongly agree to strongly disagree). Two remaining questions asked students how many hours per week were spent on the course and how many hours per week were spent using the LMS. It should be noted

that the students in the online section were off-campus students only who were enrolled full time in the master s program and taking more than one online course over the span of the semester. Out of 10 students registered in the course, all were employed full-time. Additionally, all students in the face-to-face section were full-time on-campus students who may or may not have been employed. Some of these students were also taking an online course within the master s program in the department. Overall, the questionnaire was developed to evaluate student attitudes and perceptions toward the course as it relates to course expectations and organization; time spent on course; and attitudes and perceptions of online learning among students enrolled in the online section versus students enrolled in the face-to-face section. To compare performance of the students in the two sections, the mean percentage on the three exams was calculated as well as the overall grade percentage in the course. Table 1 provides a comparison of the time spent on the course and time spent navigating through the LMS for both online and face-to-face students. Based on the t-test results, there was a significant difference between the time spent on the course for online students versus face-toface students, with the former spending more hours per week on the course. Face-to-face students had a 3-hour lecture every week in which material was presented to them during class time and there was an opportunity to have questions answered regarding the topic covered. Online students listened to recorded lectures asynchronously and, if needed, repeatedly to gain understanding of the material. This could account for the higher number of hours per week spent on the course for online students. Online students spent more time in the LMS than face-to-face students, based on averages. However, the variation in time spent in the LMS was also large across the spectrum of online students. This is not surprising since some of the online students were taking courses online for the first time and would have had a learning curve with the LMS. Table 1. Comparison of time spent on course/lms for face-to-face and online and face-to-face Hours per Week Face-to-Face Online Spent on course 4.227 2.306 10.750 8.396-2.476 0.0241 Spent in LMS 5.182 5.091 13.813 13.711-1.930 0.0705 The performance on exams and overall performance between online and face-to-face students is shown in Table 2. Most striking was the difference in performance on the third exam, with faceto-face students performing better than online students. Content on exam 3 consisted exclusively of statistical calculations that students had to perform. Even though a practice exam was provided, several students in the online section did not access this review material in preparation. The instructor was aware of a few students who had work travel prior to the exam, which may have prevented them from allocating sufficient time to review.

Table 2. Comparison of mean percentage of exams and overall percentage between face-to-face and online students Face-to-Face Section Online Section Mean Exam 1 Performance (%) 82.9 86.8-2.265 0.034 Mean Exam 2 Performance (%) 90.3 87.4 1.591 0.127 Mean Exam 3 Performance (%) 90.6 62.0 4.852 0.0001 Mean Overall Performance (%) 89.7 87.5 0.876 0.391 To assess the differences in survey responses between the two sections of the course, points were assigned to the Likert scale, with a strongly agree response evaluated at a 5 and a strongly disagree assigned a value of 1. In order to determine if there were differences between groups, independent-sample t-test were performed on 16 questions related to course organization and expectations; course workload and time spent; attitude toward online learning; and accommodation of online courses. Significant differences between sections were found on three of the questions, two on attitude towards online learning and one on accommodation of online courses (Tables 3-6). Table 3. Course organization and expectations for face-to-face and online students Question Traditional Online 1. The course was well organized 4.545 0.522 4.625 0.518-0.331 0.745 2. My expectations for the course were met 4.455 0.522 4.625 0.518-0.703 0.492 Table 4. Course workload and time spent for face-to-face and online students Question 3. The amount of work required in the course was appropriate 6. In comparison to my initial expectations, I spent more time on the course than anticipated 7. The time I spent on the course was sufficient to meet or exceed the course requirements Traditional Online 4.273 0.647 4.250 0.707 0.074 0.942 3.364 0.924 3.875 0.991-1.155 0.264 4.091 0.831 3.625 1.061 1.075 0.297

Table 5. Attitude toward online learning for face-to-face and online students Question 8. I believe that I can learn the same amount in an on-line course as in a traditional course 9. I believe that I can make the same grade or better in an on-line course as in a traditional course 10. I believe that I can learn more or would learn more through on-line material than through lectures 11. I (would) prefer on-line courses to traditional courses. 12. On-line courses (would) make me uncomfortable 13. I (would) feel comfortable taking on-line courses 17. It is/(would be) difficult to contribute to class discussions in an online course * Statistically significant (α = 0.05) Traditional Online 2.727 1.272 3.625 1.188-1.561 0.137 2.727 1.009 3.500 1.069-1.609 0.126 2.545 1.368 2.875 0.991-0.579 0.570 1.727 0.647 3.250 1.035-3.954 0.001* 2.818 0.874 2.750 0.886 0.167 0.870 3.455 1.036 4.000 1.069-1.117 0.279 4.272 0.647 3.250 1.282 2.289 0.035* Table 6. Accommodation of online courses for face-to-face and online students Question Traditional Online 14. On-line courses save me time 3.727 0.905 3.625 1.061 0.226 0.824 15. On-line courses work well with my 3.909 0.701 4.375 0.518-0.003 0.998 schedule 16. On-line courses (would) enable me 2.909 1.300 4.375 0.518-3.002 0.008* to attend classes more frequently than traditional courses. 18. On-line enables (or would enable) 3.545 0.820 4.000 0.535-1.367 0.190 me to take more courses than the traditional methodology in a year. * Statistically significant (α = 0.05) There was a statistically significant difference in the preference of taking an online course for students in the online section versus students in the face-to-face section of the course. When asking the question, I (would) prefer on-line courses to traditional courses, the average response of online students was 3.25 ( equating to a response between agree and neither agree or disagree ) compared to an average response of 1.727 for face-to-face students (equating to a response between disagree and strongly disagree. It should be noted that all the students taking the online class had taken undergraduate classes in a traditional format and, therefore, had

experience with both formats. It is apparent that the students taking the face-to-face class have a less favorable attitude towards online courses; the majority of these students had no prior experience taking any online course. Another significant difference between online and face-to-face student responses was regarding the question, It is/(would be) difficult to contribute to class discussions in an on-line course. The results are interesting in that the online student response was less favorable with an average response of 4.272 (between strongly agree and agree ) compared to the face-to-face student response averaging 3.250 (between agree and neither agree or disagree ). The final significant difference was the response to the question, On-line courses (would) enable me to attend classes more frequently than traditional courses. The average response of online students was 4.375, which equates to a selection between agree and strongly agree. On the contrary, face-to-face students have an average response of 2.909 (falls between disagree and neither agree or disagree ). Online Course Development Recommendations Although the Quality Assurance course has been offered in the master s program for many years, the online section was developed in fall 2011 and offered for the first time in the spring 2012 semester. The instructor developing the online course had the advantage of having taught the face-to-face course and refining the content over a span of more than a decade. In addition, since the online section follows the same schedule as the face-to-face (16 weeks), the instructor was able to set assignments and deadlines that were appropriate and realistic for the online students. In cases where courses are taught online only, it is important to be aware of an adequate scope of content and timeframes that are reasonable for students. Particularly since it seems that online students more predominantly have added responsibilities, such as family and careers, that they are dealing with. Based on the experiences of the instructor in developing and teaching the online section, and feedback received from online students, the following recommendations are given: Personalize content within the course so that students have a flavor of who you are as an instructor; this can be accomplished by being actively involved in discussions; voice recordings of lectures, problem-solving examples, etc.; providing timely feedback on assignments; and opening the course with an ice-breaker. Try to relate material in the course, as much as possible, to real-world experiences and examples. A significant number of graduate students taking online courses have work experience or are currently working and value content that is directly applicable to their jobs. Set clear guidelines for discussion questions and interact regularly so that students don t get off topic, which creates a rapport among students but does not contribute to the intellectual nature of the assignment. Utilize case studies that incorporate multiple topics within the course so that students can better see the big picture. When explaining difficult concepts be concise and thorough. Use multiple examples to explain the topic and provide additional sources for students to access on their own, if

needed. Since online students do not have the luxury of asking questions real-time or interacting with other students in the course, it is important to equip them with the tools needed to enhance self-learning. Be flexible. Students taking online courses value the inherent flexibility of this method of delivery and their satisfaction is enhanced if the instructor offers flexibility and sensitivity to the unique challenges they face. Do not expect the online course to be perfect the first time. It is important to survey students at the end of the course to determine what techniques are effective and what areas need improvement. Conclusion A comparative study between online and face-to-face students was performed. Many of the responses were comparable between online and face-to-face students. The analysis, however, found that online students tend to prefer the online format over face-to-face students who, in general, had never taken an online course but did not view the format as favorable. Online students also found Web-based delivery to be more favorable in terms of regular attendance in the course, possibly due to the fact that job commitments might prevent them from attending class at times. Finally, online students felt that it was more difficult to contribute to discussions than their face-to-face counterparts. Since past studies indicate that student interaction, including discussions, is an important part of creating satisfied students, innovative ways for involving students in the course should be developed. This might involve having regular times every week for students to log on and communicate on a particular topic. Since it becomes difficult to find a time when all students are available, multiple times could be arranged for synchronous learning with smaller groups of students. In addition, experiences that the instructor faced in the development of the online section were presented. A list of recommendations when developing and teaching online courses was also provided. References 1. U.S. Department of Education, Center for Education Statistics. The Condition of Education 2011(NCES 2011-033), Indicator 43, 2011, http://nces.ed.gov/fastfacts/display.asp?id=80, Accessed December 30, 2012. 2. Allen, I.E. and Seaman, J., Going the Distance: Online Education in the United States, 2011, Babson Survey Research Group, http://sloanconsortium.org/publications/survey/going_distance_2011, 2011. 3. Beqiri, M.S., Chase, N.M., and Bishka, A., (2009). Online Course Delivery: An Empirical Investigation of Factors Affecting Student Satisfaction, Journal Of Education For Business, Vol. 85, No. 2, pp. 95 100, 2009. 4. Koenig, R., A Study in Analyzing Effectiveness of Undergraduate Course Delivery: Classroom, Online And Video Conference From A Student and Faculty Perspective, Contemporary Issues in Education Research, Vol. 3, No. 10, pp. 13 25, October 2010. 5. Kim, K. S., and Moore, J. L., Web-based learning: Factors affecting students satisfaction and learning experience, 2005, Retrieved December 30, 2010, from http://www.firstmonday.org/issues/issue10_11/kim/index.html 6. Young, A. and Norgard, C., Assessing the quality of online courses from the students' perspective, The Internet and Higher Education, Vol. 9, Issue 2, pp. 107-115, 2006.

7. McFarland, D. and Hamilton, D., Factors Affecting Student Performance and Satisfaction: Online Versus Traditional Course Delivery, Journal of Computer Information Systems, Vol. 46, Issue 2, p. 25, Winter 2005/2006. 8. Summers, J., Waigandt, A., and Whittaker, T., A Comparison of Student Achievement and Satisfaction in an Online Versus a Traditional Face-to-Face Statistics Class, Innovative Higher Education, Vol. 29, No. 3, pp. 233-250, 2005. 9. Tudor, G. Teaching Introductory Statistics Online Satisfying the Students, Journal of Statistics Education, Vol. 14, No. 3, 2006. 10. Tallent-Runnels, M.K., Thomas, J.A., Lan, W.Y., Cooper, S., Aheru, T.C., Shaw, S.M., and Liu, X., Teaching Courses Online: A Review of the Research, Review of Educational Research, Vol. 76, No. 1, pp. 93-135, 2006. 11. Dutton, J., Dutton, M., Perry, J., Do Online Students Perform as Well as Lecture Students? Journal of Asynchronous Learning Networks, Vol. 6, No. 1, July 2002.