MyLab Economics educator study examines quiz and exam scores in an online course at Normandale Community College School Name Normandale Community College, Bloomington, MN Course name Principles of Macroeconomics Course format Online Timeframe Spring 2017 Submitted by Brooks Herrboldt, Instructor Results reported by Candace Cooney, Pearson Customer Outcomes Analytics Manager Course materials MyLab Economics with Macroeconomics by Hubbard and O Brien Key Findings Data indicate a strong, positive correlation between MyLab quiz scores and MyLab exam scores. Students with MyLab quiz scores above average earned MyLab exam scores eight percentage points higher than students who scored below average. 97 percent of respondents on an end-of-semester survey agreed that their understanding of the course material increased as a result of using MyLab. Setting Locale: large, urban, public, two-year community college located south of Minneapolis Enrollment: more than 14,000 credit students Full-time students: 42 percent Full-time retention rate: 55 percent First-time, full-time students: 17 percent Three-year graduation rate: 16 percent Faculty-student ratio: 25:1 Age: 26 percent of credit students are 25 or older Gender: 55 percent of credit students are female Diversity: 38 percent of credit students are students of color 1
About the Course Brooks Herrboldt has been teaching for approximately 14 years, spending the last 12 years at Normandale, where he has been teaching Micro- and Macroeconomics. Principles of Macroeconomics is a one-semester, three-credit course which enrolls about 1,100 students per year. Online sections typically enroll 35 students while face to face sections seat approximately 45 students. The course is required of business and economics students, and is also a general education requirement for students in political science, history, etc. Principles of Macroeconomics is a companion course with Principles of Microeconomics, and together the courses introduce students to the economic way of thinking. The course covers theories that explain the overall performance of the economy, measurement of national income, unemployment, and inflation, and the role of money and the banking system. Additional topics include policies that stabilize the economy and promote economic growth, and international trade and finance within the framework of the global economy. Upon successful completion of this course, students will be able to: Explain the implications of scarcity for society and decision-making; Apply the model of supply and demand to analyze market behavior; Interpret macroeconomic indicators; Analyze the causes and consequences of unemployment, inflation, and economic growth; and Evaluate the effectiveness of using monetary and fiscal policy to achieve macroeconomic goals. Challenges and Goals About ten years ago, Normandale began offering economics courses online. A challenge for offering the course online was providing engaging, interactive tools that promote personal responsibility for learning and motivate students to participate in the course. Additionally, creating an online course from scratch can be time consuming and expensive, with a need for PowerPoints to explain concepts, hands-on activities such as simulations and videos, graphics, critical thinking activities, and assessments to certify learning gains. Knowing that publishers have filled this gap by using their vast resources and textbook content, Herrboldt adopted MyLab Economics in 2007 for his sections. Implementation Learning Management System Integration The Pearson Learning Management System (LMS) integration service gives students and instructors easy access to MyLab from their existing school LMS. Herrboldt chose to integrate his MyLab course with Desire2Learn (D2L) for the following reasons: Single sign-in process students are ready to work in MyLab on the first day of class; and Content linking ability to link to MyLab directly from D2L. 2
Herrboldt s students have just one access code and a single sign-on process instead of the need to log in to D2L and additionally sign in to MyLab. This results in a simple way for students to start their MyLab assignments, ensuring that they are ready to work from the first day of class. On a voluntary, end-of-semester survey (69 percent response rate) in Spring 2017: 93 percent of students strongly agreed or agreed that they were able to access MyLab through D2L and appreciated not having a separate, second log in and password for MyLab. MyLab Economics is required; the program is used by students working at home for understanding content, homework assignments, and assessments. As the course instructor, Herrboldt s role is to assign content and homework in MyLab and provide remote support to students using the program at home. He anticipates that students will spend approximately 2 3 hours per week working in MyLab, and students confirmed this on the end-of-semester survey 46 percent of students said they spent 2 3 hours per week working in MyLab Economics, while an additional 34 percent of students said they spent more than three hours working in the program. MyLab Economics Herrboldt encourages his students to utilize all the study tools in MyLab, even those not required. In particular, he urges students to view two videos the first week of the semester that explain how to get the most out of the etext and the learning aids in MyLab: the Enhanced etext Overview video and the MyLab Learning Aids video. Students are encouraged to use the etext, as Herrboldt has embedded self-created narrated and annotated PowerPoint slides and other video clips into the etext where content appropriate. This allows for seamless reading and processing of all chapter material. In order to help his students understand economics, Herrboldt assigns problem sets in MyLab as weekly homework for skills practice and applying the textbook material. There are no points associated with these assignments, but students are required to complete the homework problems with a minimum score of 70 percent before the MyLab quiz will open. Students have unlimited attempts on homework problems, given that practice is the ultimate goal. On the end-of-semester survey, 100 percent of students said they appreciated having unlimited attempts on MyLab homework assignments to help prepare for the MyLab quizzes. One student summed up the comments of many, noting, A benefit of the MyLab was the multiple tries on homework, to make sure I understood things correctly. Learning aids are turned on during these homework assignments to provide just-in-time support as students practice newly acquired skills. On the end-of-semester survey, 51 percent of students said they usually or always use the learning aids when unable to start or complete a homework problem in MyLab, and an additional 40 percent indicate they sometimes use the aids. In particular, students on the survey commented on the Help Me Solve This tool: When not knowing an answer on homework or not knowing how to do a problem, the Help Me Solve This option really helped me understand it. I liked having the Help Me Solve This option, I am a person who learns better when being shown step-by-step. Students are also assigned a short video clip with questions in D2L; each question is worth five points and the two lowest scores are dropped for final score calculation. Herrboldt also assigns a 3
ten-point quiz weekly in MyLab, and students have three attempts at completion with no time limit. The highest quiz score is recorded in the gradebook and the two lowest quiz scores are dropped for final score calculation. MyLab assignments are due Sunday night at 11 p.m. and no late work is accepted for any reason. Four exams in MyLab comprise the additional summative course assessments. Each exam consists of 33 questions and is worth 100 points Most questions are multiple-choice but the exam may also include between three and eight MyLab questions. The exams cover all course content: assigned reading, MyLab assignments, MyLab quizzes, PowerPoint presentations, MyLab problem sets, MyLab video questions, and instructor-created videos. Students have 60 minutes for completion and just one attempt per exam. No make-up exams are given, but students with a valid or prearranged exam absence are required to take a comprehensive final exam covering parts 1 3 of the course, as well as complete the fourth regularly scheduled exam. Assessments 71% MyLab exams (4) 18% MyLab quizzes (13) 11% Video questions (13) Results and Data Figure 1 is a correlation graph; correlations do not imply causation but instead measure the strength of a relationship between two variables, where r is the correlation coefficient. The closer the r value is to 1.0, the stronger the correlation. The corresponding p -value measures the statistical significance/strength of this evidence (the correlation), where a p -value <.05 shows the existence of a positive correlation between these two variables. A strong positive correlation exists between average MyLab quiz scores and average MyLab exam scores, where r =.72 and p <.05. Additionally, the correlation of average video assignment scores to average MyLab quiz scores (not depicted here graphically) is strong, where r =.71 and p <.05. For students, the formative MyLab assignments are intended to help them identify where they are in terms of successfully completing the summative quizzes and exams. Empirically, Herrboldt suggests that the MyLab assets do have a positive impact on quiz and exam scores, noting that students who spend time on homework assignments and use the Study Plan are generally successful. 4
Correlation between MyLab quiz score and exam score Figure 1. Correlation between Average MyLab Quiz Score and Average MyLab Exam Score, Spring 2017 ( n =99) Students were divided into two groups based on the average MyLab quiz score. Students who earned quiz scores above average earned higher average exam scores (figure 2), and the difference is statistically significant. Average MyLab quiz score: 77 percent Students who scored equal to or above the average MyLab quiz score earned average exam scores 8 percentage points higher than students who earned a quiz score below the average. Results of a t -Test show that students with a quiz score above average (average = 77 percent) earned higher average exam scores than students with a quiz score below average, where t (183) = -3.29 and p <.05, indicating that this difference is statistically significant. Average MyLab quiz and exam scores Figure 2. Average MyLab Quiz Score and Average MyLab Exam Score, Spring 2017 ( n =99) 5
Figure 3 shows that students who earned higher average MyLab quiz scores also earned higher overall final course grades. Students exhibiting course success by earning an A, B, or C as their final course grade had average MyLab quiz scores of 87 percent, which is 37 percentage points higher than students who earned a D or F as the final course grade (50 percent average MyLab quiz score). Average MyLab quiz scores and final course grades Figure 3. Average MyLab Quiz Scores and Final Course Grades, Spring 2017 ( n =99) The Student Experience Responses from the Spring 2017 end-of-semester, voluntary survey of Herrboldt s students indicate that the majority of responding students recognize the value of MyLab Economics. 97 percent of students strongly agree or agree that their understanding of the course material increased as a result of using MyLab. 96 percent of students strongly agree or agree that the use of MyLab positively impacted their quiz and exam scores. 98 percent of students strongly agree or agree that they would recommend MyLab to another student. Student survey responses to the question, What are the benefits of MyLab for you as a student include: MyLab makes the work easier to understand by giving different approaches to the same problem, allowing different learning styles to understand the same material. I liked how the MyLab homework explained how to do the questions step-by-step until you got the question right. 6
I really enjoyed being able to work through as many or as few problems as needed based on my previous knowledge of the material. It was also very helpful to be able to go over the homework as many times as needed before taking quizzes and tests. Everything was there, no need to have a [print] textbook. I was able to study anytime and anywhere as long as I had access to the internet. MyLab was an easy and fast way to do homework and quizzes. I learned a lot this semester through MyLab! Conclusion MyLab Economics enabled Herrboldt to offer his online students a complete course: reading, practice, and assessment all in MyLab. Online students who lack face to face contact with their instructor receive just-in-time support and many opportunities for practice and remediation. The content is engaging and interactive, prepares students to be responsible for their own learning, and motivates them to participate. A student on the end-of-semester survey sums up the MyLab experience well: With so many options, it was helpful to try different approaches to see which one worked best for me at that specific time. MyLab helps Herrboldt deliver the same quality course to his online students as he does to his face-to-face students. 7