Use of Games and Guided Labs in an Introductory Probability and Statistics Course Kevin Cummiskey This paper was completed and submitted in partial fulfillment of the Master Teacher Program, a 2-year faculty professional development program conducted by the Center for Teaching Excellence, United States Military Academy, West Point, NY, 2012. Abstract: Playing Games with a Purpose is a National Science Foundation Grant 1 to produce online games and corresponding guided labs that reinforce concepts common in an introductory probability and statistics course. Students play a video game to generate data which they investigate in the guided lab. The guided lab reinforces a particular concept, in our case hypothesis testing, while exposing students to the entire research process. One of the first game-labs developed for this grant is based upon Tangrams, an ancient Chinese puzzle game. This paper discusses the testing of the game and lab in my four sections of an introductory probability and statistics course at the United States Military Academy. 1 NSF DUE #1043814 Dr. Shonda Kuiper, Grinnell College, (Principle Investigator) and Dr. Rodney Sturdivant, USMA (Co-Principle Investigator)
Acknowledgements I would like to acknowledge Dr. Kuiper and COL Sturdivant for giving me the opportunity to participate in the research grant. The Playing Games with a Purpose project involves the creation of several computer-based games, including a graphics-rich video game called TigerStat currently under development, and associated guided labs. My principle contribution to the research has been the development of a version of the Tangrams lab, testing the lab in class, and presenting the results at the Joint Mathematics Meetings. I would also like to acknowledge two other group members, LTC William Kaczynski and LTC John Jackson, for their feedback on the Tangrams lab. Grinnell College students Andy Applebaum, Alex Cohn, Nathan Levin, and Jeffrey Thompson programmed the games, including Tangrams, with Dr. Kuiper and Dr. Sam Rebelsky advising. Introduction MA206 is a calculus-based, introductory probability and statistics course at the United States Military Academy. Every student, other than approximately 1% that validate, takes the course as the last of four courses in the core mathematics program. Topics covered in the course include: descriptive statistics, fundamentals of probability, counting, conditional probability, discrete and continuous random variables and distributions, central limit theorem, confidence intervals, hypothesis testing, and simple linear regression. Of these subjects, hypothesis testing can be the most difficult to understand at the conceptual level. It is also one of the most frequently encountered statistical topics in research science. In MA206, we focus hypothesis testing on the one sample t-test. Students receive three 55 minutes lessons on hypothesis testing. The first lesson introduces the students to hypothesis
testing, selecting appropriate null and alternative hypothesis, and types of error. The second lesson covers the test statistic, the sampling distribution of the test statistic, and p-values. In the third lesson, instructors bring in applications or additional problems to reinforce concepts from the first two lessons. With only three lessons, it is challenging to both teach the mechanics of hypothesis testing and give students an appreciation for its use in research. The Tangrams game-lab is an effective tool for instructors to both reinforce hypothesis testing and allow students to experience research science. At the end of the second lesson on hypothesis testing, students play a web-based puzzle game called Tangrams 2. In Tangrams, a player must solve a puzzle by covering an image with a set of shapes. We measure their performance by the amount of time it takes to complete the puzzle. The instructor decides variables to investigate that relate to puzzle completion time. In my classes, we chose to investigate the relationship between academic major and puzzle completion time. Specifically, we wanted to see if students that major in math, science, and engineering subjects are faster at completing the puzzle than students majoring in other subjects. In addition, we also investigated gender and puzzle completion time. The game s website outputs puzzle completion times and player data (academic major, in our case) to an easily accessible database. Students follow a guided lab as if they were a research scientist. The first part of the lab is a step-by-step, workshop-style introduction (Kuiper 2009) that seeks to determine if a relationship exists between academic major and puzzle completion time. In the second part, we ask students to develop their own research question. The student has to design an experiment and choose an appropriate statistical test. The students stop at the point of collecting data. The Tangrams game-lab gives students exposure to the entire 2 http://www.cs.grinnell.edu/~kuipers/statsgames/tangrams/
research process to include developing research questions, formulating hypotheses, gathering data, performing a statistical test, and arriving at appropriate conclusions. This teaching technique has the potential to be more effective in reinforcing statistical thinking and getting students interested in statistics than simply going through the mechanics of calculating test statistics and p-values. Literature Review Research suggests a vast generational gap between today s students and their instructors. Today s students, coined Digital Natives (Prensky 2001), have used the internet, smartphones, and social media since they were very young. Prensky asserts that their whole way of thinking and learning is different. They prefer fast interactions with material, multitasking, and networking. Their instructors, called Digital Immigrants, have adopted these technologies later in life, like a person learning a second language as an adult. The Digital Immigrants tend to teach the way that they learned in school: slowly, sequentially, and with step-by-step instructions (Prensky 2001). As a result, Digital Natives are disinterested in much of the material they see in academic settings. Digital Game-Based Learning (DGBL) has become increasing common in higher education since the spread of the internet in the mid-1990 s as a way to reach the Digital Natives. There is significant research that supports the use of computer-based games in the classroom to reinforce concepts across a wide variety of disciplines. When students play games, they are interested, competitive, cooperative, results-oriented, actively seeking information and solutions (Prensky 2003). Educators need to carefully design or select games that have the best characteristics of successful video games (interesting and challenging) while reinforcing concepts of the course. Educational
software frequently throws away what is best about the contribution of game designers to the learning environment and replaces it with what is worst about the contribution of school curriculum designers (Papert 1998). As Van Eck states, it is not true that all games are good for all learners and for all learning outcomes (Van Eck 2007). A successful game requires a constant cycle of hypothesis formulation, testing, and revision. This process happens rapidly and frequently while the game is played, with immediate feedback. (Van Eck 2007) In our research, we combine computer-based games with investigative laboratory modules, or guided labs, that introduce students to the research process. In statistics education, it is challenging to get students to experience course concepts inside the context of the larger research process. Obtaining Institutional Review Board approval, designing an experiment, collecting and analyzing data, and deciding upon and carrying out the appropriate statistical test are usually too time-consuming for the instructor to allow the students to carry out in an introductory statistics course. As a result, most students learn statistical concepts without a tie to the context for which researchers use them. Guided labs are a way for students in an undergraduate statistics course to experience the role of a research scientist (Kuiper 2009).
The Tangrams Game In Tangrams, the player gets a set of shapes (triangles, parallelograms, etc.) of various sizes in blue. The player must use the shapes to fully cover the image in red on the right. The player can flip or rotate each blue shape. The game ends when the player completely covers the red image with blue shapes. The player s goal is to complete the game as fast as possible. Whenever a player solves the puzzle, the completion time is recorded to a database accessible through the website. Figure 1 shows the web interface of the Tangrams game. Figure 1. Tangrams Web Interface. Prior to starting the game, the player inputs data about themselves that the class wants to investigate. For my classes, students entered type of major (either MSE for math, science, engineering majors or Other for other) and gender. When the game is complete, this data is recorded in the database along with puzzle completion time. If every student in the class plays the game under similar conditions, a large sample of data is immediately available through the website for analysis. Students use this data in the guided lab to investigate the relationship
between major and puzzle completion time. Figure 2 shows the setup screen of the Tangrams game. The Tangrams Lab Figure 2. Tangrams Setup Screen The Tangrams lab is a guided lab that reinforces the concept of hypothesis testing while exposing students to the entire research process. In an introductory statistics course, students usually perform the calculations involved with hypothesis testing on scenarios and data sets the instructor provides. Students become proficient at executing the hypothesis test, but leave the course without an appreciation for hypothesis testing s role in research. The two-part guided lab exposes students at the introductory level to more advanced statistical concepts such as: developing the research question, designing the experiment, collecting/analyzing the data, choosing an appropriate statistical test, validating assumptions, drawing appropriate conclusions.
We have developed versions of the lab for different level courses and for investigating different variables. The version described below is included as Appendix 1 to this paper. In Part 1, we guide the students through the investigation of a possible relationship between academic major and Tangrams performance. First, we ask students to translate the research questions into the following testable hypotheses. H 0 : MSE majors perform the same on Tangrams as other majors H a : One type of major performs better at Tangrams Next, students calculate basic summary statistics and plot histograms of the Tangrams completion times. In the survey following the lab, many students cited this part as their favorite part of the lab. This was the first time they got to analyze data that they had created. They had to address outliers and make decisions on what data to exclude. They commonly referred to this data as the first real data that had seen, a comment which surprised me given all the real data sets I had already brought in. Table 1 shows the sample mean and standard deviation of the completion times of the 96 MSE majors and 32 other majors that played the puzzle game. MSE Majors Other Majors Sample Size (# of students) 96 32 Sample Average (seconds) 70.2 90.7 Sample Std Dev (seconds) 63.5 95.4 Table 1. Completion Time Descriptive Statistics by Type of Major In the next step, students conduct the hypothesis test and report a p-value. For type of major, the p-value for the two sample t-test was 0.26, not significant at the 95% confidence level (α = 0.05). (Tests resulting in p-values less than α are significant.) This led to a lot of interesting discussion because the MSE majors average was 22% faster, which seems to many
like we should be able to say conclusively that they outperformed other majors. However, these students were ignoring the large standard deviation in the completion times, which decreases our confidence in the location of their means. Following the hypothesis test, we ask students to validate the basic assumptions of the t- test. One important assumption is that the sample of cadets participating in the research is a random sample. This exposes students to the notion that research involves creativity. In practice, obtaining a random sample, particularly when not conducting a controlled experiment, is difficult. There are many reasons why the sample we obtained is not random. We section students by major, and only four sections participated in the study. As the last step in Part 1, we ask students to draw appropriate conclusions from the study based upon the data and results of the statistical tests. In Part 2 of the lab, we no longer guide students through the process. We ask them to develop their own research question, develop appropriate hypotheses, and design an experiment to test them using their work from Part 1 as a guide. Classroom Results and Feedback Students played the game at the end of the second day of hypothesis testing. It was not much of a surprise that they enjoyed playing the game as a break from the normal classroom activities. I was surprised at the level of interest in trying to explain Tangrams performance. Ideas ranged from number of hours of sleep to SAT scores to age of the player. Some students who were otherwise quiet for the semester were contributing ideas. It was a level of interest that I had not experienced in my previous two semesters teaching hypothesis testing. Students progressed through the lab slower than I had anticipated. I asked them to do some histograms and basic descriptive statistics (mean, standard deviation, etc.), which I thought
should take the students a few minutes to complete. Most students took 10-15 minutes to complete this step. I designed the lab with the intention that students be able to complete Part 1 in one lesson. At the end of the first lesson, most had completed the hypothesis test and were working on verifying assumptions. We discussed Part 2 as a group in the last ten minutes of class and I collected the labs at the end of class. In the future, I would allocate two class periods to the Tangrams exercise. This would allow for the completion of Part 2 and class discussion to reinforce important concepts from the lab. At the end of the lab, we had students complete a feedback survey. The following is a summary of the results. Question 1: What did you like most about this lab? Typical Responses: The data set was real and we played a part in creating it. To be able to see an actual scenario where what we learned can be used. The fact that we could collaborate work at own pace, ask questions as needed. I liked that getting the data was very quick and easy. Playing the game! The first comment above, about this being the first time in the course to use real data, appeared in many students comments. This was a surprise to me because I had been bringing in datasets that I thought were real that I gathered from sports, economics, and military websites. From my perspective, those datasets were real. I learned from this lab that if students are not involved in the data collection, they are not as interested in the results and assume that it is a
textbook example. The students liked having to make decisions about outliers and many gained an appreciation for validating assumptions. The advantage of Tangrams is that the data collection is not time consuming, which is typically a challenge when involving students in data collection. Question 2: What should be improved? Typical Responses: Take two classes to give more time Provide directions for the game (such as how to shift an object) on opening begin play screen. I would have liked to have more conversation on the assumptions and better tests to use. Also, it would be cool to have done analysis of real world data and compare our conclusions with those the actual scientists achieved. As written, the lab is too long for one class period and many cadets expressed an interest in having more time to complete the lab. Most of the cadets made it through Part 1 and very few adequately addressed Part 2. In addition to more time for Part 2, I should have allocated more class time to discuss results. Students also commented that they gained an understanding of how hypothesis testing and statistical procedures fit into the research process. For many, it was their first exposure to the research process. A few students stated they would have liked a lab on a topic of more importance than just performance on a game of Tangrams. Most likely, these students did not see the potential connection between Tangrams performance and things of deeper interest such as spatial reasoning, learning and adaptation, and other aspects of human intelligence. In the future, I will dedicate more time to explaining how we frequently design experiments with
variables that we can measure (time to complete a game of Tangrams) in order to investigate something we cannot measure directly such as spatial reasoning ability. In addition to the free response questions above, we also asked students to rate their level of agreement with several statements. 81% of students either agreed or strongly agreed that the Tangrams lab was a good way of learning about hypothesis testing. 10% either disagreed or strongly disagreed with the same statement. 74% either agreed or strongly agreed that the Tangrams lab improved their understanding of using statistics in research. Complete survey results appear in Appendix 2. Conclusion Tangrams was an effective way to reinforce hypothesis testing while giving students exposure to research science. Many students expressed an interest not just in playing the game, but in trying to explain the variation in completion times. They liked being involved in the data collection process; it made the data real to them. The data was messy with outliers, bias, and many of the other problems that come with achieving significant results in research science. Hopefully, students who viewed statistics as following step-by-step procedures to obtain a number experienced the creativity involved with conducting research. For the lab in Appendix 1, about two hours of class time is required for an introductory level student to complete it. Allowing adequate time for students to develop their own research ideas will make the lab the most effective. Whenever possible, instructors should allocate some class time to discussing results as a group. In the future, I want to quantify if this type of instruction is successful at reinforcing hypothesis testing. It may be possible to use performance on the hypothesis testing question on
both the block exam and final exam to measure understanding while controlling for obvious effects like student GPA.
Bibliography Kuiper, Shonda and Linda Collins (2009): Guided Labs That Introduce Statistical Techniques Used in Research From Multiple Disciplines, The American Statistician, 63:4, 343-347 Papert, Seymour. (1998). Does Easy Do It? Children, Games, and Learning. Game Developer. 88. Prensky, Marc. 2003. Digital game-based learning. Computers in Entertainment. 1, 1 (October 2003), 21-21. Prensky, Marc. 2001. Digital Natives, Digital Immigrants Part1. On the Horizon. Vol. 9, Iss: 5, 1-6. Rieber, L. P. 1996. Seriously considering play: Designing interactive learning environments based on the blending of microworlds, simulations, and games. Educational Technology Research & Development, 44(2), 43-58. Van Eck, Richard. 2006. Digital Game-Based Learning: It s Not Just the Digital Natives Who Are Restless EDUCAUSE Review. Vol. 41, 2 (March/April 2006).
Appendix 2. Survey Results (128 Students). Strongly Agree Agree Neutral Disagree Strongly Disagree 1. The Tangrams lab was a good way of learning about hypothesis testing. 2. Students who do not major/concentrate in science should not have to take statistics courses. 3. Statistics is essentially an accumulation of facts, rules, and formulas. 43 % 38 % 8 % 7 % 3 % 5 % 10 % 23 % 37 % 24 % 10 % 34 % 30 % 19 % 6 % 4. Creativity plays a role in research. 30 % 47 % 12 % 7 % 4 % 5. If an experiment shows that something doesn t work, the experiment was a failure. 6. The Tangrams lab had a positive effect on my interest in statistics. 7. The Tangrams lab improved my: a. Understanding of using statistics in research b. Understanding of obstacles that can occur in the research process c. Skill in interpretation of statistical results. d. Ability to integrate theory and practice. e. Understanding how scientists work on real problems. f. Skill in how to effectively communicate results to others. g. Understanding of how scientists think. 9 % 2 % 5 % 31 % 52 % 17 % 38 % 32 % 13 % 1 % 25 % 49 % 16 % 8 % 2 % 20 % 45 % 22 % 10 % 3 % 14 % 57 % 23 % 6 % 1 % 18 % 54 % 22 % 4 % 2 % 15 % 44 % 33 % 5 % 3 % 10 % 39 % 35 % 16 % 1 % 6 % 42 % 33 % 13 % 5 %
Classroom Lab 1 For use with Tangrams http://www.cs.grinnell.edu/~kuipers/statsgames/tangrams/ Tangrams is an ancient Chinese game where players arrange geometrically shaped pieces into a particular design by flipping, rotating, and moving them. Solving puzzles such as Tangrams requires the use of the spatial orientation functions of the brain. These same brain functions are used extensively by mathematicians, scientists, and engineers to solve complex problems. The online Tangrams game allows students the opportunity to design many versions of the original game. In this lab, we will investigate two questions: 1. Do students who major in math, science, and engineering perform better (on average) at Tangrams than students with other majors? 2. Does the completion time of spatial reasoning games, such as Tangrams, depend on gender? Configure and Play the Game. Tangrams Interface Go to the web site http://www.cs.grinnell.edu/~kuipers/statsgames/tangrams/. a. Time: No Restriction b. Display Timer: Yes c. Hints Enabled: Yes d. Type of Puzzle: provided by instructor e. Check the Participant Info box Player Name: use a secret name, any combination of letters and numbers with no spaces. Do not use your name or term that will identify you. Group Name: provided by instructor f. The first external variable should be labeled MSE Major with value: Yes if your current or intended major is in math, science, or engineering and No otherwise. g. The second external variable should be labeled Gender with value: M for males and F for females. Part 1. 1. State all null and alternative hypotheses corresponding to the two objectives of this study. 2. After all students complete the games, use the Recorded Data button to view all data from our course (use same Group Name as above). Copy and paste the data into Minitab, Excel, or other statistical software package. Create an individual value plot, dot plot, or 1
side by side box plots of your data to compare all four groups of data. Are there any outliers or skewness shown in this plot? Are there any errors in the data? Correct or delete the erroneous data. Comment on at least three interesting features of your data (such as comparing the shape, center, and spread of particular groups). 3. Use appropriate statistical techniques to test your hypotheses. 4. What assumptions need to be checked before we can conclude the analysis in Question 3) is appropriate? State each assumption and provide a graph or other justification to determine whether the assumptions are correct. 5. Use the plots in Question 2) and the p-value(s) to summarize your findings. Explain. 6. Can we use this data to conclude that our results hold for all students at our school? Explain. Part 2. Design your own experiment. Note that these games allow you to develop new factors of your own choice. Which factor do you hypothesize has an impact on game completion time or whether some people choose to use hints? Address each of the following points: 7. State the null and alternative hypothesis. Clearly define a problem and state the objectives of your experiment. 8. Identify the response variable, explanatory variable, and units. 9. Verify that the response variable provides the information needed to address the question of interest. Are you interested in the time it took to win or whether hints were used? 10. Are there other factors that may be of importance or potentially cause bias in your results? Identify what other factors need to be controlled during the experiment to eliminate potential biases. 11. Choose an experimental design. Will you use a paired test, two independent sample test or other technique? Keep the design and analysis as simple as possible. A straightforward design and analysis is usually better than complex designs. If the design is too complicated and the data are not collected properly, even the most advanced statistical techniques may not be able to draw appropriate conclusions from your experiment. 12. How many trials (games) will be played? If each subject plays more than one game, how will you determine the order in which each game will be played? 2