Rapid Development Of Games Inspired Metacognitive Learning Experiences Using Moodle And Gamemaker Darryl Charles 1a, Chris Hanna 2a, Richard Paul 1b, Therese Charles 2b 1a dk.charles@ulster.ac.uk, 1b paul-r@email.ulster.ac.uk University of Ulster, Northern Ireland 2a chris@silverfishstudios.co.uk, 2b therese@silverfishstudios.co.uk SilverFish Studios, Northern Ireland Abstract: There are many ways that learning metaphors from games may be harnessed in the construction of motivating and effective game based learning (GBL) experiences. However, significant difficulties exist in the creation of GBL software for use within education and GBL related research. Two of the primary difficulties, which provide the focus for this paper, are cost (both time and monetary) and quality. There is often a requirement to build specific games experiences for a particular research investigation that cannot be fulfilled through the use of existing COTS games. However, research budgets rarely offer adequate expenditure to facilitate the construction of bespoke games for learning experiences that compete favourably with commercial games. Additionally, within research teams the expertise and experience necessary for making games is often limited in comparison to an industry based team. This is further compounded by the need for experimental software to be completed before an academic term/semester starts. Failing this requirement often results in the experiment being delayed until the next academic semester or year. In this paper we discuss a short term project undertaken by a development team comprised of two academics and two commercial game developers. With only two weeks of design and development a game based learning solution is created and deployed over a five week period for a class of 2 nd year university computing students. This work was intended both as a technological proof-of-concept and as an investigation into the students' reaction to the use of games to enhance their understanding of their learning experience. We report on our design and development experiences within a multidisciplinary team, and in particular discuss the use of Moodle and GameMaker as potentially powerful rapid development/prototyping toolsets. A summary and analysis is provided of the extensive student interaction statistics within the learning system and the associated game. This quantitative information is utilised to inform the discussion of additionally gathered qualitative student feedback regarding the system. Key Words: games-enhanced learning, gamification, metacognition, rapid development Introduction The study of games for use in serious contexts is a rapidly expanding area of interest, due in part to the recent growth of interest in gamification research (Deterding, Sicart, Nacke, & O Hara, 2011; Lee & Hammer, 2011), which has emerged out of more established games related research areas such as serious games (Abt, 1970), game based learning (Shaffer, Squire, Halverson, & Gee. J., 2005; Squire, 2005) and virtual worlds (Bartle, 2003). Within education, research indicates that the gamification of learning processes can increase user engagement and motivation (T. Charles, Bustard, & Black, 2011). Elsewhere, businesses initiatives such as Foursquare and SalesForce have used gamification techniques to increase customer engagement and encourage desirable user behaviour. Much of the use of gamification in educational or business processes is based on the simple game metaphor of reward, for example the provision of points and achievements for completing tasks. Effective approaches that use this metaphor employ the gamification process to expose the mechanics and processes that must be learned by users/learners (Charles, Bustard, & Black, 2008). In this way the underlying mechanics of a serious process become the rules of the game and the rewards provided by the system provide feedback to the user on how effectively they
are engaged with the system. There are many game metaphors that can be used within a gamification process, and within this paper we primarily consider aesthetic representations as a means of communication. Game worlds are graphically rich and inherently interactive and as such can be used to represent complex information in a relatively straightforward manner, especially because it is a well-understood media form. For example, games can represent spatial and temporal relationships very well, and weather and atmospheric world conditions can be used to represent information in a manner that can have an emotional connotation (Herbert, Charles, McNeill, Moore, & Charles, 2011). Outlined within this paper is a continuation of our research into game based feedback mechanisms (D. Charles, Charles, & McNeill, 2009) and in particular relates to an on-going project where we seek to harness metaphors obtained through the study of games to provide rich metacognitive learning experiences (Herbert et al., 2011). The specific realisation of our ispiral (Interactive Stellar Platform Integrating Reflection and Learning) system architecture described within this paper is designed to meet three main objectives: to provide a workable solution that is quick to deploy and easy to manage, to improve students motivation, through active participation, and to provide graphically rich interactive mechanisms to help students gain a deeper understanding of their own meta-cognition processes. The work described is a collaborative effort between two researchers at the University of Ulster and two game developers at SilverFish Studios, a commercial serious game development company based in the north of Ireland. Design and development of the system was collective, with the academics having primary responsibility for system design and members of SilverFish primarily involved with the game design and programming. System Architecture and Design This paper outlines work as part of a larger on-going project that seeks to use a wide range of game metaphors to improve the design of digitally based learning systems. As part of the process of developing a comprehensive framework and test-bed for long-term experimental work, it was deemed necessary to test key ideas using focus tests (Herbert et al., 2011) and proof-of-concept experiments. ispiral was conceived as a complete games enhanced metacognition and feedback learning system that could be rapidly constructed and meaningfully tested. The ispiral system had the following necessary requirements: 1. The system must be accessible both onsite and offsite 2. It must be constructed, deployed and tested within 7 weeks a. Two weeks design and development b. Five weeks deployment with a 2 nd year Computing class 3. A web based LMS was required for teaching material and class management that had a. Ease of use b. Convenient access to LMS data, e.g. access to MySQL database information via usernames 4. A suitable game (or multimedia) technology that supported a. Rapid development of the required game metaphors b. Could be integrated readily into the LMS course c. Was capable of running efficiently within a web browser As the ispiral system was to be designed, constructed and deployed within a very short period of time, both the pedagogical structure of the course and the choice of game metaphors to be integrated were kept simple. An Agile oriented development process was utilised with very short iterative design and collaborative programming sprints. The overall system architecture was primarily designed by the academic team and detailed site and game development was mainly the responsibility of the collaborating company.
Software Choice Three possible learning management system (LMS) choices for the system under consideration were: a Bespoke LMS, Moodle and Blackboard. We are currently building a bespoke LMS software using.net technologies tailored for experiments within our own virtual game world; this has been under development for the previous six months however it isn t as flexible as existing LMS software and so for the ispiral experiment this approach did not meet requirements. Blackboard is currently used throughout the university where ispiral would be implemented. Although this would provide an advantage because students are already familiar with it, Blackboard was ruled out because of difficulties in accessing student data from the database. The Moodle LMS was chosen as a basis for the system because it provided most of the educational tools required by this approach (quizzes, login, communication, blogs, wikis, assignments, and course content). Also, it provided us with complete access to the course database, and we considered it to be the most suitable for rapid development and integration of games inspired learning software. Moodle was installed on our own server and populated with content in a very short time. This content would ultimately facilitate the collection of a rich set of statistical information about student interactions that could be accessed directly via SQL for use in our representation system and game. The choice of tool for the implementation and integration of the game metaphors within the system was more complex as there are a wide range of possible approaches. For example, game engines such as Shiva3D or Unity3D could have been used if a 3D game aesthetic were necessary. These tools are relatively easy to use and solutions can be deployed to many different platforms including web browsers. A 3D game engine was not chosen because of the very short time to create the necessary solution; it was decided that these tools required more time due to the cost of asset creation. Second Life (or its open source versions) was a possible option though was not considered due to the flexibility required for integration and interaction. We decided that an approach based on a 2D aesthetic would be the most productive; however there are also many tools that could be used within this scope. Flash and Silverlight technologies support rapid development of web based multimedia and met our requirements. However, GameMaker:HTML5 (http://www.yoyogames.com) had just been released at the time of our technology review and supports very rapid development of game solutions as well as browser deployment on an HTML5 / Javascript platform. Our crossdisciplinary team were also familiar with the tool having constructed several games with earlier versions of GameMaker and so this was chosen for the creation and integration of the game metaphors. Moodle Course Structure The Moodle site had to be quickly created on a remote server; in this case it was set up over a single night. Also, the course materials needed to be put in place quickly so that from the very beginning students would always be able to see the current and subsequent week s work at any time (as well as materials from all previous weeks). Additionally, we required the system to monitor and record student participation and performance automatically (with minimal intervention by the teaching staff) in order that feedback to students would be instantaneous. For these reasons the course structure illustrated in Table 1 was used, and for this proof-of-concept experiment student activities were predominantly monitored in a quantitative manner, e.g. we recorded clicks on links, numbers of blog posts, wiki posts, and scores on quizzes. We also recorded qualitative scoring of wiki posts but didn t use this in the game based feedback system. As will be shown below, ispiral provides feedback based on a student s participation and performance using game technology. Through this, a simple learning process (prepare learn practice reflect improve learning technique) can be exposed to a student in a novel and potentially informative and engaging way.
Table 1: Course Structure for each week, e.g. week 4 Preparation Tasks Lecture Practice Tasks Reflection Tasks and Self-tests Feedback Further work Summary of what's going on this week Reading material and web links The structure of 2D games Lab 1: Game parameter management Lab 2: Game entities management Task 1: Blog Post. Outline game design. Task 2: Blog Post. Development plan. Game Design Quiz C# Quiz XNA Quiz Past Exam Examination Questions Was the work difficult this week? How much is your programming improving? How much of the work were you able to complete? Extend your skills Next week s activities ispiral Architecture The ispiral Architecture is shown in Figure 1. All interactions with the system begin with logging into Moodle. This single centralised login approach was implemented because we wanted to make the link between the game enhanced feedback system and course engagement explicit. From a technical point of view this meant that a student s user ID could also be used for the game technology interactions, i.e. they didn t have to log in twice. The Moodle MySQL database is central to the operation of the system; course content is stored on this but it also stores all of the student interactions within the site that are illustrated in the visual engagement map, which is used to augment a player avatar within an interactive game. A separate MySQL database was maintained to hold a table of high scores that could be viewed by students on a browser without logging in. Students can access their personal engagement map via a link on the Moodle class site, and from there they can gain access to the interactive game, which is personalised on the basis of their class engagement statistics. The full system is accessible through any web browser with HTML5 capabilities (e.g. Google Chrome). Moodle LMS Site Moodle MySQL Database MySQL High Score System and High Score Web Page Engagement Map, e.g. Star constellation Learning Feedback, e.g. side scrolling space shooter Figure 1: ispiral System Architecture
Game Design At the heart of the ispiral system is the integration of game technologies to enhance aspects of the learning process, e.g. feedback, engagement and reflection. In our experiments we are particularly interested in enhancing a student s metacognitive qualities; by using game metaphors in our design we may help the student users to develop improved learning techniques, so that they may be more effective at learning how to learn. Two game metaphors are used in this paper: character skill representations inspired by skill trees in RPGs similar to the skill constellations found in the RPG Skyrim (http://www.elderscrolls.com/skyrim/), and avatar embodiment and development, which is also inspired by the RPG genre. Figure 2: Engagement Map. Galaxy course view (left) and constellation (week) view (right) In Figure 2 the screenshot on the left shows our galactic representation of the course where each constellation within the galaxy represents a week of learning and each star shines brightly when the content for that week is open, otherwise it is not displayed. Selecting a constellation allows the learner to zoom in on that particular weekly constellation, e.g. in Figure 2 (right) the Mario constellation represents a student s engagement within week 1 of the course. The stars within the constellation light up when a student engages with a particular aspect of their learning (in a predetermined way that we will discuss below) through the Moodle site. Links between the stars are visualised in a logical, pedagogical manner, i.e. stars at the bottom of the constellation should light up first and a student thus should understand or learn to engage with learning material in a particular order. There are individual constellations for each week (each with different game characters) that provide students with a graphical form of feedback on their engagement with the course. Students can examine their own personalised constellations to identify incomplete aspects if their learning which they can then engage with when they return to the Moodle site. The stars are also linked to characteristics of a student s space ship (Figure 3) in the associated space shooter game. This means that if a student is doing well in class the stars in their constellations that light up will result in their space ship's attributes being augmented in a pre-specified way. The high score achievable by a student in the ispiral space shooter game thus relates to both their skill as a gamer and their current state as a learner. This provides a very literal representation of a student s educational state; one that the student can experience in a highly interactive manner (a video can be seen here http://bit.ly/ksjnxd).
Figure 3: Learning augmented ispiral spacer shooter game For brevity we only discuss aspects of the game that are required to explain the essential concepts of our system. The space shooter genre was chosen, despite understanding that the style of game would not appeal to all students equally, because it is a quick game to build, the gameplay is readily understood, and play sessions are quite short (generally less than 5 minutes). In ispiral the gameplay is built on familiar mechanics for this genre. It is a side scrolling game with the player s ship orientated with a right facing viewpoint. Enemy space craft and other game entities enter the game screen from the right hand side and move left towards the player. There are four basic enemy types, each with different characteristics, e.g. size, health, AI, weapon fire. The player ship can normally fire standard missiles (with a rate dependant on class performance) but two special weapons can be unlocked through attempting quizzes: heat seeking missiles and a blaster gun. There are also two types of pickup entities that enhance the status of the player s ship: health and energy items. The frequency with which these items appear relates to a player s luck attribute value, which is related to an aspect of the student s class engagement. Table 2 shows the initial and maximum ship attribute values, which were decided upon after play testing in the design and development phase. These values are crucial to gameplay quality and it is especially important to balance these since they are the attributes of the player s ship that are upgraded based on the student engagement indicators as discussed above (Table 1). Table 3 provides an illustratration on how the personalised configuration of a player s ship is calculated for each student s play session. Each attribute is scaled to increase linearly between the minimum and maximum attribute values over the 5 weeks of study. Table 2: Player Ship Attributes Upgradable Ship Attributes Minimum Ship Attribute Values (Initial Base Values) Medium Ship Attribute Values Maximum Ship Attribute Values Speed 1 1.5 2.5 Health 50 150 200 Fire Rate 2 4 7 Damage 6 25 50 Lives 2 5 10 Energy Capacity 50 200 500 Energy Regeneration 0.025 0.1 0.25 Rate Shield Recovery Time 20 15 10 Shield Strength 200 300 500 Luck 0.1 0.4 1
The game has 20 levels (or waves of attack) and the difficulty level increases linearly throughout by increasing the number of enemies. As with optimizing the player attribute values, the minimum (initial level) and maximum (end level) number of enemies that were required for quality gameplay was decided through extensive gameplay testing. Table 3: Player Ship Attributes Upgrade Rates (illustration from a single week) Course Task (per week) Read all preparation task notes View lecture material Download load notes for 2 lab sessions Write 2 blog posts Cumlative Quiz Attempts over several weeks Complete 3 feedback tasks View further work tasks Player Ship Attribute Shield recharge speed Ship missile fire rate 1. Energy recharge 2. Lives (+9%) 1. Energy Capacity 2. Shield Strength (+9%) 1. Missles (+1) 2. Blaster unlock (+1) 1. Health 2. Luck (+10%) 3. Speed (+15%) Shot damage System Usage Analysis ispiral was deliberately set up to be very low maintenance, i.e. all of the chosen engagement indicators could be monitored automatically without any input being required from the lecturer. This facilitated very rapid updates within the system and the provision of almost instant feedback via the game system. Moodle provides a statistical summary of activity on the site and this is very useful in analysing student s engagement within the overall system. ispiral was used in a game development class with 72 registered 2 nd year computing students. A graph of overall student views and posts (wiki, blog, discussion and news) is shown in Figure 4. The average number of content views per student was 60.84 and the average number of posts per week per student was 6.98. This behaviour is reasonably consistent across the weeks with the exception of the spike in week 4, caused by coursework submissions that where scheduled for week 5. A slight increase in posting can be seen from week 1 to week 4, which could be due to several reasons including improved familiarity with the course structure or the learning of the rules of the ispiral game. Figure 4: Overall content viewing and posting over 5 weeks The statistics relating to clicks per resource can also be uncovered from the Moodle site, which can help gauge interest or engagement with various resources (though it must be noted that stats from teacher clicks are also included). Over the five weeks of the experiment the ispiral game was accessed 469 times overall by 53 of the students within the class (73.6% of the class). These are encouraging statistics, however from the logs and other feedback that we received it is evident that some students were more engaged with the game than others; one student accessed the game 65
times, while another accessed it 53 times (between the two students they were responsible for 25% of ispiral activity). In comparison, the lecture resources were accessed an average of 60.8 times per resource, lab resources were accessed an average of 374.2 times per resource, blog task were viewed an average of 235.1 times, the quizzes were accessed an average of 217.5 times, and the forum was accessed 1788 times in total (although there were only 135 original threads). Statistics on student engagement with course Wiki support the view that some students were generally more engaged with the class than others; out of the class 33 students made no contribution to the voluntary Wiki (perhaps because it wasn t linked to the ispiral game nor any explicit assessment), while 6 students made the majority of 67 unique Wiki (compared to 254 Wiki views). These statistics provide some quantitative perspective on the student s engagement with ispiral. It appears from the Moodle Logs that the ispiral learning resource was similar in popularity among students as most of the other class resources. However, it was possible that students simply enjoyed the game and perhaps didn t understand the point of the system. With this in mind, we surveyed students throughout the course and invited them to provide some qualitative feedback on the approach when the experiment was concluded. Each week, students were asked three questions: 1. Do you feel that this week s work was easy or hard? 2. Do you feel your programming ability is improving? (Except in week one which instead asked: How do you rate your programming ability? 3. Did you complete all of the week s tasks? Though we have detailed statistical responses from the students what we mainly wished to gain from this survey was a sense of how seriously the students took their own self-evaluation and whether they felt that they were improving. These self-reflection tasks did not count towards a student s assessment but participation in them did light up their ispiral constellations and power-up their ispiral ship (thus enhancing their chance of getting a high score). In week 1, 36 students completed the surveys whereas in week 2 only 12 completed the surveys, perhaps reflecting a busier week of work. However, in week 3, 31 students completed the surveys and this trend continued into week 4, as 31 completed the surveys. This change in behaviour appeared to be related to students gaining an understanding of the mechanics of the ispiral game (learning the rules of the game). 75% of students provided a positive response to question 2 over week 2 to 5, while students typically indicated that they were finding that the work was getting harder and more difficult to complete. In week 5, which was a busy week for coursework hand-ins, 21 students completed the surveys. In week 5, a further question was added: Has ispiral made you think about your class participation more?. Twenty students answered this question, 70% of whom responded positively that it helped them consider their participation and 10 % who felt it did not. Students were also invited to provide written feedback on the system and to respond to the following questions: 1. ispiral was designed to help you understand your current academic state and progress within class. How well did you understand this? 2. Did ispiral motivate you to study/work differently, more consistently, harder or in a more focused way? 3. Was ispiral useful in helping you think about your progress and/or your strengths and weaknesses in this module? Did it help you reflect on your learning? 4. Your space ship is powered by your achievements in class (i.e. quiz results). How do you think this succeeds as a mechanism to provide feedback on module/learning progress, improve understanding of strengths and weaknesses and motivate to improve?
5. Do you think any other game styles or genres would work better? It was clear from the feedback that students understood the system though they were a little conflicted over the use of the game. Most enjoyed playing the game and understood the connection between the ship power-ups and their academic state. However, some students didn t find that it motivated them to work better although it helped them to understand what they had achieved in class. Design suggestions were made in relation to enhancing the tracking mechanisms (i.e. track and use more complex data than clicks). Students felt that the reflection process was too shallow, which is true for this proof-of-concept, and it was interesting that students understood that this could be improved. Some students used ispiral to check on their progress while others didn t engage with it at that level. The genre for the game didn t particularly seem to matter to them but some students recommended more emphasis on multi-player, social aspects, competitive play, enhanced reward systems, and more meaningful links to the course content. The collaboration between academia and industry in the creation of the ispiral learning system was considered by the team to be a success. While there are understandable limitations to the current system, it was created as a proof-of-concept to test how rapidly a full system could be created using our architecture. It was also valuable for us to experience working in a mixed academic/commercial team in the realisation of the system as we consider this to be important to the creation of effective games based learning tools for research. Similarly, industry can benefit from academia in the creation of state-of-the-art commercial systems. The collaboration worked well due to good levels of communication and in particular because much of the critical work was completed with the key parties working in the same location. One drawback was that due to the commercial commitments of the company much of the work had to be completed in the evenings or at the weekend. The collaboration facilitated rapid development (with many issues resolved much quicker than they would have) and the software performed more effectively and looked much better than it would have without industrial input. Two straightforward game metaphors were used, of these the constellation was the more effective communication tool, and it has a lot of potential for use in a more complex pedagogical set up. Students found the game to be fun and understood the point but most students did not use it much to evaluate their learning but rather it encouraged them to complete tasks without considering the learning implications. After students realised that we mainly monitored clicks on resources (i.e. they learned the rules of the system) some students' behaviour deteriorated into clicking on resource links simply to unlock their space ship power-ups within the game. During the design phase we understood that this might have occurred but we still believed that it was an interesting approach to investigate, due to it being one of the most straightforward ways to examine developmental embodiment within an avatar (the space ship). Now that the basic system has been tested successfully there are many other game metaphors that can be explored using the same principles, and the shooter game can be enhanced in many ways, e.g. an item shop, more emphasis on educational cause and effect (pedagogical structure) within the game design, multiplayer, social, and gamification features (achievements). Conclusions Within this paper the ispiral games enhanced learning system design, implementation and deployment with a 2 nd year degree class was described and discussed. This was a potentially complex system that we set out to build rapidly (within 7 weeks), and was a collaborative academic/industrial venture. Moodle and GameMaker were chosen to meet the requirements that we laid out at the beginning; in particular because they could be used to build the system quickly and to an acceptable level of quality. Though there were limitations on specific design issues, the collaboration was successful and the system was completed on time and tested with the class in question. Moodle is a well-known LMS and it proved to be flexible enough to meet our needs as well as being easy to deploy and use. Moodle site statistics along with qualitative and quantitative
feedback from students provided positive indications that most students understood how the ispiral system worked and levels of engagement were generally better than expected. The results have encouraged us to expand on this proof-of-concept using other more complex game metaphors and the information that we obtained from the experiment will enables us to improve the design and deployment of future technology. References Abt, C. (1970). Serious Games. New York: Viking Press. Bartle, R. (2003). Designing virtual worlds. New Riders Games. Charles, D., Charles, T., & McNeill, M. D. (2009). Using player and world representation techniques from computer games to improve student engagement (pp. 36-42). Coventry: IEEE. Charles, T., Bustard, D., & Black, M. (2008). Game Inspired Tool Support for e-learning Processes (p. 188). Academic Conferences Limited. Charles, T., Bustard, D., & Black, M. (2011). Experiences of Promoting Student Engagement Through Game-Enhanced Learning. In M. Ma, A. Oikonomou, & L. C. Jain (Eds.), (pp. 425-445). Springer London. doi:10.1007/978-1-4471-2161-9_21 Deterding, S., Sicart, M., Nacke, L., & O Hara, K. (2011). Gamification. using game-design elements in non-gaming contexts. Sociology The Journal Of The British Sociological Association, 19(3), 2425-2428. Retrieved from http://dl.acm.org/citation.cfm?id=1979575 Herbert, B., Charles, D., McNeill, M., Moore, A., & Charles, T. (2011). Dynamic Virtual Learning Landscapes to Enhance Student Reflective Processes. European Conference in Game Based Learning. Athens: ACI. Lee, J. J., & Hammer, J. (2011). Gamification in Education: What, How, Why Bother? Academic Exchange Quarterly, 15(2), 2. Shaffer, D., Squire, K., Halverson, R., & Gee. J., P. (2005). Video Games and the Future of Learning. Phi Delta Kappa, 87(2), 105-111. Squire, K. (2005). Game-based Learning Present and Future State of the Field. Masie Center e-learning Consortium.