Do Mobile Device Applications Affect Learning?

Do Mobile Device Applications Affect Learning? Doug Vogel Dept. Information Systems City University of Hong Kong isdoug@cityu.edu.hk David M. Kennedy Information & Tech Studies Hong Kong University dkennedy@hkucc.hku.hk Kevin Kuan Dept. of Information Systems City University of Hong Kong kkykuan@cityu.edu.hk Ron Kwok Department of Information Systems City University of Hong Kong isron@cityu.edu.hk Jean Lai Department of Information Systems City University of Hong Kong isjlai@cityu.edu.hk Abstract Mobile devices (e.g., PDAs and smartphones) are increasingly emerging as part of daily life, particularly with university students. The City University of Hong Kong has embarked on a long-term program to develop and integrate mobile learning activities into the context of undergraduate courses. This paper reports on the development, introduction and evaluation of a portfolio of collaborative mobile learning applications. Results support convictions that intrinsic and extrinsic motivation to embrace mobile applications correlates with enhanced performance, albeit with constructive alignment of student learning interests as a moderator. 1. Introduction Technology in education is not new but times are changing as we move into more distributed contexts with a variety of pedagogical motivations [11]. Early efforts focused on collaboration within a more traditional learning environment [1]. More recently (e.g., [2]), attention has been given to institutional direction and options. Aspects of successful integration of web-based course management systems have been explored (e.g., [15]). Increasingly, institutions recognize that the learning environment extends beyond the classroom and are thus exploring a variety of technological support options. Mobile devices are increasingly prevalent and are seen as a birthright in some parts of the world (e.g., Hong Kong); accordingly, a variety of mobile learning applications have been developed (e.g., [6]). However, little exists to illustrate what difference (if any) they make, especially in terms of learning. This is curious given the ubiquitous nature of these devices and applications and the ability to potentially connect anytime and anyplace for interactive learning. Digital natives are at the center of their own personal learning environment (e.g., smartphones, PDAs, ipods, etc.), yet limited attention has been given to the impact on learning of mobile devices and associated applications in education [12]. As mobile device applications become even more pervasive, it is incumbent that we understand their (potential) impact and integrate them into the portfolio of educational and learning options. Further questions exist as to what degree and in what ways institutions can become a part of this emerging student-centered learning environment and how they can maximize the impact of that involvement. Students have a variety of alternative uses for any mobile device. In particular, a question exists as to the characteristics of learning applications that will be accepted and meaningful to students in a collaborative learning space with a variety of interaction channels. To research this domain, the City University of Hong Kong (CityU) has provided wireless PDAs to all incoming business student over the past two years (over 1600 in total). These devices have been used extensively on and off-campus in conjunction with a portfolio of applications to support learning. In this paper, we explore the impact on learning of these applications designed for use outside the classroom. Particular attention is given to a multiple choice learning application with summative and formative feedback running on mobile devices that enables students to prepare remotely for their final exam. Comparisons are made first between students using the learning support versus those who chose not to do so, and second between topics that are chosen. 1530-1605/07 $20.00 2007 IEEE 1

Usage is further correlated with performance as exhibited on exams. Particular attention is given to the pattern of mobile learning application use, e.g., for exploration of alternatives. Conclusions are drawn. 2. Literature There is a variety of literature from different disciplines regarding motivation (both intrinsic and extrinsic), learning, student performance and technology support. However, there is relatively little in the intersection of these literatures, especially in conjunction with mobile devices. Alavi and Gallupe [2] recognize the high levels of cultural change at individual and institutional levels, and note the importance of performance assessment in conjunction with technology-mediated learning. Mobile devices, e.g., PDAs and smartphones, are a categorically different form of technology with different behavioral consequences. The ubiquitous nature of these mobile technologies in terms of being constantly within reach of the users and continuously connected to a broader communications network give them a unique status in the realm of technology support for education and learning. In this sense, these devices become a form of wearable technology that places students in the center of their own unique learning environment. They have the opportunity to choose what to access, when and where, and with more degrees of freedom than other forms of technology support e.g., PCs. Self-guided exploration has been found to be conducive to the development of intrinsic motivation in more traditional educational settings [7]. We can enhance learning motivation by emphasizing the importance and applicability of the material and by trying to connect the material to students intrinsic motives [5]. They particularly note that learning motivation is likely to be greater if a student feels a particular class is consistent with their interests and with personally satisfying career goals. However, learning motivation is malleable and can change over time [13]. The early development of high learning motivation is critical. Thus it is important to focus on building learning motivation for education in general, as well as for specific classes [5]. Mobile devices, e.g., PDAs and smartphones, are increasingly pervasive, especially in student populations. In some parts of the world (e.g., Hong Kong) this exceeds 100%, i.e., each student may have more than one such device. The pervasive nature of these devices provides an unprecedented opportunity for creation of learning applications. As with paper and books, we can increasingly expect that our students will (or can be mandated to) have personal access to a mobile device to complement their traditional learning activities. Developing effective learning applications requires a number of considerations. We need to consider both the human dimension (students and instructors), as well as the design dimension (content, technology, interaction, etc.) [10]. There are additionally a wide range of institutional considerations along the way towards broad-based institutional use [16]. A key aspect of mobile device applications is pedagogical integrity. The research of John Biggs [3] [4] provides pedagogical direction that is particularly relevant in terms of aspects of constructive alignment and deep learning. Biggs points out that students who are given learning materials consistent with their interests and aspirations are likely to perform better; he has provided a taxonomy (SOLO) to assist in evaluating the degree to which students have gone beyond surface learning towards deep learning that can better enable transfer to other tasks. Wood et al., [17] further note that enactive exploration on the part of students also tends to enhance performance on transfer tasks. The research of Biggs and Wood is especially complementary to that of Cole and Feild [5] who emphasize motivation. 3. Research environment At the City University of Hong Kong (CityU), the development of mobile applications has focused on providing more flexible possibilities for student learning. Students in Hong Kong are like most others, juggling busy lives and moving in and out of wired and wireless environments. Zheng & Ni [18] have suggested a model for the manner in which one may consider the elements of mobile computing which is more congruent with current practice and infrastructure. The Hong Kong students move between the: connected mode (at the campus), nomadic mode (at home or connect to a desktop computer on campus or at home), and disconnected mode (on public transport, away from wired or wireless connections). Creation of the mobile environment and associated applications has proceeded along the lines proposed by Nunamaker et al. [14] and Hevner et al. [8]. The current project is a holistic approach that seeks to: 1. Develop and research the use of a range of tools that are designed to more readily support academic teachers in their quest to match the 2

student learning outcomes with appropriate activities, assessment and feedback; 2. Develop the technical infrastructure that enables academics and students to collaboratively author activities to the web or personal digital assistants (PDAs) or smart phones; 3. Develop the technical infrastructure to allow lecturers to monitor student activity, record student learning outcomes about student interactions on their PDAs to the lecturers from within the university learning management system (LMS), BlackBoard; and 4. Provide advice (mainly pedagogical) and support (with examples) for developing content suitable for mobile learning. To create an environment for application and research, CityU provided 150 wireless PDAs to first year undergraduate students in the Fall of 2004. Assessments of usefulness and acceptance were made during the semester, while at the same time, existing infrastructure, support, and adoption problems were addressed. The experiences proved enough of a success to enable a faculty wide roll-out one semester later. By late January 2005, 800 freshman students had successfully been equipped with PDAs, and were using them every week in their core courses. Positive feedback from instructors and students led to the decision to repeat the initiative for a further year (with plans for the year following). Hence, in August 2005, the Faculty of Business provided a further 800 wireless PDAs to equip the second entire freshman class with interactive computing devices. A series of mobile-oriented applications have been developed and evaluated. These include: e-tips provides exam questions from the previous year. No answers are supplied. Crossword Puzzle (as its name implies) provides hints and clues for PDA users to solve a disciplinary-centric puzzle. Mobile Exercise provides sets of multiple choice questions that the students can complete offline. Tatoes is a quiz application based on Hot Potatoes (freely available to non-profit educational institutions;http://hotpot.uvic.ca) that provides detailed instantaneous formative feedback for incorrect (as well as correct) choices in sets of multiple choice questions. The project team has created a set of icons to provide a consistent interface and usability experience for students that crosses applications. Additionally, an authoring environment provides support for creating application instances. An etoken system has also been created to help motivate student participation. Using the e-token system, students download an application and then are rewarded with points based on the degree of application completion success. Results are encrypted to discourage inappropriate sharing of results, e.g., students are not able to simply upload results files to multiple student accounts from a single PDA. As such, there was reasonable confidence that students were independently submitting results. Students (and their scores) are recorded on the e-token server which is subsequently linked back to the BlackBoard LMS for selective incorporation into course grading. Top ranking students are also recognized on the e-token website. Additional applications have been (and are continuing to be) developed as more is learned. These include: IGO (Interactive Graphing Object) was originally developed for the web [9]. IGO enabled students to sketch a graph on screen in order to articulate their knowledge directly. This is in contrast to watching an animation or choosing the correct graph from a set of static images. Repeated feedback is given as students adjust the graph they create to meet problem specifications. Phototate provides field trip support for students to take a picture with their PDA, annotate it and add a voice tag, all of which is then uploaded to a BlackBoard portfolio. Rogaining is a collaborative scavenger hunt in which student teams are given a budget and urged to buy a set of products (using Phototate) for a simulated department store (e-organization). These newer applications have not yet been formally evaluated with large numbers of students but initial trials have been conducted, for example, Phototate has supported a group of students on a field trip to Norway to examine plant life above the Arctic Circle. 4. Assessment approach The anytime/anyplace/any pace availability of mobile devices is felt to provide an environment in which barriers to participation are accordingly lowered, noted by Deci and Ryan [7] as being a factor in intrinsic motivation. There is also an element of extrinsic motivation in conjunction with the e-token 3

approach and availability of support designed to help students prepare for their final exam. The theoretic foundations for PDA impact assessment are based on two sources. The first is the work of John Biggs [3] [4] who proposes that constructive alignment between the desires of students and learning materials is key to performance. We additionally draw on the work of Wood et al. [17] who note that those students who explore the range of possibilities associated with a task tend to outperform students who stop after arriving at a correct answer when faced with the need to transfer learning to new (related) tasks. This leads to the model illustrated in Figure 1 and associated postulates (P). Mobile-induced Motivation Figure 1. Research model P1. Motivated use of mobile learning applications will positively influence student performance. P2. Mobile learning applications that align with student learning interests will positively moderate performance. Both quantitative and qualitative data have been gathered to evaluate these postulates. Surveys have been conducted and focus groups run. etoken results have enabled comparison of student activity with grades to give indications of performance impact. Specifically, analysis of covariance (ANCOVA) has been used (with prior student performance as a covariant) to recognize results of PDA use within the normal variances in student performance as a function of interest, motivation or natural ability. Additional field experience with Phototate has also been reported to shed further light on results from use of Tatoes. 5. Results Constructive Alignment Performance To date, perceptual data on a large scale have been gathered from two courses of study: 1) from 416 students in multiple sections of an introductory business course (of approximately 800 students) in Semester 1, of the 2005-2006 year, and 2) an introductory business class in Semester 2 of 2006. The use of the PDA was not mandated in either course, but was left a voluntary activity. Current feedback (voluntary) indicates that students who engaged in the use of mobile devices for learning found that the experience of using a PDA enhanced their learning experience. In particular, they noted that the mobile environment made the course more interesting, enjoyable and fun in addition to being useful for understanding and remembering, as well as generally helpful for motivation and increasing confidence. However, they saw the environment questionable for changing learning behavior and not especially useful in stimulating students to explore new topics. We begin by reporting data based directly on PDA use, accompanied by focus group comments and followed by ANCOVA results. Recently, 186 students actively used the etoken system from a total of 812 enrolled in the Semester 2 course, as summarized in Table 1. Max indicates the maximum number of instances accomplished by an individual student. Sum indicates the total number of instances accomplished by all students participating in the learning activity. S.D. is the standard deviation around the mean student accomplishments. Mobile Applications Max Sum S.D. E-Tips downloads 11 288 1.15 Crossword puzzle downloads 7 116.75 Crossword puzzle uploads 8 49.57 Mobile exercise downloads 8 201 1.07 Mobile exercise uploads 9 72.69 Tatoes downloads 10 305 1.35 Tatoes uploads 2 36.29 Total count 31 1067 3.22 Table 1. Descriptive statistics of applications More applications were downloaded than uploaded due, in part, to the relative small amount of extrinsic motivation compared to intrinsic motivation and individual curiosity. The use of the etoken system was entirely voluntary. There is also higher variability of downloads as opposed to uploads, i.e., students were in a more exploratory mode when downloading and those who chose to upload results were more internally consistent and definitive as a group. The total count is less that the sum of the individual application Max, as one would expect as a result of students choosing different applications. The students who downloaded e-tips were also inclined to download tatoes (i.e., a 4

correlation of.57), indicative of feedback immediacy impact in those two applications. Each of the applications will be discussed separately, based on demonstrated usage as well as comments from two focus groups, comprising a total of 26 students conducted using GroupSystems to record and organize results. Student comments have been reported in their own words, i.e., complete with grammatical and spelling errors to retain authenticity. e-tips had 288 download instances but the results were mixed in terms of helping to achieve a higher academic grade. Students noted: etips is useful and a good tools to help us to prepare for the examination not really as it is quite short and not detailed. It can tell the main key point, but not all key points, so, I also need to revise other things concise and precise, but sometimes, too short sometimes the contents of the sentances are too "wide" that i don't exactly know which materials is important. Suggestions for improvement included: if possible, it should include a diagram to summarise and present the linkage to other topic, becasuse students like diagram more and easy to remember Crossword puzzle had 47 students downloading a total of 116 crossword puzzle instances from the 17 different puzzles available. However, only 49 instances were uploaded for e-token credit. Student focus group comments included: After doing the crossword puzzle, I could understand some words that I don't familiar with before. if you can learn more words of this subjects, it is more easier for you to revise and you are more willing to do so. Therefore, after you "know" the which topics you are not familar with, you can put more effort on that topics. it can indirectly upper your grade. Some frustrations were also present, for example: some words are too long and difficult and some are technical words and difficult for us if there is more guildlines, we can follow it more easily and more willing to do the exercises. tooo difficult to play,becasue no choice are given, we need to very familiar with the topic first be4 i can know one or two ans. From the word "Crossword Puzzle", I think it is quite silly and will not be helpful and useful enough for my learning, so I don't use it. Mobile exercise experienced 201 download instances with 72 uploads. Student comments included: it can help us to review the lesson and test our understanding and it is easy and not troublesome in inputting the answeres It is good for us to test whether we are understand to content or not It can help me to achieve a better grade. It can test my understanding on each chapters. If I find out that I am not doing well on the MC questions, then I will revise that chapters again. There were, however, some frustrations including: Sometimes, I only want to check whether I understand a few questions but not all. There is no explaination on the answers. I think that it will be better if there is some explaination for each choice of each question. sometime the programm will shut down because of hang computer, and i cannot check the answere immediately and have to answere all the questions again. Tatoes recorded 305 download instances from 87 students. However, only 28 students uploaded 36 results that could be analyzed. No students (of those who uploaded data) actively explored alternative answers after arriving at the correct answer. The students stopped once they discerned the correct answer, sometimes after 1, 2 or 3 wrong attempts. Comments from the student focus groups were largely positive. The students especially appreciated both the immediacy and substance of the feedback, for example: for sure is the immediate response function. also, i can know the answer without connecting the internet. 5

So, I can downlaod it at school and then do it at home when I have time. Hints. This feature save our time to find the right answer and related topics form textbook and notes. However students also noted some problems in that: Hints is useful sometimes but not always because some hints made me more confuse. I think the lecturers and tutors should promote this during lectures or tutorials so that students may know it. Also, the download speed should be fast enough so that students do have to wait for a long time. With respect to the Tatoes supported ability to continue to explore the wrong answers after arriving at the right answer (to enable further exploration to better understand the topic), the students noted: When I get the right answer which is just because I'm lucky, I'll click all the other answers to see why not them; but for those answer that I'm quite sure, I won't as I think it is just a waste of time. when i got the right answer, i won;'t waste time to click on the other choices because i know well on that question already. Comparison of student use of the mobile applications correlated with their performance in a number of ways. Students who made use of the mobile applications performed significantly better on the midterm exam (p<0.01) and final exam (p < 0.01) and received a better course final grade (p < 0.01) compared to those students who chose not to use the mobile applications. Among the applications, Tatoes and e-tips were especially salient. Tatoes is of special interest since it had the highest instance usage and was also introduced only in the last two weeks of the course, i.e., after the mid-term exam. The content given to the students was exclusively oriented to supporting material unrelated to the midterm but directly related to the final exam. Since the mid-term exam was a good indicator of prior student study performance (p < 0.01), we adopted it as an ANCOVA covariant. Students who downloaded Tatoes instances had significant higher scores on the Final Exam (p < 0.05) and a higher Final Grade (p < 0.05) than students who did not download Tatoe instances. Tatoes was also especially well received as noted in focus group comments, for example: It gives some hints and explanations to the users. It is useful because the users can know the reasons for their wrong answers I can immediately know whether my answer is correct or not. It is a good way, because it can arouse my awareness if similar question occurred in mid-term or exam. Even the questions are not exam-oriented, you can still gain more knowledge on this subject. It is another way to help us to achieve a higher academic grade. tatoes help me most because it help me to understand the content more. 6. Discussion In general, our results provide some support for our research model (Figure 1) and associated postulates. As noted by one focus group student, i think all of these can help us to achieve better academic performanace. Empirically, those students who were motivated to use the mobile applications tended to achieve higher levels of performance as indicated on the mid-term exam, final exam and ultimate course grade. This is independent of whether the motivation was intrinsic or extrinsic noting that fewer than half of the students who downloaded application instances bothered to upload to results to receive e-token credits. The more general support for Tatoes particularly provides an element of support for constructive alignment as a moderator since the exercises were more directly in line with students learning interests and choice (based on higher levels of use and focus group comments) relative to other applications. However, we caution that these results should be seen as preliminary and not necessarily demonstrative of enhanced learning on the part of the students. For example, there is no special way to discern from these results that the pool of students who chose to use the mobile applications were not just simply better performing students compared to those who chose not to use the mobile applications, or that their learning style was more compatible with PDA use in accordance with application characteristics. Further, it is questionable whether exam performance has any special correlation with learning. Whether constructive alignment is confirmatively a moderator on performance also remains a question. There was no stratification in the sample that enabled such level of detailed comparison. How to get students 6

to explore a domain beyond the scope of a particular instance is also a remaining challenge. It is clear from the log data that students generally ceased exploring alternatives once they arrived at the right answer. However, there may have been some confusion with respect to trying to achieve the best possible mark. Active use of the distracters to confirm or query why these components of each question were not the correct answer remains to be seen. We also note that more can be done to influence general use of the mobile applications. As focus group students noted: tutors and lecturers should spend some time to promote this system. let students know it is useful is very important as it can encourage them to use it. I dont know where to do/download these exercises. Also i am not necessary to go the e-token website There should some guildlines for students to teach them how to use this system. Field trip use of Phototate amplified the importance of instructor and administrative support to create an unambiguous and seamless learning environment. As the field trip group leader stated, I look forward to how we can maximise the PDA device to engage or motivate students to better learning. Unlike Tatoes, Phototate use was preceded by in-depth pedagogical instruction given to the students by the Education Development Office (EDO) representatives, focusing on a number of issues: Students were encouraged to use the PDAs to capture their immediate reactions in their native language (Cantonese) from which they could subsequently create summarized notes and conclusions in English supported by their photos. Students were especially encouraged to capture the essence of the context surrounding the photos (and associated verbal comments) to assure that they appreciated the bigger picture and didn t miss the forest for the trees. Students were also encouraged to reflect on the use of Phototate in conjunction with support for their learning. To jumpstart the process, they were given a presentation on how to use the PDA s to enhance both short and long-term learning as well as recall. On return from the field trip, Phototate usage data was examined and a debriefing focus group was conducted. The results were interesting, albeit somewhat disturbing. The students did not use the PDAs in the expected fashion; rather, they used them casually, much as they would use their mobile phone (e.g., to take some pictures of interest) but did not make them an active part of their learning environment. Upon investigation, the reasons were both technological and behavioral, resulting in unrealized pedagogical innovation. From a technological perspective, the cameras on the PDAs were unable to capture the detail of the samples desired by the students. There was simply a lack of resolution at the focal distances they desired. On a larger scale, there were problems with the sampling and analyzing technologies the students were supposed to use that negated field analysis. From a behavioral perspective, the students were not particularly encouraged to personally explore their field environment. In particular, there were a number of academic advisors (none of whom were PDA users) who not only immediately responded to any student questions, but additionally directed the students explicitly to collect samples for later analysis. In summary, learning interventions require a prodigious amount of concerted effort with iterative and prolonged student and staff involvement and interaction as process and technology join in pedagogical evolution. This simply did not exist in this field trip. Lacking a concerted level of effort and seamless integration of technology, the students did not change habits and engage in new forms of learning. Overall, we feel that general use of the mobile applications leaves much room for improvement, both in terms of the existing applications and, especially, the newer applications that have yet to be evaluated. However, we note that currently fewer than 10% of the students in these courses actually purchased a textbook. From that perspective, mobile applications can become an increasingly important segment of the student learning time and space; we feel that we have made a solid start in that direction. We are not suggesting that PDAs will replace books. In fact, expecting students to read a traditional text on a PDA is even less feasible than expecting them to read from a book. What it does suggest is a pedagogical rethink of the way we educate digital natives to include the kinds of learning activities that excite them and, from which, they may indeed learn. 7

Our purpose in creating a portfolio of PDA-based applications is an attempt to explore the impact of learning activities that continuously travel with the students and can be accessed at the time and place of their choosing. We also increasingly expect that technological barriers to mobile use will continue to decline. For example, we expect that within a few years, the type of smartphones that all students are likely to have will eliminate the necessity to provide them with wireless PDAs and further enable more personalized mobile application development and support. The current need to download and upload from wireless hot-spots will simply disappear. Students will be continually surrounded with a learning environment that can be better customized to their interests. There are, however, a number of institutional challenges that remain to be addressed as the proliferation of mobile applications and devices increases. As Wagner and Vogel [16] have noted, emphasis moves from individual to more systemic initiatives and, ultimately, policy considerations over time. Technology interventions without examination of the broader learning systems and practices in place can easily result in a failed innovation. A key consideration is how to keep faculty effectively engaged (and supported) as technology changes. Mobile application use is a sustaining innovation to students but a disruptive innovation to many faculty and institutions. Opportunities for research are many, as we move forward to better understanding the effect of mobile device applications on learning. Some research opportunities can address limitations of the current research, e.g., populations characteristics, application stability and use over longer periods of time. Many opportunities exist for the creation and evaluation of more sophisticated applications, e.g., in the area of collaborative gaming. The current research is but a start on a long and winding road. 7. Conclusion This paper explores the impact on learning of a portfolio of mobile applications designed for use outside the classroom. Towards that end, particular attention is given to a multiple choice learning application with summative feedback running on PDAs that enabled students to prepare remotely for their final exam. Comparisons between students using the learning support versus those who chose not to do so illustrate positive support for performance enhancement with some support for constructive alignment as a moderator. However, a number of caveats are in order, especially in terms of population characteristics and consideration of the broader relevant system issues. Thus, to answer our question do mobile device applications affect learning, we feel that the evidence is tantalizingly positive, albeit weak and much in need of further study. Ultimately, we conclude that the future is challenging but bright with respect to mobile application effect on learning. 8. References [1] Alavi, M.; Yoo, Y.; and D. Vogel, Information Technology to Add Value to Management Education, Academy of Management Journal, 40(6), 1997, pp. 1310-1333. [2] Alavi, M. and B. Gallupe, Using Information Technology in Learning: Case Studies in Business and Management Education Programs, Academy of Management Learning and Education, 2(2), 2003, pp. 139-153. [3] Biggs, J. Approaches to the enhancement of tertiary teaching, Higher Education Research & Development, 8(1), 1989,.pp. 7-25. [4] Biggs, J. Teaching for Quality Learning at University. 2 nd Edition, Maidenhead, Berkshire UK: Society for Research into Higher Education and Open University Press, 2003. [5] Cole, M.; Feild, H. and S. Harris, Student Learning Motivation and Psychological Hardiness: Interactive Effects on Students Reactions to a Management Class, Academy of Management Learning and Education, 3(1), 2004, pp. 64-85. [6] Csete, J., Wong, Y. and D. Vogel, Mobile devices in and out of the classroom, In L. Cantoni & C. McLoughlin (Eds.), ED-MEDIA, 2004, Proceedings of the 16 th World Conference on Educational Multimedia and Hypermedia & World Conference on Educational Telecommunications, Lugano, Switzerland: Norfolk VA: Association for the Advancement of Computing in Education, 2004, pp. 4729-4736. [7] Deci, E. and R. Ryan, The Empirical Exploration of Intrinsic Motivation Processes, in Berkowitz (ed.), Advances in Experimental Social Psychology, New York: Academic Press, 13, 1980,.pp. 39-80. [8] Hevner, A.; March, S; Park, J. and S. Ram, Design Science in Information Systems Research, MIS Quarterly, 28(1), 2004, pp. 75-105. [9] Kennedy, D. M. Continuous Refinement of Reusable Learning Objects: The Case of the Interactive Graphing Object, In L. Cantoni & C. McLoughlin (Eds.), ED-MEDIA 2004, Proceedings of the 16th World Conference on Educational Multimedia and Hypermedia & World 8

Conference on Educational Telecommunications, Lugano, Switzerland: Norfolk VA: Association for the Advancement of Computing in Education, 2004, pp. 1398-1404. [10] Kennedy, D. M., Vogel, D. and T. Xu, Increasing opportunities for learning: Mobile graphing, Beyond the Comfort Zone: ASCILITE, Proceedings of the 21st Annual Conference of the Australian Society for Computers in Learning in Tertiary Education, University of Western Australia, Perth, Academic Computing Services, 2004, pp. 493-502. [11] Leidner, D. and S. Jarvenpaa, "The Use of Information Technology To Enhance Management School Education: A Theoretical View," MIS Quarterly, 19(3), 1995, pp. 265-291. [12] Naismith, L., Lonsdale, P., Vavoula, G., & M. Sharples, Literature review in mobile technologies and learning (No. 11). Bristol, UK: FutureLab, 2005. [13] Noe, R. (1986). Trainees Attributes and Attitudes: Neglected Influences on Training Effectiveness, Academy of Management Review, 11, pp. 736-749. [15] Piccoli, G.; Ahmed, R. and B. Ives, Web-Based Virtual Learning Environments: A research Framework and a Preliminary Assessment of Effectiveness in Basic IT skills Training, MIS Quarterly, 24(4), 2001, pp. 401-426. [16] Wagner, C. and D. Vogel, Pathway to Massive Mobility in Educational Computing, Proceedings of the 2005 International Conference on Informatics Education Research, Las Vegas, Nevada: International Academy for Information Management, 2005, CD. [17] Wood, R.; Kakebeeke, B.; Debowski, S. and M. Frese, The Impact of Enactive Exploration on Intrinsic Motivation, Strategy, and Performance in Electronic Search, Applied Psychology: an International Review, 49(2), 2000, pp. 263-283. [18] Zheng, P. And L. Ni, The Rise of the Smart Phone, IEEE Distributed Systems Online, 7(3), pp. 1541-4922, retrieved on 12 June 2006, from http://csdl2.computer.org/comp/mags/ds/2006/03/o3003.pdf [14] Nunamaker, J.F., Chan, M. and T. Purdin, Systems Development in Information Systems Research, Journal of MIS, 7(3), Winter 1990/91, pp. 89-106. 9