Available online at www.sciencedirect.com ScienceDirect Procedia - Social and Behavioral Sciences 171 ( 2015 ) 897 904 ICEEPSY 2014 Evaluating effectiveness of IDEWL using Technology Acceptance Model Ahmad Tajudin Baharin a *, Habibah Latehb, Shelena S. Nathan a, Hurhudzaifah mohd Nawawi a a School of Computing, Universiti Utara Malaysia, Sintok Kedah Darul Aman, 06010 Malaysia b School of Distance Education, Universiti Sains Malaysia, Penang, 11800 Malaysia Abstract Development in technology motivates the way of teaching and learning from different view. Electronic learning has become a trend in institutions since usage of Internet is growing high. This study evaluated the effectiveness towards the e-learning usage and acceptance of it among users. Evaluation is necessary to ensure future usability and the acceptance of online learning among users. The web based online learning IDEWL was tested in this study. IDEWL is an enhanced e-learning with interactivity module such as web conferencing which differs from normal video on demand based online learning. Total of 250 students participated in this evaluation through answering the questionnaire distributed. Components that was evaluated consist of interactivity, perceived usefulness, perceived ease of use, intention to use and effectiveness. Data collected was analyzed using SPSS based on TAM approach through hypothesis test. The result indicates that students find e-learning is effective with the enhancement of the inter-activity modules and perceived usefulness as well as perceived ease of use has influenced on each other and seamlessly connected with the effective-ness of the e-learning system. 2015 2015 The The Authors. Authors. Published Published by by Elsevier Elsevier Ltd. Ltd. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/). Peer-review Peer-review under under responsibility responsibility of the of the Organizing Organizing Committee Committee of ICEEPSY of ICEEPSY 2014. 2014. Keywords: e-learning, TAM, effectiveness, evaluation. * Corresponding author. Tel.:+06-04-928-5198; fax: +6-04-928-4259 E-mail address: atajudin@uum.edu.my 1877-0428 2015 The Authors. Published by Elsevier Ltd. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/). Peer-review under responsibility of the Organizing Committee of ICEEPSY 2014. doi:10.1016/j.sbspro.2015.01.207
898 Ahmad Tajudin Baharin et al. / Procedia - Social and Behavioral Sciences 171 ( 2015 ) 897 904 1. Introduction For the past few decades Internet usage and the application connected with it has be-come popular and demanding among computer users. Together with this rapid change in communication, knowledge is given an important role and it has grown beyond expectation and influences our daily life. Impact of these rapid growth, challenges in education has in-creased, thus research are being conducted tremendously and help the development of technology in learning [1]. Even though current e-learning embedded with asynchronous communication such as forums and video on demand, yet it may not give immediacy that needed for an attractive e-learning system. The time lapsed which could happen between question and answer between users may not effective and the engagement with the system than using synchronous communication such as web conferencing since it happen on real time basis. Both learner and instructor are able to have two way communication in the process of teaching and learning more effective [2]. 1.1. Interactivity Distance Education Web Learning (IDEWL) Interactivity plays an important role in distance learning, thus implementation of it done to a distance learning portal and named as Interactivity Distance Education Web Learning (IDEWL). This is the combination of web conference tool in open source learning management system where it helps the user in creating online learning materials more interactive. And besides having the normal learning tools as many e-learning does, this IDEWL incorporate web conference to enable user to communicate with additional functions such as e-blackboard, online survey, writing and drawing as compromising the purpose of the development[3]. The primary objective of the IDEWL is creating the all new distance learning material that enhances with interactivity tool that attract users. This interactivity tool will lessens the effort of learning and becomes more engaging tools for users especially for distance learners. Besides that, IDEWL also focus in creating asynchronous communication skills among users and provide an effortless system available all the time with location independent for users to access and get connected with the content [4]. Figure 1: Screenshot of IDEWL
Ahmad Tajudin Baharin et al. / Procedia - Social and Behavioral Sciences 171 ( 2015 ) 897 904 899 2. Related Works Interactivity is what allows learner communication in the third category of e-learning interaction which is interaction between student and content. It enhances the engagement into the system and users became active participant. It is an interaction that enables two thing to work together and communication between each other such as between two learners, between learner and instructor and between learner and the communication device itself [5]. Interactivity also can be seen as communicating verbally and non-verbally through Internet and channels to communicate via e-learning is determine by the programs offered to them. According to Merrill [6], effectiveness is thus seen as degree of determining online learning characteristic in delivering educational content to learners. According to him also, higher levels of interactivity are hypothesized to generate higher student performance. Interactivity is seen as an important aspect for e-learning since it enable the user to participate actively in e- learning and the content available in it. It grab the attention for the individuals need thus instructor should be trained so that user could search and observe them-selves and get more attention on e-learning. As Bloom [7] said in his studies that education much have direct communication within learner and instructor and opportunities should be created for autonomous learning. TAM is the most frequent used model in measuring user acceptance level for IT field. This is also a tool that predict the user intention in using the technology [8]. It is the relationship between system design, PU, PEOU and user acceptance. TAM is widely used for empirical studies in user acceptance research and contributes in determining user perception of the technology [9]. 3. Methodology In this study, I adopted the Input Process Output (IPO) model [10] in conducting the evaluation on the interactivity module in IDEWL. We distribute 250 questionnaires to students before and after class hours and collected it personally. Out of the 250 questionnaire distributed, only 223 questionnaire were found usable for the analysis which is sufficient for multivariate data analysis techniques. Figure 2: Input Process Output (IPO) Questionnaires is prepared based on variable in the revised Technology Acceptance Model (TAM). The scales are adopted from initial scales used by Chuttur [11] but modified according to the e-learning that being evaluated.
900 Ahmad Tajudin Baharin et al. / Procedia - Social and Behavioral Sciences 171 ( 2015 ) 897 904 TAM model is described as below. To test the objective data was collected and analyzed through hypothesis test. Hypothesis that was created to be tested as below is: H1: System interactivity will have a positive effect on perceived usefulness of the e-learning systems. H2: System interactivity will have a positive effect on perceived ease of use of the e-learning systems. H3: Perceived ease of use will have a positive effect on perceived usefulness of the e-learning systems. H4: Perceived ease of use will have a positive effect on the intention to use the e-learning systems. H5: Perceived usefulness will have a positive effect on the intention to use the e-learning systems. H6: Intention to use will have a positive effect on the effectiveness of e-learning systems. Figure 3. Proposed Model of TAM 3.1 Interactivity (I) This measures the implementation used in the technology such as tools and modules develop. This is also to define the enhancement of the technology to be evaluated [12]. 3.2 Perceived Usefulness (PU) TAM is determinant in providing acceptance of technology and explain the behavior of user across broad range of end user technologies. Perceived usefulness as discussed by Davis [13] where a person use a system enhances engagement of them in performing and using the technology. Davis [13] also defined PU as improvement in user performance and an application that increase the user engagement in the technology. 3.3 Perceived Ease of Use (PEOU) On the other end, PEOU is the degree where a particular system is free from effort and prove to have efficiency and effectiveness [14]. Davis [13] also define that PEOU is judgment of the efforts that required to use in handling the technology or the system. 3.4 Intention of Use (IU) In TAM, IU is for predicting the user acceptance where it shows the user enthusiasm in using the technology which increase in proving the effectiveness of the system. It also measures the user satisfaction where if they have positive influence on usefulness and ease of use, it derived the continuous use of the system. Thus, IU is re-liable for this study as well because it will provide positive satisfaction if the system effectiveness.
Ahmad Tajudin Baharin et al. / Procedia - Social and Behavioral Sciences 171 ( 2015 ) 897 904 901 3.5 Learning Effectiveness (E) In this study, effectiveness is measured in IDEWL which are the dependent variable to other variables to be measured. This will measure the effectiveness reached by the respondent in using IDEWL and their perception to use it for future as learning medium. The result of measured questionnaire was analyzed to identify the objective of the studies. The increasing number of user in e-learning is studied by the implementation of interactivity module calculated and displayed the result and concluded. Questionnaire goes on a process of measuring according to the model that will produce result. In this process part, TAM approach was used in analyzing the data by using Statistical Package for the Social Sciences (SPSS) an analysis software and AMOS, an embedded tool in SPSS. Cronbach Alpha is used in calculating reliability which is a degree testing of what questionnaire is measuring. Average of the variable calculated and measurement value is identified. Mean is used in this study to identify the acceptance of the interactivity in the IDEWL which influence the effectiveness of the system. And finally conclusion is made ac-cording to the analysis done. 4. Results These are based on the results analyzed using SPSS and AMOS as stated earlier. As for the demographic part, it was measured and illustrated with the respondent s information. The proposed hypothesis was tested using path analysis. 4.1 Path Analysis In this analysis, the collected data was tested to match the hypothesis created as dis-cussed below. As to meet the objective suggested, these results obtained. Path analysis allows the simultaneous modelling of several related regression relationships. In path analysis, a variable can be a dependent variable in one relationship and an independent variable in another. These variables are referred to as mediating variables. Table 1. Regression Weights: (Default model) Estimate S.E. C.R. P Mean_PEOU <--- Mean_I.330.043 7.582 *** Mean_PU <--- Mean_I.380.060 6.343 *** Mean_PU <--- Mean_PEOU.199.082 2.419.016 Mean_IU <--- Mean_PU.147.063 2.326.020 Mean_IU <--- Mean_PEOU -.009.080 -.112.911 Mean_E <--- Mean_IU.141.082 1.718.086 4.1.1 To examine relationship between PEOU and PU towards Interactivity in using e-learning In order to test the objective and the effectiveness of E-learning system with interactivity we proposed the hypothesis H1: System interactivity will have a positive effect on perceived ease of use (PEOU) of the e-learning systems. The Coefficient of Interactivity is 0.33 with positive sign and significant as (p<0.05). H1 is accepted as p<0.05. The results are shown in Table 1.
902 Ahmad Tajudin Baharin et al. / Procedia - Social and Behavioral Sciences 171 ( 2015 ) 897 904 We analyse the interactivity with perceived usefulness for assessing the role of interactivity in using e-learning. The results of path analysis for PU and I, has positive sign with coefficient value 0.38. We can accept (H2: System interactivity will have a positive effect on perceived usefulness of the e-learning systems) as the p<0.05. 4.1.2 To examine relationship between PEOU and PU towards Interactivity in using e-learning To assess the effects of Intention to use towards the IDEWL model. We assess the relationship between intention to use with perceive usefulness and perceived ease of use. We test the hypothesis H3: Perceived ease of use will have a positive effect on perceived usefulness of the e-learning systems. The hypothesis is supported as p<0.05 and also sign of the coefficient showed positive relationship 0.19. The results are shown in Table 1. The hypothesis H4: Perceived ease of use will have a positive effect on the IU of the e-learning systems, the results of the path analysis indicates that IU has positive impact on perceived usefulness as the coefficient is 0.147, thus we cannot reject the null hypothesis which is supported as p<0.05. Our hypothesis H5: Perceived usefulness will have a positive effect on the Intention to Use the e-learning systems, is not supported as the p>0.05 but the sign also the sign of the coefficient is also negative. Thus we cannot accept the hypothesis that perceived ease of use has positive affect on intention to use e-learning system. The results are shown in Table 1. The system interactivity and the effectiveness is assessed by H6: Intention to use will have a positive effect on the effectiveness of e-learning systems. The results of the path analysis showed that the intention to use have positive relationship but the relationship is in the form of weak relationship as p>0.05 but also less then p<0.10, which can be considered as weak form of relationship. The results are shown in Table 1. Figure 4. Technology Acceptance Model (TAM) Path Analysis 5. Conclusion The main objective of this study was to evaluate the acceptance of the interactivity in e-learning and in this case we use IDEWL in order to evaluate. This is due to the concern about student view on e-learning that become the main thing for a successful development for education purposes [16]. As for this study, we use TAM to test the
Ahmad Tajudin Baharin et al. / Procedia - Social and Behavioral Sciences 171 ( 2015 ) 897 904 903 hypothesis proposed which consist of Interactivity, PU, PEOU, IU and Effectiveness. Both PEOU and PU are seen to be positively influence students attitude towards the system that leads to the intention of using the system. This intention of use is also influences the effectiveness of the e-learning since it is rich in IT learning environment that makes the system more attractive and easy to use. Therefore, all the generated hypothesis in this study were supported even though H6 is quite weak but still it is seen to be supported due to other items in the study. 6. Discussion and Future Work According to analysis of the results obtained from the questionnaire, it is strongly agreed that it is acceptable of having interactivity module in e-learning as it leads to the effectiveness of the system. The results of the analysis obtained from this study are ready to be used in contrasting e-learning that embedded with interactivity modules since it is acceptable among users. Interactivity measures the level if the modules in e-learning while PU and PEOU the efficiency of the system that leads to the IU and indirectly enhance the level of effectiveness of the system overall. In future this study can be expanded for a large number of respondents and including instructors since both are involved in using the e-learning in education matter. This study incorporates to TAM approach in evaluating the effectiveness. In future, this can be done in other relevant approach and more information and results can be gained. Acknowledgements I would like to thank everybody for this study for the thoughtful comments and assistance with the manuscript. References Ahmad T. Baharin, Habibah (2011). Implementing of Interactivity Distance Education Web Learning (IDEWL) as a Learning Tools in Distance Education. Kolokium Pascasiswazah PPPJJ. Alessi, S. M., & Trollip, S. R. (2001). Multimedia for learning. Methods and development. Amer, T. (2007). E-learning and Education. Chuttur, M. (2009). Overview of the technology acceptance model: Origins, developments and future directions. Clark, R. C., & Mayer, R. E. (2011). E-learning and the science of instruction: Proven guidelines for consumers and designers of multimedia learning. John Wiley & Sons. Davis, F. D., Bagozzi, R. P., & Warshaw, P. R. (1989). User acceptance of computer technology: a comparison of two theoretical models. Management science, 35(8), 982-1003. Gay, L. R., Mills, G. E., & Airasian, P. W. (2009). Educational research: Competencies for analysis and applications (ed.). Columbus, OH: Pearson.
904 Ahmad Tajudin Baharin et al. / Procedia - Social and Behavioral Sciences 171 ( 2015 ) 897 904 Hegazy, A., & Radwan, N. M. (2010, November). Investigating learner perceptions, preferences and adaptation of e- learning services in Egypt. In Education and Management Technology (ICEMT), 2010 International Conference on (pp. 167-172). IEEE. Levin, T., & Long, R. (1981). Effective Instruction. Association for Supervision and Curriculum Development, 225 North Washington Street, Alexandria, VA 22314 (Stock No. 611-80212, $6.50). Merrill, M. D. (2002). First principles of instruction. Educational technology research and development, 50(3), 43-59. Money, W., & Turner, A. (2004, January). Application of the technology acceptance model to a knowledge management system. In System Sciences, 2004. Proceedings of the 37th Annual Hawaii International Conference on (pp. 9-pp). IEEE. Sekaran, U. (2006). Research methods for business: A skill building approach. John Wiley & Sons. S., Karayan, S. and J., Crowe (1997). Student perspectives of electronic discussion groups, The Journal: Technological Horizons in Education, 24. The Impact of Learning Management Systems in Universities Commonwealth of Learn ing.[online]11june2009,http://www.col.org/resources/speeches/2009presentation/pages/2009-06-11.aspx (21 March 2014). Titchkosky, L., Arlitt, M., & Williamson, C. (2003). A performance comparison of dynamic Web technologies. ACM SIGMETRICS Performance Evaluation Review, 31(3), 2-11. Venkatesh, V., & Davis, F. D. (2000). A theoretical extension of the technology acceptance model: four longitudinal field studies. Management science, 46(2), 186-204.