MODELLING COLLABORATIVE COMPETENCE LEVEL USING MACHINE LEARNING TECHNIQUES

MODELLING COLLABORATIVE COMPETENCE LEVEL USING MACHINE LEARNING TECHNIQUES Laura Mancera Valetts, Silvia Baldiris Navarro, Ramón Fabregat Gesa Institute of Informatics and Aplications (IIiA) ABSTRACT Using open e-learning platforms as a tool to support the learning process has become an international tendency. Specially, in order to motivate the achievement of desired competences in a lifelong learning process. In this context, personalization of the e-learning process according to user characteristic is a critical point, in particular, to deliver activities or learning resources adjusted to the user. In this paper, we are focusing in how to model the collaborative competences level achieved for users in a virtual learning environment. With this proposal, a user model based on competences definition and also on user interaction is proposed. Two Machine Learning techniques have been applied in order to generate the proposed model, specifically Clustering Techniques. KEYWORDS Competence based E-learning process, User modelling, Collaborative Competency Level, Machine Learning, Learning Management System. 1. INTRODUCTION The Learning Process based on competences is a focus of great interest in recent times, not only in educational field but also in business field, because this ensures a continuous and efficient learning. Lifelong Learning Process [Burgos, 2006] has been created like a global trend looking for a new vision of education and it is specially supported by information technologies in order to obtain best learning results. In this context many researchers have contributed to this interest. E-learning platforms or virtual learning environment, and applications related with E-learning Process have also been developed [Baldiris, 2007]. However, two facts are necessary to distinguish at this moment. The first one is limitations of actual e- learning platform, especially their restricted capacity to offer personalized content and activities to each student according to their abilities and interests. The second fact is related to cultural learning, which indicates that the biggest learning effectiveness is obtained when students collaborate and learn from other people by debating ideas and analyzing multiple perspectives [Bustos, 2006]. In this paper, we propose to address the personalization and cultural learning problems through analysis of student s collaborative competence levels as a mechanism to take adaptation decisions. Clustering machine learning techniques to perform a specific user model, in particular, Kmeans and EM algorithms were compared with the purpose of obtain an adequate collaborative user model. The necessary data for the model creation have been extracted from Logs and databases stored by e-learning platform (dotlrn). Modelling process can be used to take decisions such as: to perform recommendations to each student, to give a global vision of the level of collaborative competence of the class to the teacher, to analyze if these collaborative competences contribute to the achievement of the desired level of a certain specific competence or constitute groups according to their interaction characteristics. 2. RELATED WORKS Machine learning techniques can be used to create user models in order to support future actions [Webb et al, 2001] and the non static nature of user data permit to create realistic and dynamic user models which change 56

IADIS International Conference e-learning 2008 over time and generate the best results of adaptation [Gaudioso and Boticario, 2003]. Below we mention some projects that have made use of these techniques. [Millán et al, 2000] uses of probability theory for student diagnosis, specifically, Bayesian Networks techniques. In this research learners are classified in accordance with their knowledge level. [Durán and Costaguta, 2007] proposed a work to discover learning styles, in which FarthestFirst algorithm is used for modelling a group of students according to their strengths and preferences to take and process information. [Sanabria, 2006] applied clustering and classification techniques to group students by their obtained qualifications. [Gaudioso, 2002] described a suitable framework to develop user models that contain interaction data and combine machine learning techniques to make these models more flexible. [Barros and Verdejo, 2000] proposed a system that allows developing collaborative learning experiences across distance and analyze operational methods of the groups when performing common tasks. The system uses artificial intelligence, specifically agent technology for the tasks. 3. METHODOLOGICAL APPROACH The learning process based on competences begins establishing and characterizing those attributes that guarantee an adequate performance, such as knowledge, attitudes, values and skills. The European Higher Education Area divides these competences into two dimensions, the generic or transversal competences and the specific competences. Generic competences are those that are transferable to a multitude of functions or formation programs. The specific ones are directly related to each thematic area [Tuning Project, 2004]. Our interest is based in research works that suggest that a higher effectiveness in a learning process is achieved when students collaborate and learn from other students and teachers, questioning ideas and creating multiple perspectives [Bustos, 2006]. Due to this reason, our approach is focused on transversal competences, in particular on collaborative competences. E-learning platforms offer a set of tools to generate collaborative activities in courses. The most common ones are forums, chats, shared files, comments, and e-mail services, among others. The question is How to define user s collaborative competences level through user interactions and behaviour on collaborative tools in E-learning platforms? In order to answer this question, we propose a user model to describe user s collaborative competence. The proposed model consists in two sub models: a participation model inferred through student participation with collaboration tools and an access model inferred through student register of platform access. Participation model data are obtained from a database. Access model data are obtained from navigation logs. We modelled these aspects separately to get a better collaborative competence profile definition of each student. In this manner, we guaranty that a student with high collaborative profile is the one that keeps their participation level throughout the entire course. Likewise, with this approach is possible to detect strangeness in the student collaborative behaviour. In order to make the participation model we consider different collaboration categories defined in the logical frameworks approach [Santos and Boticario, 2004]. The learner can move from Non_Collaborative to Communicative, until the highest appreciated level (Useful). These six collaboration values can also be grouped into three levels, i.e low, medium and high. Characterizations of these levels are shown in table 3. Table 1. Interaction levels COLABORATIVE VALUE DESCRIPTION LEVEL NonColaborativo_Learner Student who behaves as if there were no collaborative tools or their level of collaboration is very low compared with other students. Low Communicative_Learner Share information with other learners using the available communications tool Participative_Learner Interacts frequently in the course Medium With Initiative_Learner Stars the proposed activities without waiting for other student s contributions Insightful_Learner Make contributions and comments on activities from other learners that later receive high scores. High Useful_Learner Makes comments and contributions that are considered by other learners Otherwise, in order to implement access model, three levels based on the user navigation performance are considered. Characterizations of these levels are shown in table 2. 57

Table 2. Activity levels COLABORATIVE VALUE DESCRIPTION LEVEL pasive_learner Do not access to platform constantly Low intermediate_learner Moderated access to platform very often. Medium active_learner Log-in frequently in the course Hight The collaborative competency levels that we proposed result of join those models presented above. Table 3 presents this join. COLLABORATIVE COMPETENCY LEVEL High collaborative competency level Medium collaborative competency level Low collaborative competency level Table 3. Collaborative competency level DESCRIPTION If a student log-in often and participate actively his profile is clearly collaborative. If a student log-in and participate moderately, their competence level is medium. Likewise, if he participates moderately and log-in actively, their profile is medium. These students can improve their participation actively. If a student log-in and participate rarely his profile is non collaborative and requires to participate. If a student has many participations and low level of access, he probably may not have a collaborative profile, it requires attention. 4. RESULTS OF APPLY THE APPROACH USING CLUSTERING ALGORITHMS We have considered the problem of the student's location at a given level of collaborative competence as a grouping problem. This decision was taken because it is impossible to know previously, in which class the student belongs [Boticario et al, 2006]. Clustering techniques allow grouping students in subsets (classes) according to their participation level and access similarities (see above section). 4.1 Participation Model To develop this model we have worked with data stored from forum and chat tools. However, it is convenient to mention that it is feasible to use other tools. Each data type has a different treatment. In case of the forum, two possibilities of actions were considered and quantitatively differentiated. The first one is when the user adds an original intervention in the forum (new thread), and the second one, when the user answers previous set interventions (respond to an existing message in the thread). A student who proposes new ideas in the forum does not have the same characteristics of collaboration that the one that responds to ideas previously proposed. On the other hand in the chat only sent messages are used. It is also necessary to mention that interaction through forum has more weight than interaction through the chat. Based on them, considerations proposed by [Gaudioso, 2002] were taken into account as shown in the following table: Table 4. Chat and Forum Interaction Levels CHAT FORUM High Medium Low High High Medium Medium Low High Medium Low The clustering algorithm used to implement the models was Expectation-Maximization (EM). This is an algorithm used in statistics for finding maximum likelihood estimates of parameters in probabilistic models. To obtain an adequate model, we realized a comparison with other algorithms such as simple Kmeans algorithm. EM algorithm performed better according to the location in the clusters. Table 5. Samples of training examples Student_Id Course_Id Forum_Participation1 Messages2 Responses3 Ratio1 Ratio2 Ratio3 Chat4 Ratio4 1 1 10 6 4 1.81 1.95 1.63 8 1.37 58

IADIS International Conference e-learning 2008 Total attributes for the model correspond to the labels mentioned at the table 5. Attributes and their values represent the training set used to realize the model. The Algorithm s performance was tested using WEKA tool. In table 6, the results of the algorithm are presented. 4.2 Activity Model Table 6. Results of EM algorithm applications CLUSTERED INSTANCES Cluster Students % 0 20 50 % 1 8 20 % 2 12 30 % This model uses stored data in logs. We take into account inputs to the platform every week, month and also every three months. EM algorithm was also utilized to generate the model. In table 7 and 8, we show some of the final data used for the data training and results of the algorithm. Table 7. Samples of training examples User_Id Course_Id Visits_Week Visits_Month Visits_ThreeMonths Ratio1 Ratio2 Ratio3 1 1 5 17 40 1.70 1.51 1.35 Table 8. Results of EM algorithm application CLUSTERED INSTANCES Cluster Students % 0 9 23 % 1 19 48 % 2 12 30 % Finally, by the analysis of the participation and access models, a classification of students according to their completely collaborative competency level was obtained (see table 9). Table 9. Results of Collaborative Competence Level COLLABORATIVE COMPETENCE LEVEL Level Students % 1 6 15 2 16 40 3 13 32.5 4 5 12.5 5. CONCLUSION AND FUTURE WORKS In this work a user model to define collaborative competence levels of students through collaboration behaviour is presented. User modelling process was supported by using machine learning techniques, in particular, it was verified the efficiency of the clusters techniques to model student groups. Model is used by adaptation process to generate different actions in a learning platform, such as realize recommendations to the students or teachers, to propose activities to reinforce the student level to collaborate, or to support the task of creating groups. The study carried out provides an opportunity to analyze whether inducing Collaborative Competences we are also contributing to achievement of desired specific competencies of the learning process. Model implementation will be integrated with the model proposed by ADAPTPlan project [Baldiris, et al, 2008] which uses different kinds of IMS Specifications in order to offer an integral and adapted learning design to users. This learning design includes collaborative works and the model presented in this paper will be a support to define this type of activities. 59

ACKNOWLEDGEMENT Authors would like to thank the Spanish Science and Education Ministry for the financial support of ADAPTAPlan project, and by Programme Alban, the European Union Programme of High Level Scholarships for Latin America, scholarship No. E06D103680CO. REFERENCES Baldiris, S., 2007. Degree Project memories: Modelado de competencias en sistemas de gestión de aprendizajes. Universitat de Girona. Spain. Baldiris, S., et al, 2008. Integration of educational specifications and standards to support adaptive learning scenarios in ADAPTAPlan. International Journal of Computer Science and Applications (IJCSA). Special Issue on New Trends on AI techniques for Educational Technologies. Vol 5, 1. Barros, B. and Verdejo, M.F., 2000. Un sistema para la realización y evaluación de experiencias de aprendizaje colaborativo en enseñanza a distancia.revista Iberoamericana de Inteligencia Artificial, Vol. 9, pp 27-37. Boticario, J.G. et al, 2006. Aprendizaje Automático.Sanz y Torres Publishers, Madrid, Spain. Burgos D., et al, 2006: TENCompetence: Construyendo la Red europea para el Desarrolo Continuo de Competencias. Inteligencia Artificial. Revista Iberoamericana de Inteligencia Artificial. No.23 (2004), pp 34-52. Bustos C, 2006. Evaluación de apoyo al aprendizaje colaborativo en entornos de e-learning. Chile. Durán, E. and Costaguta, R., 2007. Minería de datos para descubrir estilos de aprendizaje: Revista Iberoamericana de Educación (ISSN: 1681-5653). Vol. 42/2. Gaudioso, E., 2002. Degree Project memories: Contribuciones al Modelado del Usuario en Entornos Adaptativos de Aprendizaje y Colaboración a través de Internet mediante técnicas de Aprendizaje Automático. Universidad Complutense de Madrid. Gaudioso, E. and Boticario, J.G., 2003. Towards web-based adaptive learning communities. In Artificial Intelligence in Education. Amsterdam, Netherlands, pp. 237-244. Millán, E. et al, 2000. Adaptive Bayesian Networks for Multilevel Student Modelling. Proceedings of the 5th International Conference on Intelligent Tutoring Systems. London, UK, pp. 534-543. Sanabria, J.A., 2006. Degree Project memories: Sistema de personalización Web para el proceso de aprendizaje en una plataforma de educación virtual. Universidad Nacional de Colombia. Santos, O.C., Boticario, J.G., 2004. Supporting a collaborative task in a web-based learning environment with Artificial Intelligence and User Modelling techniques. Proceedings of the VI International Symposium on Educative Informatics (SIIE 04). Santos, O.C. et al, 2007. Why using dotlrn? UNED use cases. FLOSS International Conference. Jerez, Spain. Tuning Project, 2004. Educational Structures in Europe. Universities contribution to the Bologna Process. Witten, I.H., Frank, E, 2005. Data Mining. Practical machine learning tools and techniques, 2nd Edition. Morgan Kaufmann Publishers, San Francisco, USA. 60