The 9 th International Scientific Conference elearning and software for Education Bucharest, April 25-26, / X

Transcription

1 The 9 th International Scientific Conference elearning and software for Education Bucharest, April 25-26, / X DATA MINING SOLUTIONS FOR DETERMINING STUDENT'S PROFILE Adela BÂRA, Anca ANDREESCU, Iuliana BOTHA, Alexandra FLOREA, Manole VELICANU Department of Economic Informatics and Cybernetics, University of Economic Studies, Romana Square, Bucharest, Romania bara.adela@ie.ase.ro, anca.andreescu@ie.ase.ro, iuliana.botha@ie.ase.ro, alexandra.florea@ie.ase.ro, manole.velicanu@ie.ase.ro Abstract: The importance of this article comes from the complex issue of determining the students profile in order to develop educational activities and to improve their technical or social skills. The paper presents our experimental results and methods used to determine students profile based on questionnaires collected from our university. We applied different methods of data extraction and analysis in order to assess their comparative effectiveness for determining the profile of our students in order to enroll them in extra curriculum activities such as international competition, internships, research and development. We consider the data mining techniques to be more efficient and thus we applied several techniques, supervised and unsupervised learning algorithms. We consider very useful to determine the profile for each student and also to group them in clusters. Based on questionnaires we extract and load data into a database and build the data mining models that function for construction, implementation, testing and manipulation of data. These models are based on a series of algorithms for classification, prediction, regression, clustering, association, selection and data analysis. The results are relevant; we manage to obtain an accuracy of 95% in several models. Therefore, the subject of students profile is a major research area due to its effect. In this research various attribute selection and data mining techniques have been used to build a few predictive models for this subject. It has been found that Logistic Regression performs very well, followed closely by the Support Vector Machines. Further work is under progress to apply the results in terms of clustering and developing new educational programs based on this clusters. Keywords: data mining, student s profile, questionnaire, cluster, classification. I. CONSIDERATIONS ABOUT THE IMPORTANCE OF DATA MINING SOLUTIONS Our research was conducted within the Faculty of Economic Cybernetics, Statistics and Informatics, which is one of the most prestigious educational institutions in Romania in the field of cybernetics and economic informatics. Through its academic curriculum, the ECSI faculty has a wide opening to the values and standards of European higher education, providing to its graduates multiple possibilities and great opportunities for professional affirmation. The overall objective of the research was to determine the profile of our students, in order to enroll them in extracurricular activities such as international competitions, internships, scientific research and development. In order to achieve this objective and to predict individual student profile, we used data mining techniques on questionnaires that collect data like personal information of the student, academic and technical background, but also previous implication in extracurricular activities. In [1] we have defined data mining as the process of extracting previously unknown and potentially useful knowledge from large databases. Data mining enables data analysis, by integrating multiple technologies, such as artificial intelligence, statistics, mathematics etc. All these are used for discovering correlations and association between data, and also for building predictive models, as we 284

2 shown in [2]. Knowledge gained by applying these methods must be interpreted and validated, which can be extremely difficult. To better understand the importance of data mining techniques and their multiple possibilities of implementation in the educational area, we have identified the current state of art. In the context of the new economy, higher education institutions have become interested in predicting the profile of the students involved in e-learning activities. Thus, the study [3] presents some data mining techniques on data collected from the e-learning activities of the students who use Moodle platform [4], uses data mining classification technique for predicting students final grades based on their online activity on the specific web-platforms, [5] and [6] developed profiles for the users of the web-based learning systems, in order to analyze students activities and performance, and also for providing them some guidelines. The problem of students retention is discussed in the papers [7] and [8], in which are analyzed the students performance through different data mining techniques such as decision trees, neural networks and linear discriminate analysis. An estimation of students retention and academic success by implementing statistical analysis in SPSS software is presented in [9]. Also, in another study [10], the author uses data mining to group students in clusters for predicting the educational paths of the students and alumni. In our research, the knowledge discovered by applying data mining techniques will enable the faculty management to have advanced approaches in instructing students, predicting courses performance, determining individual students behaviors and their availability for being enrolled in extracurricular activities. Also, the results can produce important information for the faculty evaluation report regarding students insertion on the labor market (preceded by internships) and international recognition of the faculty name (through competitions and scientific research dissemination). II. DATA MINING ALGORITHMS In particular, the concept of data mining indicates the exploration and analysis of a data set, usually large, in order to identify patterns in the data that would lead to the extraction of relevant information and obtain significant repetitive rules [11]. In [12] the concept of data mining is defined from two perspectives: a) technically, it is the use of statistics or other analytical techniques for data processing and analysis in order to identify patterns and representative trends; b) economically, the concept means extracting and using information and relevant knowledge for building recommendations and decisions. The success of a data mining project consists of properly combining the two perspectives. Currently, data mining plays an increasingly important role both in theoretical research on data analysis and in practical applications. The most commonly methods used are classification, clustering, regression or dependencies modeling [13]. Cluster analysis is used to determine subgroups of the population who "belong" together. In other words, it seeks to identify "islands of simplicity" in data [14]. Clustering technique works by calculating a multivariate distance measure between observations. Observations that are close to each other are grouped in a cluster. Clustering algorithms belong to three broad categories: agglomeration methods, divisive methods and k-means clustering. In our project we used k-means algorithm. This approach is an attempt to divide the dataset into a fixed integer number of clusters. The algorithm starts with k observations randomly chosen as cluster centers and then attempts to group all other observations into these clusters. Iteratively, the process adds and deletes observations from different clusters until no relevant improvement is obtained. In data analysis and pattern recognition, classification is an elementary task and it involves the creation of a classifier, which is a function that assigns a class label to instances represented by a set of attributes. Besides several approaches based on functional representation, such as decision trees, neural networks or rules, one of the most effective classifiers in terms of predictive performance is the so-called Naïve Bayes classifier [15]. For representing, using and learning probabilistic knowledge Naïve Bayes offers a simple approach, with clear semantics [16]. It also has many practical applications including document classification and medical diagnosis. 285

3 As the name suggests it is based on Bayes Theorem, having the following formula: P(h/D)=P(D/h)*P(h)/P(D). The aim of this theorem is to determine the conditional probability of the h hypothesis to happen given the data D, knowing the a priori probabilities of h and D and the inverse conditional probability P(D/h). Practically, this theorem is applied because, in many situations, it is easier to estimate the three probabilities on the right-hand side than to estimate the probability of the left-hand side directly [17]. III. DATA ACQUISITION AND PROCESSING Data necessary to accomplish the analysis are collected using an online survey, a questionnaire organized into two distinct parts. The first part completed by professors involved in the evaluation process, and the second part completed by the students. These two categories are the target respondents for the questionnaire. One of the most important elements in the design of such a questionnaire is the establishment of accurate information to be obtained from participants, information that must be closely linked to research objectives. To do this, we identified the attributes that are needed in the data mining analysis that follows, building around them the questions contained in the survey. The questionnaire section addressed to teachers contains information about students assessment. This assessment takes into account aspects related to academic performance of students, both during the course of the class and in the final assessment, technical skills, communication and the likelihood of participation in extracurricular activities, based on the degree of involvement previously demonstrated. Of these, the EVAL_SCORE attribute requires the calculation according to an algorithm that takes into account four factors, and the formula that underlies this attribute is as follows: EVAL _ SCORE = p1. Pn + p2 Po + p3 Pc + p4 Pi (1) where: P n = marks obtained in the subjects of study (Databases, Oracle DBMS, Programming I and II); P o = participation in Oracle Academy (OAI) programs; P c = participation in student competitions or scientific sessions; P i = participation in internship or other training programs for students; p i, i=1..n = indicator s weight Teachers realize this attribute calculation before completing the questionnaire, with the other four questions being evaluated on the spot, as they require issuing value judgments about particular skills of students and quantifying these skills. Attributes extracted based on this section of the questionnaire are summarized in the table 1 and those taken from data provided by students can be seen in table 2. TABLE 1 - ATTRIBUTES EXTRACTED FROM THE TEACHER S SECTION Completed by Attribute Data type Meaning EVAL_SCORE NUMBER Scores obtained after final evaluation of targeted subject matters. EVAL_TECHNIQUE NUMBER Score given by the student s coordinating professors of Database and DBMS Oracle courses for technical abilities. Professor EVAL_ACTIVITY NUMBER Score given by the student s coordinating professors of Database and DBMS Oracle courses for activities during the term. EVAL_COMMUNICATION NUMBER Score given by the student s coordinating professors of Database and DBMS Oracle courses for communication abilities. PROBABILITY VARCHAR2 Probability of selection / participation in extracurricular activities (internships, competitions etc) When designing the questionnaire, we wanted it to be as simple and clear as possible. Therefore we chose to divide the representative questions asked to students in two categories, personal questions and study-related questions. 286

4 TABLE 2 - ATTRIBUTES EXTRACTED FROM THE STUDENT S SECTION Completed by Attribute Data type Meaning NAME_STUDENT VARCHAR2 Student s name UNIQUE_CODE VARCHAR2 Personal numerical code: a personal identifier for each Romanian adult MAT_NO VARCHAR2 Student identifier AGE NUMBER Student s age SEX VARCHAR2 Sex. Possible values: M/F YEAR VARCHAR2 Student's academic year. Possible values: II, III, MS CLASS VARCHAR2 Student s class DATE_COMPLETION DATE Date of questionnaire completion DATE_MODIF DATE Date of questionnaire modification Student DATE_END DATE Date the student completes his/her studies and can no longer change the questionnaire PROFILE VARCHAR2 Profile of the activity he wishes to undertake. Possible values: Research and Development, Consulting, Technical DOMAIN VARCHAR2 Field they wish to pursue for their work. Possible values: Research and Development, Consulting, Programming, Management, Testing Services, Analysis, Design and Economics. CS_MASTER NUMBER Intention to continue studies in master programs. Possible values: 3-in Database & DBMS, 2-in Informatic Economics, 1-in other areas CS_DOCTORAL NUMBER Intend to continue their studies in doctoral programs. Possible values: 3-in Database & DBMS, 2-in Informatics Economics, 1-in other areas In figure 1 we can see the questionnaire section which is to be completed by students. As mentioned previous, this section is trying to offer a clean visual effect in order to face the completion process as simple as fast as possible. Similar questionnaire is targeted to professors. Figure 1. Student evaluation questionnaire Development of the questionnaire was done using an online tool that allows customization according to needs, and also provides quick and easy access for completion. The questionnaire will be presented to students during courses and it will likely be completed either at the end of practical classes or from home. Therefore the online version offers the most freedom and is likely to result in the highest number of finished questionnaires. Also with this tool we can pre-determine the format of responses, indicating the data type, size and desired restrictions to be applied to each response. The data obtained can be exported according to needs in multiple formats (including CSV, XLS and PDF) and can then be easily imported and processed in order to apply data mining algorithms. IV. IMPLEMENTATION OF DATA MINING ALGORITHMS After collecting and processing data, we load all records into the database. Thus the database table STUDENT_APPLICATION is form by the attributes described in the previous table. The attributes are used in the Data Mining algorithms, first to determine the most important attributes (we ll use the Attribute Importance algorithm), second to determine the probability of the application 287

5 being approved and thus the student being accepted in the program (using classification algorithms), and also to determine the student s profile by clustering the students collectivity. We ll use the facilities and the data mining algorithms implemented in the Oracle Data Miner tool. The interface provides options for data processing, model implementation, results testing and evaluation on new data sets. First, we have applied the algorithm in order to establish the significant attributes for our collectivity. The algorithm "Attribute Importance" will take into account all the attributes from the questionnaire, including the evaluation score obtained from the previous exams and tests. The results shows that the most important attributes for determine the probability of student being accepted in the research program are: the intention to continue doctoral and master studies and also his/her evaluation obtained by the scoring algorithm based on the previous exams and tests. So, we ll apply the classification algorithm in two ways: with all attributes and with the most significant attributes determined before. Figure 2 The Naïve Bayse accuracy in the case of the significant attributes We do not register a significant difference between the two implementations, the accuracy in the first case (all attributes) is about 93% and the accuracy in the second case (only the significant attributes) is about 95% (figure 3). In conclusion we ll take into account all attributes for determining the student profile. For clustering algorithm we used k-means implementation and we selected 5 clusters. The students are clustered in 5 groups in 2 major nodes: the first node in divided in other 2 sub-nodes, one with value 0 for the acceptance and one with value 1 for the acceptance; the second node contain only values 0 for the acceptance. The rule for the cluster with value 1 for the acceptance is: IF YEAR in (III) and CS_DOCTORAL in (3.0) and CS_MASTER in (3.0) and DOMAIN in (Research and Development, Consulting, Programming, Management, Testing Services, Analysis, Design and Economics) and EVAL_ACTIVITY in (2.0,3.0,4.0) and EVAL_COMMUNICATION in (1.0,2.0,3.0,4.0) and EVAL_SCORE in (5.0) and EVAL_TECHNIQUE in (3.0,4.0,5.0) and PROBABILITY in (0.0,1.0) and PROFILE in (Research and Development, Consulting, Technical) and SEX in (f, m) and AGE in (20.0,21.0,22.0,23.0,24.0) THEN Cluster equal 9; The confidence for this cluster is 85.47%. In conclusion we can obtain a cluster with students that can be accepted for extracurricular activities such as research and development, scientific clubs and doctoral studies. V. Conclusions This paper has presented the application of the data mining techniques in higher education, to determine the profile of the students, in order to enroll them in extracurricular activities such as international competitions, internships, scientific research and development. 288

6 The obtained data should be used to raise awareness on the possibilities and need to use the data mining techniques and dedicated informatics solutions. In the paper, we have compared different student segments by performing cluster analysis and classification. The identified profiles are used in two ways: first, to investigate which are the effects of the extracurricular activities on education; and second to distribute students in different competitions and research projects, as well as to revise educational strategies of the faculty related to the curriculum. The future research will be directed towards the design of a decision support system for analyzing and visually present the students profiles, their involvement in extra-activities and the implications on their school situation. Acknowledgements This paper presents some results of the research project PN II, TE Program, Code 332: Informatics Solutions for decision making support in the uncertain and unpredictable environments in order to integrate them within a Grid network, financed within the framework of People research program. References [1] Tudor, A.; Bâra, A.; Botha, I. (2011). Solutions for analyzing CRM systems - data mining algorithms, International Journal of Computers, no. 2, vol. 5, pp , ISSN: [2] Tudor, A.; Bâra, A.; Botha, I. (2011). Data Mining Algorithms and Techniques Research in CRM Systems. In Recent Researches in Computational Techniques, Non-Linear Systems and Control, pp , ISBN: [3] Romero, C.;Ventura, S. (2007). Educational Data Mining: A Survey from 1995 to 2005, Expert Systems with Applications, vol. 33, no. 1, pp , ISSN: [4] Minaei-Bidgoli, B.; Kortemeyer, G.; Punch, W.F. (2004). Enhancing Online Learning Performance: An Application of Data Mining Methods, In Proceeding of Computers and Advanced Technology in Education, pp , ISBN: [5] Kwok, J.; Wang, H.; Liao, S.; Yuen, J.; Leung, F. (2000). Student profiling system for an agent-based educational system, In Proceedings of the annual Americas' conference on information systems, paper 355. [6] Blocher, J. M.; Sujo de Montes, L.; Willis, E. M.; Tucker, G. (2002). Online Learning: Examining the successful student profile. The Journal of Interactive Online Learning, vol. 1, no. 2, ISSN: [7] Superby, J. F.; Vandamme, J. P.; Meskens, N. (2006). Determination of factors influencing the achievement of the first-year university students using data mining methods. In Proceedings of the 8th international conference on intelligent tutoring systems, Educational Data Mining Workshop, (ITS`06), pp [8] Tinto, V. (2000). Taking student retention seriously: rethinking the first year of college, NACADA Journal, vol. 19, no. 2, pp [9] Herzog, S. (2006). Estimating student retention and degree-completion time: Decision trees and neural networks vis-a-vis regression, New Directions for Institutional Research, pp.17-33, ISSN: X [10] Luan, J. (2002). Data mining and knowledge management in higher education potential applications. In Proceedings of Association for Institutional Research (AIR) Forum. [11] Vercellis, C. (2009). Business Intelligence: Data Mining and Optimization for Decision Making, John Wiley & Sons. [12] Chiu, S.; Tavella, D. (2009). Data Mining and Market Intelligence for Optimal Marketing Returns, Elsevier [13] Kantardzic, M. (2003). Data Mining: Concepts, Model, Methods, and Algorithms, Wiley-IEEE Press [14] Refaat, M. (2007). Data Preparation for Data Mining Using SAS, Elsevier, p. 25. [15] Friedman, N.; Geiger, D.; Goldszmidt, M. (1997). Bayesian Network Classifiers, Machine Learning, Volume 29 (2-3), p [16] Witten, I. H.; Frank, E.; Hall, M. A. (2011). Data Mining: Practical Machine Learning Tools and Techniques (Third Edition). Morgan Kaufman. p. 99. [17] Hoffmann, A.; Mahidadia, A. (2010). Machine Learning. Scientific Data Mining and Knowledge Discovery: Principles and Foundation. Edition 1 by Mohamed Medhat Gaber. Springer-Verlag, p