Fuzzy Multicriteria nalysis for Student Project Evaluation. Pejić *, P. M. Stanić **, Sz. Pletl **,. Kiss *** * Óbuda University, udapest, Hungary ** SuboticaTech/epartment of Informatics, Subotica, Serbia *** Technical Faculty Mihajlo Pupin, Zrenjanin, Serbia pejic.aleksandar@gmail.com, pmolcer@yahoo.com, pszilvi@vts.su.ac.rs, kis@tippnet.rs bstract This article contains a description of fuzzy evaluation method for student project works. Fuzzy technique for order preference by similarity to ideal is proposed for ranking the projects. Ranking is based on linguistic attributes of five criteria with three different weights. The scores show how far is the evaluated criterion from the ideal case for the project in question. The overall score depends on the results of all evaluated projects, while the ranking order remains the same. Numerical examples with triangular fuzzy sets for three cases of are given. I. INTROUCTION The proper system for evaluating the learning achievement of students is the key to realizing the purpose of education [1]. Evaluation system of the students in higher education should be regularly reviewed and improved and should be fair and beneficial to all students. Whenever a subjective evaluation is there, it may lead to difference of opinion. Fuzziness arises by virtue of difference in opinion [2]. fuzzy clustering method is used in some applications for classification of objects, for example in [3] fuzzy based expert system for tourist destination classification is described. Recently, fuzzy set theory is used to solve the problem of vagueness in decision making when evaluating human performance, especially students in higher education. The generic performance assessment scoring system - PSS model of assessment -in higher education is described in [4] and a way of including multiple productivity factors and multiple assignments into the model by means of fuzzy logic operations is shown. n implementation of predictive fuzzy systems for helping to capture the students needing extra assistance is reported in. In new methods are presented for evaluating students answerscripts using fuzzy numbers associated with degrees of confidence, where the satisfaction levels given by the evaluator awarded to the questions of the students answerscripts are represented by triangular fuzzy numbers associated with degrees of confidence. In [1] the proposed method considers the importance, the complexity, and the difficulty of the questions given to students, as factors of evaluation. The system has been represented as a block diagram of three fuzzy logic controllers. Each fuzzy logic controller generates an output from two inputs using Mamdani s max min inference mechanism and the center of gravity (COG) defuzzification. The inputs of the first FLC are accuracy rate matrix and time rate matrix, the inputs of the second FLC are the output of the first FLC and complexity matrix, and finally, the inputs of the third FLC are the output of the second FLC and importance matrix. However, in there is underlined that the difficulty factor is a very subjective parameter and may cause an argument about fairness in the evaluation. n improvement of the three node system described in [1] is proposed in, where Gaussian membership functions are used in the fuzzy system instead of triangular membership functions. The varying of the parameters of the triangular MF s resulted in different scores and different ranking orders while the same scores and the same ranking orders were obtained for Gaussian MF s of various widths. In [2] three methods of student evaluation are combined: assessment of answersheet done by traditional or classical method, assessment of students answerscript using satisfactions levels of examinar with the degree of confidence, and method of finding the type of the examinar. The outputs of the methods are combined in a three-node fuzzy controller and a final result adjuster, which provides scores and ranks of students. In fuzzy logic is used for evaluation of student performance in laboratory applications, where the teacher responsible for the laboratory application can edit the ranges of membership functions and rules, permitting nonhomogenous but flexible and objective performance evaluation. In, development of an academic staff performance evaluation system based on fuzzy rules is given. In the method described in this paper, project works of students are evaluated by fuzzy multicriteria analysis. The ratings of various alternatives versus various subjective criteria and the weights of all criteria are assessed in linguistic variables represented by fuzzy numbers. Section II. presents the methodology of student project evaluation based on a fuzzy extension of the technique for order preference by similarity to ideal solution (TOPSIS). In this method, the ratings of various alternatives versus various subjective criteria and the weights of all criteria are assessed in linguistic variables represented by fuzzy numbers. Some numerical examples of the assessment based on the described methodology in section II is presented in Section III, as well as the analysis of the result. Finally, we conclude this paper in Section IV. II. METHOOLOGY While it is relatively easy to store information about students' progress technically and to classify assignments as "easy", "intermediate", or "difficult" by hand, it is surprisingly difficult to automate the process of classifying students with respect to these semantic labels in terms of crisp computing. ssuming a good teacher with only a few students, this problem is largely irrelevant. However, when the number of students
increases or they are out of reach, seeking computer supported means becomes interesting. Evaluation of student performance can be made based on Criterion- Referenced Evaluation (CRE) and Norm-Referenced Evaluation (NRE). In CRE, evaluation is carried out with respect to established criteria of performance. One of the drawbacks of CRE is the lack of its ability in reflecting the that has been used to support the evaluation, unable to show what criteria the 'final result' or 'score' refers to. Instead of using CRE, evaluation may also be made on the basis of NRE, a method of assessment based on comparison and utilizing information gathered from previous student performance data [11]. In TOPSIS method the ratings of various alternatives versus various subjective criteria and the weights of all criteria are assessed in linguistic variables represented by fuzzy numbers. Fuzzy numbers try to resolve the ambiguity of concepts that are associated with human being s judgments. The aim of the described method is to help the grader to express the vagueness in his or her opinion. Used fuzzy multicriteria analysis model is based on the MF-SS (Multiple-ttribute Fuzzy ecision Support System) developed by [12]. The main entities to be considered in the multicriteria analysis are alternatives, criteria, attributes and weights. Weights express the relative importance, attached by the teacher (evaluator), for each criterion. Each criteria is given its own impact parameter. The assessed criteria of a particular assignment may differ with respect to the learning outcome representing the goal of the course. In [13] ranking of m-learning materials is accomplished by fuzzy multicriteria analysis. Criteria are described by linguistic attributes and linguistic weights. The uncertainty of subjective perception in the situation of evaluating learning materials is incorporated by fuzzy sets. The block diagram of the algorithm for the computation of project evaluation is shown in Fig. 1. y the fuzzy TOPSIS method students get a score about each criterion of the project and also a score considering each criterion for the learning material. This score will show how far is the evaluated criterion from the ideal case for the student in question. Projects are assessed by means of five criteria, presentation of the completed work, functionality of the prepared model or tool, documentation of the entire project, compliance of the deadline which is named timing, and of the theoretical background of the prepared project. Scores are given as six linguistic variables, and are represented by six triangular fuzzy membership functions (Fig.2). The fuzzy sets are open ended. The attributes of every variable are: very poor, poor, average, good and excellent. Each attribute of the variables can be presented as triangular fuzzy number. triangular fuzzy number is defined as:,, (1) where a is the value for which the membership function has the value exactly 1: 1 (2) m is the left spread and n is the right spread. Method for preparing the fuzzy decision matrix is proposed by [14]. Figure 2. Fuzzy triangular membership functions of the criteria Let the criteria for a specific alternative be denoted by and the performance measure of each criterion by the triangular fuzzy number,,. The coresponding weight is denoted by the triangular fuzzy number,,. For each fuzzy number the lower and upper points α- cuts are calculated for α=1 denoted by and. This way the fuzzy numbers are represented by intervals. Intervals are calculated for each criterion by normalizing. Normalization is taking into account every project: (5) Figure 1. Project evaluation algorithm (6)
(7) m is the total number of the projects and n is the total number of criteria. i=1,...m and j=1,...,n. The normalized fuzzy interval has to be transformed into fuzzy number:,, (8) The result of the multiplication is a weighted matrix of fuzzy numbers with elements equal to: (12) The ideal case expressed by fuzzy number is for every criterion the fuzzy positive ideal solution: 1,0,0 (13) The sums of the distances of every criterion from the ideal positive solution are calculated for every evaluated project. (9), (14) (10) Weights represent the impact parameter of the criteria. Weights are given as three triangular fuzzy membership functions such that the sum of the modal values of the fuzzy triangular numbers which represent the criterion weights is equal to one (Fig. 3). The linguistic values of the weights are: little important, moderate important and very important. The distance between the two fuzzy numbers and is defined as:, 1 3 Figure 3. Fuzzy triangular membership functions of the weights Weights are linguistic variables expressed by fuzzy numbers to be convenient for multiplication. The product of the fuzzy numbers,, and,, is defined in this case as:,, (11) III. NUMERICL EXMPLE Numerical examples are presented in the following tables. Examples for three cases are given. In every case eight student projects noted as,, C,, 1, 1, C1,... are evaluated. Marks, as well as weights are represented by alternatives in form of linguistic variables. Tables I.- III. correspond to Case 1, Case 2 and Case 3 respectively. The final, summed distances from the ideal solution for every criteria of the projects are presented in the last column of the tables. The cases differ from each other in their composition of the project quality. Students (projects),, C, and are present in every case and make the primary group in the three cases described. In the first case it was supposed that two of the other four students are same as the worst of the four which are present in every case, and two of them are the same as the student ranked as third in the primary group. In the second case two of the four additional students were taken the same as the best from the primary group and two of them the same as the student ranked as the second best. In the third case the four additional students were taken the same as the worst in the primary group. TLE I. SCORES OF PROJECTS CSE 1 Case 1 Presentation Functionality ocumentation Timing istance 4.0691
C 1 1 2 2 3.8918 The marks of the evaluated projects are represented as normalized fuzzy sets described by fuzzy numbers, as it can be seen in Table I-III. The distance represent the measure of the distance of the evaluated project from the ideal solution, when all of the attributes are marked as equal to ideal (13). Weights were taken as follows: presentation is moderately important, functionality is very important, documentation is moderately important, timing is little important and theoretical of the student is moderately important. TLE II. SCORES OF PROJECTS CSE 2 Case 2 Presentation Functionality ocumentation Timing C 1 C1 2 C2 istance 4.5810 4.7428 TLE III. SCORES OF PROJECTS CSE 3 Case 3 Presentation Functionality ocumentation Timing C 1 2 istance 4.4097 4.0219 3.8338
3 4 The rank of the projects remained the same in every case. s it can be seen, the distance values differ for the same project depending on the performances of the other students. Information gathered from the data of previous student performance evaluations, possibly through several years, can also be taken into account with this method. IV. CONCLUSION The assessment method described in this paper does not increase the time needed for the assessment compared to the traditional evaluation techniques as it is implemented in software. The overall measure of distance got by the proposed method can be transformed into a numeric, or alphabetic grade according to the institution-specific scoring methods and rules. REFERENCES [1] I. Saleh and S. Kim, " fuzzy system for evaluating students learning achievement", Expert Systems with pplications 36, pp. 6236-6243, 2009. [2] S. Ingoley and J. W. akal "Use of Fuzzy Logic Evaluating Students Learning chievement", International Journal on dvanced Computer Engineering and Communication Technology, vol. 1, no. 2, 2012. [3]. Pejić, Sz. Pletl,. Pejić M. "n Expert System for Tourists Using Google Maps PI", IEEE International Symposium on Intelligent Systems and Informatics, Subotica, Serbia, 2009. [4] P. H. Vossen, " Truly Generic Performance ssessment Scoring System (PSS)", Proc. of the International Technology, Education and evelopment Conference, Valencia, Spain, 2011. O. Nykänen, "Inducing Fuzzy Models for Student Classification", Educational Technology & Society, vol. 9, no. 2, pp. 223-234, 2006. H.-Y. Wang and S.-M. Chen, "New Methods for Evaluating Students nswerscripts Using Fuzzy Numbers ssociated with egrees of Confidence", IEEE International Conference on Fuzzy Systems, Vancouver, Canada, pp. 1004-1009, 2006. S.-M.ai and S.-M. Chen, "Evaluating students learning achievement using fuzzy membership functions and fizzy rules", Expert Systems with pplications 34, pp. 399-410, 2008. I.. Hameed and C. G. Sorensen, "Fuzzy Systems in Education: More Reliable System for Student Evaluation", Fuzzy Systems, by. T. zar, 2010 G. Gokmen et all. "Evaluation of student performance in laboratory application using fuzzy logic", Procedia Social and ehavioral Sciences vol. 2, pp. 902-909, 2010. J. Stoklasa, J. Talašová, P. Holeček, "cademic Staff Performance Evaluation Variants of Models", cta Polytechnica Hungarica, vol. 8, no. 3, 2011. [11] K. Rasmani and Q Shen, "ata-driven fuzzy rule generation and its application for student academic performance evaluation", pplied Intelligence, vol. 25, no. 3, pp. 305-319, 2006 [12] R. Ribeiro,. Moreira, and E. eclercq, " fuzzy evaluation model: a case for intermodal terminals in Europe", X. Yu, Xinghuo and J. Kacprzyk, C. Carlsson (eds.). pplied ecision Support with Soft Computing. Studies in Fuzziness and Soft Computing, 124. Springer, pp. 218-233, 2003. [13] P. M. Stanić, Z. Čović and. Kiss, "Fuzzy Multicriteria nalysis of M-learning System", IEEE International Symposium on Intelligent Systems and Informatics, Subotica, Serbia, 2012. [14] G. R. Jahanshahloo, F. Hosseinzadeh Lotfi, and M. Izadikhah, "Extension of the TOPSIS method for decision-making problems with fuzzy data", pplied Mathematics and Computation, vol. 181, pp. 1544-1551, 2006.