Bug triage in open source systems: a review

Int. J. Collaborative Enterprise, Vol. 4, No. 4, 2014 299 Bug triage in open source systems: a review V. Akila* and G. Zayaraz Department of Computer Science and Engineering, Pondicherry Engineering College, Pondicherry 605014, India Email: akila@pec.edu Email: gzayaraz@pec.edu *Corresponding author V. Govindasamy Department of Information Technology, Pondicherry Engineering College, Pondicherry, India Email: vgopu@pec.edu Abstract: Open source software development is a collaborative effort among several software developers dispersed across multiple geographical locations. The interaction among the software developers is asynchronous in nature. The participation of software developers in the software development is entirely voluntary. The set of software developers involved in the project is volatile. The expertise of the software developer may vary with time. Given this scenario, the process of bug triaging is all the more complex in open source software development. On an average, it takes 40 days to assign a bug to the correct software developer. It takes another 100 days to reassign to a new software developer if the first software developer is unable to fix the issue. This necessitates for a semi-automated support for bug triaging. To this end, this paper highlights the state-of-art techniques employed in bug triaging in open source environment as well as in enterprise environment. Keywords: bug triaging; open source software development; issue management system. Reference to this paper should be made as follows: Akila, V., Zayaraz, G. and Govindasamy, V. (2014) Bug triage in open source systems: a review, Int. J. Collaborative Enterprise, Vol. 4, No. 4, pp.299 319. Biographical notes: V. Akila is an Assistant Professor of the Department of Computer Science and Engineering at Pondicherry Engineering College, Pondicherry, India. Her areas of interest in teaching and research are mainly focused on bug triage in open source systems, particle swarm optimisation and social networks. G. Zayaraz is a Professor of the Department of Computer Science and Engineering at Pondicherry Engineering College, Pondicherry, India. He received his PhD in quantitative methods for software architecture evaluation from Pondicherry University, India. He is actively working in the areas of software architecture evaluation, network security and software testing. Copyright 2014 Inderscience Enterprises Ltd.

300 V. Akila et al. V. Govindasamy is an Assistant Professor of the Department of Information Technology at Pondicherry Engineering College, Pondicherry, India. He received his PhD in efficient and scalable uncertain event processing in business intelligence from Pondicherry University, India. His areas of interest are complex event processing, fuzzy logic and business intelligence. 1 Introduction Software maintenance expenditure is about 50% of the overall expenditure of the software project (Bhattacharya et al., 2012). In open source software development, the expenditure translates to time. Bug resolution is an important part of the software maintenance activity. Bug management comprises the following three activities: bug triaging bug assignment to the software developer for solution solving of the bug. Bug tracking tools are available to keep track of the bugs that arise in an open source software. Bugs that arise after the deployment of software are reported by the users as bug reports. The bug reports are semi-structured and contain a brief textual summary of the reported bug, fields such as severity and assignee, etc. The bug tracking tools are used to store and manage the reported bugs. Bug triaging comprises checking for validity of the bug, assigning priority, severity and assigning the bug to a correct software developer. Manual bug triaging is time consuming and fault prone (Bhattacharya et al., 2012; Anvik and Murphy, 2011; Jeong et al., 2009). The conceptual view of the bug triaging process is given in Figure 1. Once the bug report is submitted, the triager has to evaluate, prioritise, categorise and assign the bug to a software developer. The bugs may be categorised according to the content in the summary or by the component to which the given bugs belong to. The bug is assigned to a software developer based on the expertise of the software developer. The bug assigned to a software developer may get reassigned if the concerned software developer is not able to solve the bug. This process of reassignment is termed as bug tossing. After a few bug tosses, the bug may get resolved. The bug tossing relation among the software developers can be modelled as a bug tossing graph. 1.1 Bug tossing graph In the bug tossing graph, the nodes represent the software developers and the arcs represent the tossing activity. The goal of the triager is to find a software developer in a manner such that the number of tosses a bug undergoes before being resolved is minimised. Each toss takes up some lead time for the software developer to familiarise himself with the bug, thereby contributing to the bug fix time. It has been observed that the current techniques for bug triaging in open source systems are borrowed from ticket resolution techniques of the enterprise systems (Jeong et al., 2009). Bugs are called as

Bug triage in open source systems: a review 301 tickets in the enterprise jargon. The ticket transfer relations between software developers are captured as Markov process. The important difference is that the actual path models are used to capture the ticket transfer relations in the enterprise systems and the goaloriented path models are used in open source systems. Table 1 shows five sample tossing relations. For instance, the tossing path b->d->e indicates that the bug was first assigned to b then tossed to d and finally tossed to e who resolves the bug. Figure 1 Conceptual view of bug triage Table 1 Tossing paths S. No. Tossing path 1 a->b->c->d->e 2 b->d->e 3 g->a->d 4 b->d->e 5 h->c->d

302 V. Akila et al. 1.1.1 Goal-oriented path model For the given tossing paths from Table 1, the individual paths broken down for the goaloriented path model are given in Table 2. The paths from every software developer to e or d who are the final resolvers are derived. The corresponding goal-oriented path model-based tossing graph is shown in Figure 2. This model captures the relation of every software developer involved in the tossing to that of the final resolvers. The final resolvers are d and e. This model ignores the intermediate tosses between the software developers. Furthermore, the goal-oriented path model does not capture the existing software developer structure. Table 2 Individual paths for goal-oriented path model S. No. Tossing path Individual paths 1 a->b->c->d->e a->e,b->e,c->e,d->e 2 b->d->e b->e,d->e 3 g->a->d g->d,a->d 4 b->d->e b->e,d->e 5 h->c->d h->d,c->d Figure 2 Bug toss graph based on goal-oriented path model 1.1.2 Actual path model The individual paths derived for actual path model for the tossing paths given in Table 1 are given in Table 3. The corresponding actual path model-based tossing graph is illustrated in Figure 3. This model captures the underlying software developer structure in the tossing relations. The next section discusses in detail about the literature survey done in bug triaging techniques and ticket resolution techniques.

Bug triage in open source systems: a review 303 Table 3 Individual paths for actual path model S. No. Tossing path Individual paths 1 a->b->c->d->e a->b,b->c,c->d,d->e 2 b->d->e b->d,d->e 3 g->a->d g->a,a->d 4 b->d->e b->d,d->e 5 h->c->d h->c,c->d Figure 3 Bug toss graph based on actual path model 2 Related work The literature has been analysed along the following two dimensions: bug triaging techniques in open source systems ticket resolution techniques in enterprises The survey has examined the recent papers in bug triaging in open source systems and ticket resolution in enterprise systems. 2.1 Bug triaging techniques in open source systems The existing bug triaging techniques have been analysed along these dimensions: data used algorithm employed tossing graph model evaluation parameters. Figure 4 gives the classification of bug triaging techniques derived from the literature survey. The bug triaging techniques are basically based on the textual content of the summary field or on the tossing relations that exist among the software developers or as a

304 V. Akila et al. combination of the two called as hybrid. The content-based techniques are either activity based or location based. Activity-based techniques consider the past activity of the software developer is taken to account. Location-based techniques emphasise on locating the source code file that needs to modify for an incoming bug. This source code file is then used for identifying the software developers who may be assigned to the new bug. The contents are analysed using machine learning techniques or information retrieval techniques. The link-based techniques are based on goal-oriented path model or actual path model. The goal-oriented path model considers only the bug tossing relations that exist among the software developers and the final resolvers. The actual path model considers the tossing relations as well as the comments posted by the software developers. The techniques are compared based on the underlying graph model and the evaluation parameters used. Figure 4 Classification of bug triaging techniques in open source systems (see online version for colours) 2.1.1 Content-based technique Activity based In activity-based methods for bug triaging, similar bug reports for the new incoming report are identified. From similar bug reports, the potential software developers are inferred. Similar bug reports are identified by machine learning, soft computing or information retrieval methods. Machine learning Reducing efforts in making recommendations is done in a three-pronged manner (Anvik and Murphy, 2011). There are three kinds of recommenders. They are software developer-oriented recommender component recommender interest recommender.

Bug triage in open source systems: a review 305 The algorithms explored are Naïve Bayes, Support Vector Machine (SVM), C4.5, Expectation Maximisation, Conjunctive rules and Nearest Neighbour. The parameters that were used for evaluation are precision, recall and F-measure. SVM and Naïve Bayes performed better with regard to the parameter precision to make one recommendation, but when more than two recommendations had to be done, then SVM performed better. The bug reports are pre-processed and only the summary and description fields are selected. This free-form text is then converted to feature vector. The resulting data are then labelled according to the software developer. Summarising the data in the report using path group and determining the activity levels of the software developers are the two techniques implemented. The accuracy percentage of the recommender is 75%. For any content-based method, the natural language content of the bug report is converted to a bag of words. Term selection is an important aspect of data preparation in bag of words method. Selection of terms affects classification accuracy (Aljarah et al., 2011). The three-term selection methods that are examined are based on Logs Odd Ratio score. These methods are compared against Information Gain Methods and Latent Semantic Analysis. The Bayesian network classifier is used for triaging of the bugs in the bug repository. The results are compared based on precision and recall. Logs Odd Ratio provided 30% improvement in precision and 5% improvement in recall. Data preparation can also be viewed as a dimension reduction problem (Ahsan et al., 2009). The report title and report summary from the bug report are extracted. The extracted text data are then converted to term document matrix using parsing, filtering and term weighting. The dimension of the term document matrix is reduced using feature selection and latent semantic indexing. The reduced term frequency matrix is used along with seven different classifiers. It has been observed that SVM with latent semantic indexing produces the best results. The accuracy, precision and recall values of SVM and latent semantic indexing are 44.4%, 30% and 28%, respectively. A theoretical framework-based SVM to recommend software developers also takes into account preference elicitation and load balancing for triaging of bugs (Baysal et al., 2009). A Software Developer Recommendation with k-nearest-neighbour search and expertise ranking (DREX) (Wu et al., 2011) takes social network metrics to rank the software developers. This system finds a set of bugs that are similar to a new incoming bug using K-Nearest-Neighbour search. Once this set is determined, the software developers are ranked according to the frequency of bug resolution in that set as well as six social network metrics such as in degree, out degree, page rank, betweenness and closeness. The social network is constructed based on the comments made by the software developers. The system is evaluated based on the parameters: precision and recall. The bug triaging task may be visualised as a text classification problem (Cubranic and Murphy, 2004). Specifically, it can be treated as a multi-class, single label classification problem where the software developer corresponds to a class and each bug report is assigned to a class using Naïve Bayes classification. The system achieves 30% accuracy. DevRec (Xiaz et al., 2013) is a composite method that performs triage based on bug reports-based analysis and software developer-based analysis. The bug reports-based analysis finds the bug reports that are similar to an incoming bug report based on k-nearest neighbours. The second part consists of determining a software developer s affinity to bug reports in terms of the following parameters: topic, components and product. The recall@5 and recall@10 values are computed for performance evaluation.

306 V. Akila et al. Filtering of duplicate bug reports before triaging is essential to improve the efficiency of the triaging process (Tan et al., 2010). A keyword extraction algorithm is used to classify the remaining bug reports. As a final step, the candidate software developers are chosen based on history, preference, experience, credit, efficiency and workload. Apart from the textual description of the bug, seven different items that can be used while triaging a bug is identified (Anvik et al., 2006). The seven items are component, operating system that the bug occurs, hardware the bug belongs to, software version, software developer of the associated code, the work load of the software developers and the set of software developers currently contributing to the project. The SVM algorithm which used bug description, product component and the list of active software developers gave a result of 64% precision for Firefox and 86% precision for Eclipse project. The results of an exploratory analysis with machine learning algorithms concluded that SVM gives the best results (Anvik et al., 2006). The SVM is used to make recommendations for a set of software developers qualified to resolve the bug. Data preparation considering the efficient feature selection and instance selection is used to reduce the training set for bug triage (Zou et al., 2011). The feature selection is done based on chi-squared test. The instance selection is based on iterative case filter algorithm. The system is validated using the parameters accuracy, precision and recall. Triaging approach using bug reports metadata system uses the discriminating terms in the bug report including reporter of the bug and the component to triage a bug (Banitaan and Alenezi, 2013). The chi-square method is used for feature selection during data preparation. The Naïve Bayes classifier is used to perform bug triaging. The system is evaluated with the parameters: precision recall F-measure. Soft computing The Bugzie system is based on fuzzy set approach (Tamrawi et al., 2011). The set of technical terms is collected from the software artefacts. A fuzzy set C t is used to represent a set of software developers to fix a bug with term t. The system has three stages. They are training recommending updating. The system Bugzie outperforms SVM, Naïve Bayes, C4.5 and Bayesian Network in terms of the parameters: prediction accuracy time efficiency.

Bug triage in open source systems: a review 307 Information retrieval Over specialisation associated with recommender systems is handled by an approach named as content-boosted collaborative filtering (Park et al., 2011). The content-boosted collaborative filtering is the combination of content-based recommendation and collaborative-based recommendation. The recommendation problem is reformulated as an optimisation problem of accuracy and cost. The software developer profiles are created by cost-aware triage algorithm based on latent Dirichet analysis. A cost profile for each software developer is prepared based on the time taken by the software developer to solve the bug. The performance evaluation was a trade-off between cost in terms of bug fix time and accuracy. The bug fix time efficiency was improved by 30%. Knowledge management in terms of activity profile for the software developers present in the bug tracking system can enhance the bug triage process (Naguib et al., 2013). The activity profile consists of the user role (review, assign and resolve) and user s topic association. For an incoming bug, the topic association is derived based on latent discriminant analysis using the title, description and system component. Many software developers who match the new bug s topic are then ranked. The ranking is based on the software developers experience and the number of topics the software developers has experience in. The new system produces an average hit ratio of 88%. A prototype for bug triage which uses the resolved bug reports as well as the change sets has been developed (Kevic et al., 2013). Information retrieval method is used to find the bug reports that are similar to an incoming bug report based on cosine similarity. From the retrieved similar bug reports, the software developers who provided the change set are retrieved. A score that takes into account the number of change sets provided by the software developer and the similarity score of the bug report is calculated for each software developer. The software developers are ranked using this score. Furthermore, to provide the context for the assigned software developer, the files that were examined while a similar bug was resolved are provided. To foster collaboration, the prototype has been developed as a multi-touch table. Expertise and interest model of software developers based on vocabulary is used for bug assignment (Moin and Neumann, 2012). The recommender creates an n-grams of strings from a new bug. The n-gram s similarity with the expertise model is calculated using cosine, Dice, Jaccard and Overlap coefficients. The system gives a precision value of 90.1% and recalls value of 45.5%. Porch Light is the prototype for a tool for bug triaging (Bortis and Hoek, 2013). Software developers deal with grouped bug reports effectively. The Porch Light provides a tag-based grouping of bugs. The grouping may be with respect to the component the bug belongs or any dependency between the previously resolved bugs. etc. A specialised Bug Tagging Language is used for creating new tag sets. Software DEveloper Recommendation based on TOpic Models (DETROM) is based on a bipartite graph to make software developer recommendations (Xie et al., 2012). As a first step, the software developers who contributed towards the resolution of the bug from the history are extracted from the activity log associated with each bug. A bipartite graph is built with nodes as software developers and bug reports. Topic modelling is applied over the natural language contents of the resolved bugs and the conditional probability that a bug belongs to a particular topic is derived. The association between the software developer and the bug reports is calculated as a probability. For an incoming bug report, the topic the bug belongs to is determined. The probability of a software

308 V. Akila et al. developer towards resolving a bug is calculated. The software developer with the highest probability is chosen for resolving the bug. The DETROM system is validated using precision, recall and F1 measure. Time-aware bug assignment is presented in Fix Time (Nguyen et al., 2014). The method is based on topic modelling. A log-normal regression model is a used profile bug resolution time across topics. The system is validated using prediction accuracy. Location based Location-based methods scrutinise the source code file that may be altered to handle the new bug. The probable software developers for resolving the bug are identified from the natural language content of the source code file or by identifying the software developers who contributed to the source code file. Location-based bug assignments are an alternative to activity-based bug assignment (Shokripour et al., 2013). The source code files that need to be modified for the bug are identified in the first phase. The software developers associated with the source code files are identified in the second phase. Parts of speech method is used for nouns extraction from four different sources, namely identifiers such as name of classes methods commit messages the comments present in the source code previously fixed bugs for the source code files that the new bug has identified itself with. The evaluation parameter used is accuracy. The proposed system has achieved an accuracy of 89.41% and 59.76% for eclipse and Mozilla projects. Change request handling that comprises bug reports as well as feature requests is presented (Kagdi et al., 2011). In the first step, a corpus for the software system is created. Identifiers and comments in the source code are utilised to create the corpus. The indexing of the corpus is done using latent semantic indexing. For a new change request, a query is formed. The query is a set of terms formed from the change request or bug report. The similarity between the query and the source code files in the corpus is determined and the ranking is performed. The software developers who contributed to the source code files are extracted according to the commits submitted by the software developer in the version control repository. The software developer s Euclidian distance from the source code file is used to rank the software developers. The system produces an accuracy in the range of accuracies of 47% and 96% for bug reports, between 43% and 60% for feature requests. Source code contributions also form a basis for bug triage (Matter et al., 2009). The vocabulary found in the diffs of a software developer is lexically compared with an incoming bug for making recommendations. The expertise of each author is modelled as a term author matrix. The activity decay of a software developer and vocabulary decay are modelled. The proposed system achieved 33.6% precision and 71% recall. The summary of the survey in bug triaging based on content in open source system is presented in Table 4.

Bug triage in open source systems: a review 309 Table 4 Summary of content based bug triaging techniques in open source systems Machine learning Information retrieval Accuracy Classification Paper Activity Anvik and Murphy (2011) based Aljarah et al. (2011) Ahsan et al. (2009) Wu et al. (2011) Cubranic and Murphy (2004) Xiaz et al. (2013) Anvik (2006) Anvik et al. (2006) Zou et al. (2011) Zou et al. (2011) Tamrawi et al. (2011) Matter et al. (2009) Park et al. (2011) Naguib et al. (2013) Xie et al. (2012) Nguyen et al. (2014) Moin and Neumann (2012) Location based Precision Recall F-measure Accuracy Precision Recall F-measure Fix time Average hit ratio Shokripour et al. (2013) Kagdi et al. (2011) 2.1.2 Link based Actual path model Only the link data are used here. The importance of non-committers known as catalysts is acknowledged (Mani et al., 2013). The catalysts may not actually solve the bug, but may significantly contribute in terms of commenting or committing a bug. The commenting activities and the tossing of bugs are captured in a directed tossing graph based on the actual path model. A minimal essential graph algorithm has been designed to identify the essential catalyst in solving a bug. The system is evaluated by reducing the number of nodes to solve a bug. 2.1.3 Hybrid techniques Goal-oriented path model Bug tossing relationships modelled as bug toss graph based on goal-oriented path model to make software developer recommendations were first employed in open source software systems (Bhattacharya and Neamtiu, 2010). A goal-oriented path model was

310 V. Akila et al. made based on the first order Markov model. Weighted breadth first search algorithm was used to maximise the path reduction. The paper has proved that the usage of weighted breath first search algorithm reduces the tossing length from that of the original path length. The goal-oriented graph also helps to increase the prediction accuracy by finding the best tossing relation. Machine learning techniques, Naïve Bayes and Bayesian networks were used in combination with the bug toss graph to make the recommendations. The method increases the accuracy by 70% and reduces the tossing length by 72%. A bug triaging system which combines machine learning with bug toss graph based on goal-oriented path model is advocated (Bhattacharya et al., 2012). Many machine learning methods were analysed and it was inferred that Naïve Bayes gives the best result. To handle the occurrence of inactive software developers and modification of software developer expertise, they have modelled the bug toss graph as multi-featured graph. Each in the graph is labelled with the bug class that is the product/component and software developer activity count. The multi-featured graph is modelled as a goaloriented graph. A weight-based depth first search algorithm is employed to find the best tossing relationship. The whole system is incorporated in an incremental learning framework so that recent bugs have more impact on the recommendations made by the system. The technique achieved 83.62% prediction accuracy and 86.31% reduction in tossing lengths. The bug tossing graph can also be modelled using a socio-technical approach (Xuan et al., 2012). The model exploits the number of comments made by a software developer in bug resolution. The software developers are prioritised as per the out degree. This information is leveraged with SVM and Naïve Bayes classification results. An incremental learning framework with 10-folds is used to recommend software developers. A new ranking function based on classifier probability, software developer prioritisation score based on product and software developer prioritisation based on component is computed. The software developer prioritisation based on SVM produced the best results. An approach that consists of three aspects: bug tossing graph, textual similarity and sub-graph generation to narrow down the search space is presented (Chen et al., 2010). The textual similarities between the new bug and the existing bugs were calculated using the vector space model. Then, the sub-graph was generated. Over the sub-graph, the weighted breadth first search algorithm was deployed to identify the top software developers. This approach reduced the tossing length by 76.25%. Actual path model The bug repository consists of multiple types of entities. A heterogeneous network was formed from the bug repository (Zhang et al., 2013; Wang et al., 2013). The heterogeneous network consists of four types of entities: software developer bug comments component.

Bug triage in open source systems: a review 311 Furthermore, each software developer can make four types of contributions reporting bugs commenting reopening bugs fixing bugs. The network schema of the objects and their relations in the bug repository are developed. Based on the network schema, the heterogeneous network is evolved. For an incoming bug report, the list of potential software developers is extracted by using SVM and Naïve Bayes classifier. For each software developer, the best collaborating partner is identified by using the collaboration probability metric. The top five software developers along with their partners are listed for resolving the bug. A hybrid bug triage algorithm is one which combines a probability model and experience model to recommend software developers (Zhang and Lee, 2013). For any new bug, the bugs that are similar to it are shortlisted by a unigram model. From the shortlisted bugs, a social network based on probability model is constructed using the comments posted by the software developer. An experience model is extracted again from the shortlisted bugs using the number of fixed bugs and activity factor. The hybrid bug triage algorithm then utilises both models to recommend software developers. The unigram-based method is enhanced by combining it with component information to identify bug reports that are similar to the new bug report (Yang et al., 2014). From the shortlisted bug report, a Multiple software Developer Network (MDN) is constructed. In MDN, the nodes represent the software developers and edges are labelled as vector. The vector consists of two components: the number of commits made by the software developer the number of comments made by the software developer. A software developer ranking score is used to rank the software developers. 2.1.4 Inference From the literature survey, it is inferred that machine learning techniques have been broadly applied on the summary field of the bug report. The widely applied techniques are Naïve Bayes and SVM (Bhattacharya et al., 2012; Anvik and Murphy, 2011; Ahsan et al., 2009; Cubranic and Murphy, 2004; Anvik et al., 2006; Banitaan and Alenezi, 2013; Tamrawi et al., 2011; Xie et al., 2012; Xuanet al., 2012; Zhang et al., 2013; Wang et al., 2013). These two techniques give better results than the other machine learning techniques. The tossing relations that exist among the software developer are captured predominantly as bug toss graph using goal-oriented path model (Bhattacharya et al., 2012; Bhattacharya and Neamtiu, 2010; Xuan et al., 2012; Chen et al., 2010). The weighted breadth first search algorithm is broadly used to search the bug toss graph. The summary of the survey in bug triaging based on hybrid techniques in open source system is presented in Table 5.

312 V. Akila et al. Table 5 Summary of hybrid bug triaging techniques in open source system Classification Paper Hybrid Accuracy Machine learning Precision Content Recall Link Information retrieval Metric Graph model F-measure Accuracy Precision Recall F-measure Mean reciprocal rank No of steps Goal oriented path model Actual path model Bhattacharya et al. (2012) Bhattacharya and Neamtiu (2010) Jeong et al. (2009) Xuan et al. (2012) Chen et al. (2010) Zhang et al. (2013) Wang et al. (2013) Zhang and Lee (2013) Yang et al. (2014) 2.2 Ticket resolution in enterprises The survey pertaining to ticket resolution has been done along these dimensions data used algorithm employed ticket resolution graph model evaluation parameters. Figure 5 gives the classification of ticket resolution techniques in the literature. Ticket resolution is mainly done by content-based, link-based or a combination of content- and link-based methods. The dominant method that is applied on the content is machine learning. In link-based methods, the ticket transfer relations among the software developers are captured using a ticket resolution graph based on actual path model. 2.2.1 Content based Machine learning The trouble miner system uses document clustering methods to obtain related trouble tickets (Medem et al., 2009). The hierarchical clustering algorithm produces a binary tree of un-labelled nodes which capture the relation between trouble tickets. An algorithm based on depth first search labels the unlabelled binary tree. The labelled tree is then converted to a hierarchy H of n-ary tree.

Bug triage in open source systems: a review 313 Figure 5 Classification of ticket resolution in enterprises (see online version for colours) 2.2.2 Link based A statistical model is applied to capture the ticket resolution sequences (Shao et al., 2008). The order of the Markov model is fixed according to the conditional entropy obtained from the ticket data. A variable order multiple active state search algorithm generates the ticket transfer recommendations. The robustness of the algorithm was evaluated by taking three factors: the training set size time variability of the data variety of problem categories. The algorithm performed consistently when the training size was increased as well as across different problem categories. The variable order multiple active search algorithm performance increased when ticket resolutions from recent windows were taken. Expertise awareness is determined in terms of transfer effectiveness (Chen et al., 2010). A novel exclude algorithm has been designed that calculates transfer effectiveness. The exclude algorithm helps to assess the weakest components in the resolution system. It has been inferred that collaborative networks have truncated power law node degree distributions. The routing in collaboration networks are also task driven. A network model which captures the static connectivity is proposed. A stochastic routing algorithm has been developed to simulate human dynamics in collaboration networks (Miao et al., 2012a). 2.2.3 Hybrid techniques The optimised network model is a generative model which uses both content and link information (Miao et al., 2010; Miao et al., 2012b). The model captures the transition probability and the reason why the ticket was transferred between two groups.

314 V. Akila et al. Ranked resolver algorithm, greedy transfer algorithm and holistic routing algorithm have been designed. The ranked resolver algorithm is based the probability a resolver will be able to solve the ticket. The greedy algorithm makes one-step transfer predictions. The holistic routing approach finds the best resolver within k steps. The holistic algorithm is globally optimised. A hybrid model uses the ticket content and the resolution sequences to make ticket routing recommendations. The system analyses the ticket content and groups a set of semantically similar tickets. A cosine similarity-based weight function is used for model generation. The weighted Markov model is created from this set of tickets. The variable order multiple active state search algorithm is used to retrieve the resolution sequences. The algorithm is locally optimised. A unified generative model that portrays a lifecycle of a ticket using both content and routing sequence is presented (Sun et al., 2010). The optimised network model applies maximum likelihood estimation to capture the transfer profiles. The algorithm is globally optimised. 2.2.4 Inference It has been observed from the survey that actual path models have been employed to model the ticket resolution graph (Shao et al., 2008; Chen et al., 2010; Miao et al., 2010, 2012a, 2012b; Sun et al., 2010). The locally optimised ticket routing algorithm is the variable order multiple active state search algorithm (Miao et al., 2010, 2012b). The globally optimised holistic algorithm performs better (Shao et al., 2008; Chen et al., 2010; Sun et al., 2010). The summary of the ticket resolution techniques followed in enterprise systems is highlighted in Table 6. Table 6 Summary of ticket resolution techniques in enterprise systems Content Ticket resolution graph Actual path model Classification Paper Machine learning No. of steps Influence Content based Medem et al. (2009) Link based Shao et al. (2008) Chen et al. (2010) Miao et al. (2012a) Hybrid Miao et al. (2010) Sun et al. (2010) Miao et al. (2012b) 3 Issues identified in survey 3.1 Graph models Bug resolution is about collaboration among software developers to solve a bug. In order to solve a bug, it is necessary to find a set of software developers who can collaborate on a bug. It is also observed that not all reassignments are detrimental in nature. Some reassignments are necessary to determine the root cause of the bug

Bug triage in open source systems: a review 315 (Bird et al., 2011; Guo et al., 2010; Guo et al., 2011). Thus, it becomes essential that the underlying software developer structure has to be preserved in the bug toss graph. From the survey, it has been observed that goal-oriented path models are extensively researched for bug triage in open source systems. The goal-oriented path model may be useful in determining the best toss relationship with a software developer. The goaloriented path models do not capture the software developer structure in the network. Goal-oriented path model loses knowledge pertaining to intermediate tosses. Actual path models are explored to a lesser extent in open source systems (Matter et al., 2009; Zhang et al., 2013; Wang et al., 2013; Zhang and Lee, 2013). The knowledge about the software developer structure is important to retrieve a set of collaborators who can deliberate on a bug. The actual path model-based bug toss graphs need to be explored in the realm of open source systems in detail. 3.2 Concept drift Software projects evolve through time. In open source projects, the software developers collaborate to develop a software. The software developers may become inactive after sometime and their area of expertise may change. The problem of concept drift is present in issue tracking systems. Software projects goes through periods of stability and instability (Ekanayake et al., 2009). Concept drift has been explored to some depth by using Window Frequency Baseline, SVM Minibatch, Perceptron, modified Bugzie and Regression with Stochastic Gradient Descent (Chrupala, 2012). The concept drift handling mechanism discussed here takes to account only the textual contents of the bug report. In the literature, concept drift handling techniques when taking the textual content as well as the tossing relations are based on two methods heuristics of software developer activity count and (Bhattacharya et al., 2012; Bhattacharya and Neamtiu, 2010) incremental learning framework (Bhattacharya et al., 2012; Bhattacharya and Neamtiu, 2010; Miao et al., 2010). The incremental learning framework used in the literature uses fixed size window method, which is too rigid to capture the software evolution process. The bug triaging process is a learning process where the automated bug triaging system has to learn from past experience to recommend the best set of collaborators. Recent data will have more weight. In order to make the triaging process robust, methods which exploit both the contents and the tossing relations have to be explored. Attention may be given to techniques such as adaptive window-based incremental learning, mechanisms to implement gradual forgetting and consolidation of past knowledge. Furthermore, the performance loss due the use of goal-oriented path model needs to be quantitatively assessed. 3.3 Metrics The metrics that are used in the bug triaging and ticket resolution systems are: accuracy precision

316 V. Akila et al. recall mean steps to resolve. Precision is a better parameter for software developer recommendation because the cost of false recommendation is much higher than in search engine. Furthermore, the mean steps to resolve parameter codify only the number of steps in the predicted path. While the reduction in the number of steps to resolve is required, it is also vital to compare how far the predicted path is similar to the original path. The structure, ordering of nodes in the predicted path needs to be compared with that of the original path. Metrics based on graph edit distance need to be introduced. 4 Conclusion Bug triaging is an important aspect of open source software maintenance. A broad study is made on the state-of-art practices in bug triaging in open source systems. The state-ofart practices are compared with the practices in enterprises. It has been understood that many concepts of open source systems have been adapted from the enterprise systems. The most important adaptation is the modelling of the bug toss graph based on Markov model. From the survey, the classification of techniques handling bug triage has been derived. It is inferred from the survey that bug triage techniques uses the textual contents in the bug report or the tossing relations that exists among the software developers or both. The performance evaluation of the automated bug triage system is based on the parameters precision recall accuracy mean steps to resolve. There are a few open areas of research that need to be explored in the area of bug triage. At present, goal-oriented path models are widely adapted in bug triage. The performance of goal-oriented path model needs to be compared with that of actual path model in a quantitative manner. Concept drift is present in bug tracking. Efficient techniques to handle concept drift that uses both the textual content and the tossing relations are the need of the hour. Effective knowledge management techniques are necessary to convert the raw information in the bug repository to intelligence which may be exploited by the software practitioners. References Ahsan, S., Ferzund, J. and Wotawa, F. (2009) Automatic software bug triage system (BTS) based on latent semantic indexing and support vector machine, The Fourth International Conference on Software Engineering Advances, pp.216 221.

Bug triage in open source systems: a review 317 Aljarah, I., Banitaan, S., Abufardeh, S., Jin, W. and Salem, S. (2011) Selecting discriminating terms for bug assignment: a formal analysis, Proceedings of the 7th International Conference on Predictive Models in Software Engineering, pp.12:1 12:7. Anvik, J. and Murphy, G. (2011) Reducing the effort of bug report triage: recommenders for development-oriented decisions, ACM Trans. Software Engineering & Methodology, pp.10:11 10:35. Anvik, J., Hiew, L. and Murphy, G.C. (2006) Who should fix this bug?, 28th International Conference on Software Engineering, pp.361 370. Banitaan, S. and Alenezi, M. (2013) TRAM: an approach for assigning bug reports using their metadata, Third International Conference on Communications and Information Technology (ICCIT), pp.215 219. Baysal, O., Godfrey, M.W. and Cohen, R. (2009) A bug you like: a framework for automated assignment of bugs, Proceedings of International Conference on Program Comprehension, pp.297 298. Bhattacharya, P. and Neamtiu, I. (2010) Fine-grained incremental learning and multi feature tossing graphs to improve bug triaging, IEEE International Conference on Software Maintenance, pp.1 10. Bhattacharya, P., Neamtiu, I. and Shelton, C. (2012) Automated, highly accurate, bug assignment using machine learning and tossing graphs, Journal of Systems and Software, pp.2275 2292. Bird, C., Murphy, B., Nagappan, N. and Zimmermann, T. (2011) Empirical software engineering at Microsoft research, ACM Conference on Computer Supported Cooperative Work, pp.143 150. Bortis, G. and Hoek, A.V.D. (2013) PorchLight: a tag-based approach to bug triaging, International Conference on Software Engineering, pp.342 351. Chen, L., Wang, X. and Liu, C. (2010) Improving bug assignment with bug tossing graphs and bug similarities, International Conference on Biomedical Engineering and Computer Science (ICBECS), pp.421 425. Chen, Y., Tao, S., Yan, X., Anerousis, N. and Shao, Q. (2010) Assessing expertise awareness in resolution networks, International Conference on Advances in Social Networks Analysis and Mining, pp.128 135. Chrupala, G. (2012) Learning from evolving data streams: online triage of bug reports, 13th Conference of the European Chapter of the Association for Computational Linguistics, pp.613 622. Cubranic, D. and Murphy, G.C. (2004) Automatic bug triage using text categorization, 16th International Conference on Software Engineering & Knowledge Engineering, pp.92 97. Ekanayake, J., Tappolet, J., Gall, H.C. and Bernstein, A. (2009) Tracking concept drift of software projects using defect prediction quality, 6th International Working Conference on Mining Software Repositories, pp.51 60. Guo, P.J., Zimmermann, T., Nagappan, N. and Murphy, B. (2010) Characterizing and predicting which bugs get fixed: an empirical study of Microsoft windows, International Conference on Software Engineering, pp.495 504. Guo, P.J., Zimmermann, T., Nagappan, N. and Murphy, B. (2011) Not my bug! And other reasons for software bug report reassignments, ACM Conference on Computer Supported Cooperative Work, pp.395 404. Jeong, G., Kim, S. and Zimmermann, T. (2009) Improving bug triage with bug tossing graphs, Joint 12th European Software Engineering Conference (ESEC) and 17th ACM SIGSOFT Symposium on the Foundations of Software Engineering, pp.111 120. Kagdi, H., Gethers, M., Poshyvanyk, D. and Hammad, M. (2011) Assigning change requests to software software developers, Journal of Software: Evolution and Process, pp.3 33. Kevic, K., Muller, S.C., Fritz, T. and Gall, H.C. (2013) Collaborative bug triaging using textual similarities and change set analysis, 6th International Workshop on Cooperative and Human Aspects of Software Engineering (CHASE), pp.17 24.

318 V. Akila et al. Mani, S., Nagar, S., Mukherjee, D., Narayanam, R., Sinha, V.S. and Nanavati, A.A. (2013) Bug resolution catalysts: Identifying essential non-committers from bug repositories, 10th IEEE Mining Software Repositories (MSR), pp.193 202. Matter, D., Kuhn, A. and Nierstrasz, O. (2009) Assigning bug reports using a vocabulary-based expertise model of software developers, Proceedings of 6th IEEE International Working Conference on Mining SoftwareRepositories, pp.131 140. Medem, A., Akodjenou, M. and Teixeira, R. (2009) TroubleMiner: mining network trouble tickets, IEEE International Symposium on Integrated Network Management Workshops, pp.113 119. Miao, G., Moser, L.E., Yan, X., Tao, S., Chen, Y. and Anerousis, N. (2012b) Reliable ticket routing in expert networks, in Dai, H., Liu, J.N.K. and Smirnov, E. (Eds.): Reliable Knowledge Discovery, Springer Science + Business Media, pp.127 147. Miao, G., Moser, L.M. and Yan, X. (2010) Generative models for ticket resolution in expert networks, Proceedings of the 16th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp.733 742. Miao, G., Tao, S., Cheng, W., Moulic, R. and Moser, L.E. (2012a) Understanding task-driven information flow in collaborative networks, Proceedings of the 21st International Conference on World Wide Web, pp.849 858. Moin, A.H. and Neumann, G. (2012) Assisting bug triage in large open source projects using approximate string matching, 7th International Conference on Software Engineering Advances, pp.22 27. Naguib, H., Narayan, N., Brugge, B. and Helal, D. (2013) Bug report assignee recommendation using activity profiles, 10th IEEE Working Conference on Mining Software Repositories, pp.22 30. Nguyen, T.T., Nguyen, A.T. and Nguyen, T.N. (2014) Topic-based, time-aware bug assignment, ACM SIGSOFT Software Engineering Notes, p.14. Park, J., Lee, M., Kim, J. and Hwang, S. (2011) COSTRIAGE: a cost-aware triage algorithm for bug reporting systems, Proceedings of the 25th Conference of Artificial Intelligence, pp.139 144. Shao, Q., Chen, Y., Tao, S., Yan, X. and Anerousi, N. (2008) Efficient ticket routing by resolution sequence mining, Proceeding of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp.605 613. Shokripour, R., Anvik, J., Kasirun, Z.M. and Zamani, S. (2013) Why so complicated? Simple term filtering and weighting for location-based bug report assignment recommendation, 10th IEEE Working Conference on Mining Software Repositories, p.11. Sun, P., Tao, S., Yan, X., Anerousis, N. and Chen, Y. (2010) Content-aware resolution sequence mining for ticket routing, Proceedings of the 8th International Conference on Business Process Management, pp.243 259. Tamrawi, A., Nguyen, T., Al-Kofahi, J. and Nguyen, T. (2011) Fuzzy set-based automatic bug triaging (NIER Track), Proceedings of the 33rd International Conference on Software Engineering, pp.884 887. Tan, S., Hu, S. and Chen, L. (2010) A framework of bug reporting system based on keywords extraction and auction algorithm, China Grid Conference, pp.281 284. Wang, S., Zhang, W., Yang, Y. and Wang, Q. (2013) DevNet: exploring developer collaboration in heterogeneous networks of bug repositories, IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM), pp.193 202. Wu, W., Zhang, W., Yang, Y. and Wang, Q. (2011) DREX: developer recommendation with K-nearest-neighbor search and expertise ranking, 18th Asia-Pacific Software Engineering Conference, pp.389 396. Xiaz, X., Loy, D., Wang, X. and Zhou, B. (2013) Accurate developer recommendation for bug resolution, 20th Working Conference on Reverse Engineering, pp.72 81.

Bug triage in open source systems: a review 319 Xie, X., Zhang, W., Yang, Y. and Wang, Q. (2012) DRETOM: developer recommendation based on topic models for bug resolution, 8th International Conference on Predictive Models in Software Engineering, pp.19 28. Xuan, J., Jiang, H., Ren, Z. and Zou, W. (2012) Developer prioritization in bug repositories, International Conference on Software Engineering, pp.25 35. Yang, G., Zhang, T. and Lee, B. (2014) Utilizing a multi-developer network-based developer recommendation algorithm to fix bugs effectively, 29th ACM Symposium on Applied Computing, pp.1134 1139. Zhang, T. and Lee, B. (2013) A hybrid bug triage algorithm for developer recommendation, 28th Annual ACM Symposium on Applied Computing, pp.1088 1094. Zhang, W., Wang, S., Yang, Y. and Wang, Q. (2013) Heterogeneous network analysis of developer contribution in bug repositories, International Conference on Cloud and Service Computing, pp.98 105. Zou, W., Hu, Y., Xuan, J. and Jiang, H. (2011) Towards training set reduction for bug triage, IEEE 35th Annual Computer Software and Applications Conference, pp.576 581.