Learning Software Evolution: Curriculum, Approaches and an Experience Report

Learning Software Evolution: Curriculum, Approaches and an Experience Report Debora Maria Coelho Nascimento 1, Christina Chavez 2, Roberto Almeida Bittencourt 3 1 Federal University of Sergipe (UFS) São Cristovão SE Brazil 2 Federal University of Bahia (UFBA) Salvador BA Brazil 3 State University of Feira de Santana (UEFS) Feira de Santana BA Brazil dmcnascimento@ufs.br, flach@dcc.ufba.br, roberto@uefs.br Abstract. Software evolution tasks impose increasing challenges and require huge investments from industry and software engineers seem to be not prepared properly to deal with them. One possible way to handle this problem is teaching software evolution in graduate and undergraduate courses. To this end, several reference materials have been proposed and some courses try to implement them to achieve improved competencies for their students. This paper presents preliminary results of an investigation we have performed on how software evolution topics have been approached within reference curricula and some academic courses, and also reports our own experience within a graduate course on software evolution. 1. Introduction Literature provides reports of studies where software evolution accounts for more than 65% of the total software cost [Petrenko et al. 2007]. In case of systems with a long lifetime, software maintenance and evolution (SME) activities account for up to 50-75% of total software life-cycle costs [Koskinen 2011]. Thus, it is usual for software engineers in industry to spend a majority of their time understanding and evolving legacy software [McCartney et al. 2012]. These statistics provide evidence on the importance of these activities in the field of Software Engineering (SE). Evolving a large software product is a challenging task [van Deursen et al. 2003]. One needs to comprehend and modify a system that was mostly developed by others, that many users depend upon, and which is frequently complex and poorly structured. Often there is no other documentation apart from the source code, or the documentation is outdated. Previous research has been performed to facilitate such activities. However, there is still little impact of this research on software industry practices, which are frequently performed as a craft activity, without the support of proper methods. Furthermore, despite the availability of tools, software engineers were not trained for them and do not know how to interpret their outputs [van Deursen et al. 2003]. Beyond this, we can add the fact that maintaining a software system is usually the first activity assigned for recent graduates in their first jobs. Consequently, it is their first responsibility.

At the university, most SE courses overly emphasize software construction as opposed to maintenance and evolution [McCartney et al. 2012]. Students are trained in developing small programs, in size and complexity, starting from scratch, and are barely exposed to software change tasks on more realistic software projects [van Deursen et al. 2003]. Evolving large and complex software systems comprises: (i) understanding documentation; (ii) recovering and understanding the architecture; (iii) understanding some part of the source code; (iv) locating changes; (v) performing impact analysis, among other activities. These activities are quite challenging, both for industrial and academic settings. Mens et al [Mens et al. 2005] described some research challenges in software evolution, emphasizing that one of the best ways to place change and evolution in the center of the development process is to educate future generations of software engineers. In this context, one challenging question is how software evolution topics can be properly approached in software engineering curricula. This paper presents the preliminary results of an investigation we have performed on how software evolution has been dealt within some reference curricula. We have provided some examples of courses and also reported our own experience on a graduate course on software evolution. This document is organized as follows. Section 2 presents a brief summary on how important reference curricula treat this topic. Section 3 presents some interesting academic initiatives that work with software evolution topics. Section 4 reports our own onesemester initiative on teaching software evolution topics and Section 5 discusses some topics. Finally, Section 6 presents some concluding remarks. 2. Software Evolution in the Curriculum In order to better understand how software evolution is treated within the curriculum, we first analysed some reference curricula and guidelines, and their account of this subject area. Later, we provide examples of curricula implementation from undergraduate courses offered by three Brazilian universities. The Association for Computing Machinery (ACM) and the IEEE Computer Society have published the Computing Curricula Series [ACM and IEEE 2006], providing recommendations to five undergraduate programs, described below. In the 2010 Information Systems (IS) Curriculum [ACM and AIS 2010], the term maintenance is used at the discipline scope, but no specific course, topic or unit was proposed. Systems Analysis & Design is defined as a core course and Application Development as an elective course. While the former focuses on requirements, the latter focuses on design, implementation and testing. Both courses are foundations to one career track called Application Developer, what reveals the importance assigned to software development, in contrast to SME. In the 2008 Computer Science (CS) Curriculum [ACM and IEEE 2008a], Software Evolution is a core topic within the Software Engineering knowledge area (KA). The topic must have a minimum core coverage time of 3 hours, handling software maintenance, features of maintainable software, reengineering legacy systems, refactoring and software reuse. Maintenance is also mentioned in the Software Processes and Spe-

cialized Systems topics. In the 2008 Information Technology Curriculum [ACM and IEEE 2008b], only one KA, System Administration and Maintenance, references the term maintenance. However, it is not related to SE and covers topics such as operating systems, applications, and administrative activities and domains. The SE 2004 Curriculum Guidelines for Software Engineering [ACM and IEEE 2004b] states that evolution is a broad concept that expands upon the traditional notion of software maintenance and defines Software Evolution as a KA with a minimum coverage time of 10 hours. But the term maintenance is also mentioned in the Software Process and Software Quality KAs, and a topic is included in the Software Management KA. The Computer Engineering (CE) Curriculum [ACM and IEEE 2004a] describes two units: the Software Maintenance unit within the Computer Systems Engineering KA and Software Evolution within the Software Engineering KA, both with a minimum core coverage time of 2 hours. However, the topics specified for these two units have some overlapping content, such as software maintenance, impact analysis and reuse. ACM also provides recommendations for the graduate degree in SE [issec Project 2009]. In this case, Software Maintenance is a unit within the Software Engineering KA, which is subdivided into topics about maintenance fundamentals, processes and techniques. There is also another unit called Operation, Maintenance and Supporting within the Systems Engineering KA, but the topics are not described. The text emphasizes that it is a mistake to view the other development KAs as independent areas of software maintenance and to focus on development from scratch. Finally, the SWEBOK Software Engineering Body of Knowledge Guide currently in the public review phase of its third version [IEEE 2013], provides a reference to Lehman s Laws of Evolution, defining that maintenance is evolutionary development and maintenance is continued development. The guide sets software maintenance as one of the fifteen knowledge areas (KA) that provide the foundational knowledge for SE, thus reinforcing the relevance of the subject for the software engineer s background. In Brazil, the Brazilian Computer Society (SBC) has also produced recommendations for four undergraduate programs. The 2003 IS curriculum [SBC 2003] mentions maintenance as an activity in the software development process, with no emphasis at all. On the other hand, the CS and CE Curricula are described in a single document, available since 2005 [SBC 2005]. This document treats maintenance, reverse engineering and reengineering as topics in the Software Engineering subject matter within the Computing Technology core. In the Computing Education reference curriculum [SBC 2002], it is recommended that the area of SE should be comprehensively studied with emphasis on developing environments and systems for use in learning and other educational processes. The reference curriculum for Software Engineering has not been yet completed. The proposal for curricular guidelines from the Brazilian Ministry of Education (MEC) [MEC 2012] cites the term evolution when describing the expected competencies for CS, SE or IS graduates. However, the topic software evolution only shows up in the SE curriculum.

The Federal University of Sergipe offers three undergraduate programs: IS, CS and CE. Software maintenance is one topic within the Software Development III course, only in the CS Undergraduate Curriculum. For the other two curricula, there is no reference to software maintenance or evolution [UFS 2008]. However, faculty responsible for the courses on SE report that they address the topic in, at least, one lecture. The State University of Feira de Santana offers the Computer Engineering Undergraduate program. By insistence of some faculty, the Software Evolution and Maintenance was added as an optional course [UEFS 2012]. Finally, the Federal University of Bahia also offers three undergraduate programs: CS, IS and Computing Education [UFBA 2013]. In the CS course, since 2007, the Software Evolution elective course is provided every year. In the other two curricula, maintenance is one topic in the Software Engineering course. 3. Some Initiatives El-Ramly described an experience whose goal was to provide MSc. students with basic knowledge and training in reengineering [El-Ramly 2006]. The course covered reverse engineering and the forward engineering aspect of reengineering. Their approach used lectures, discussion classes, and computer labs with some tools where students had to work in a project to practice and expand on what they had learned. Among the lessons learned reported in their study, we highlight the need to cover practical tools and some industrial case studies and the use of few tools to reduce students workload. Petrenko et al. offered a course to senior undergraduates and first-year graduate students, in which they learnt about tasks, tools and techniques of an evolution process and practiced on software systems of considerable size [Petrenko et al. 2007]. Students had introductory lectures about software change process and tools, and had to perform required changes of increasing difficulty in an open source project. Koskinen [Koskinen 2011] used systematic seminars to teach software maintenance and evolution. The seminars were organized as part of a Software Engineering course, usually taken during the second year for undergraduate students in computing. Before the seminars, the instructor introduced the related area by means of lectures. Students had to form groups autonomously, and each group had to analyze one scientific paper and present its analysis at the seminar, write a careful summary report of the paper, serve as the opponent to another group, and listen to the presentations of other groups. The first two papers emphasized the importance of the students participating in practical projects, especially in projects closer to industrial reality, while the last one emphasized the use of seminars to provide a broad scientific view of the area to students. 4. Our Experience In this section, we report our experience in teaching software evolution to graduate students. Two authors of this paper were responsible for the Topics in Software Engineering 1 course, in an elective course from the Computer Science Graduate Program at the Federal University of Bahia. 4.1. Methodology The topics covered in our course were basic concepts of evolution and maintenance, laws of software evolution, software aging, software comprehension, reverse engineer-

ing, change requests, change classification, concept location, impact analysis, refactoring, tests in the change context and the mining of software repositories. Since the audience was composed of graduate students, we decided for a research approach. Students had to discuss concepts and trends in the area of software evolution based on scientific papers. They also had to use techniques and tools to study attributes of one software project during its evolution and present the results of the analysis. The topics were presented in lectures or individual seminars. Except for the first class, students had to prepare critical reviews for one or two scientific articles suggested. The goal was to generate discussion about the theme and the quality of the articles. A list of themes was suggested for the seminars, so that each student could choose one with better fit to his/her background and interests. A set of papers about each theme was also provided, but the student could research other sources and tools to support activities related to the chosen theme. The seminar was an individual task. To add a more practical view of the research, students, in pairs or in groups of three, had to produce research on mining software repositories related to evolution or maintenance for one or more available software projects chosen by the students. Thus, students had to identify issues, define research questions and methodology, perform the data collection, and analyze the results. The study proposal was initially validated by the faculty. Finally, students had to present the work in an oral session and write a report. The faculty effort was distributed into planning and leading. The effort during planning was reduced because of the previous experience in similar courses. Most of the time was used to define the themes and pre-select papers related to them, and also to define the evaluation criteria. Leading, time was needed to read the suggested papers and assessing the critical reviews, seminars and practical projects. 4.2. Students Perspective To obtain student s appraisal on course relevance and methodology, they were asked to fill an anonymous web survey, which had 8 responses in an universe of 11 students. Since the students were graduate students, we decided to characterize their experience in industry. Only one student did not have any work experience and most (75%) of them had more than five years experience. For those, 57% had 50% of their time working with maintenance activities and 43% had 75% of their time. When the students were questioned if they had previously studied software evolution, 50% answered that had never done it, while 10% had studied at a company, 10% on their own, and only 30% had studied it in college. However, 100% considered important that the theme be addressed in computer courses in academy, from that 50% considers that should exist a dedicated course for the theme. The opinion about this importance was changed after their participation in this course, for 75% of the students. Regarding the used methodology, 88% considered the duration appropriate, 100% considered the content appropriate, of whom 38% considered the content very important and 25% considered it important for a software engineer working in industry. Only 12.5% considered inappropriate the approach using seminars and practical projects, while 50% considered it good and 37.5% found it very good. In a self-assessment, 12.5% found their

learning less than satisfactory, 12.5% found it satisfactory and 75% found it good. From open questions, students could suggest improvements and indicate what they liked best. Some improvements suggested were: reducing the number of assignments of critical reviews, more feedback about the reviews, dedicating more time to discussions; faculty should discuss the seminar with the student preparing it before the oral presentations, for better quality and more uniform contents; the project should start at the beginning of the course, with more discussion about it, during the whole course. Students especially appreciated: the diversity of content that allowed a broad view of the maintenance and evolution area; the free choice of theme for seminars and practical project; knowledge and experience of students and faculty allowed enriching discussions; the practical project that allowed the comprehension of some aspects related with software evolution; and finally the critical reading of the scientific papers reported in the reviews. 4.3. Faculty s Perspective The faculty answered an evaluation questionnaire. The third author of this study was in charge of elaborating and evaluating answers of both questionnaires to avoid bias. Analyzing the students results, the instructors considered: (i) good results for critical reviews, with quality increase after the first results, although some students insisted on not do a critical analysis and only summarized the articles; (ii) for seminars, the results were regular, a few were very good, some were good, but many failed; and (iii) for the projects, the results were very good, surprising the instructors. In general, they considered having good results. Regarding the methodology, they considered that the success was partial. On the one hand, seminars turn students more active and cover a broad view of topics, on the other hand, when it fails, all students are harmed, since the topic is not covered in adequate way anywhere else. The tool demonstrations were mostly incomplete, reducing the potential practical impact on the students daily work. Although the critical reviews increased the students workload, they stimulated the discussions in class. In general, the practical projects were very good, providing students some experience in performing research and a view of evolution from the perspective of mining repositories. Regarding class time, it was adequate for a graduate course, where it is required more individual and extra-class work from students. Reflecting about what could be better, they concluded that is necessary: (i) to increase the monitoring of the projects, setting some milestones during the execution phase; (ii) to assure the content coverage with more lectures and assuring that the seminars have appropriate depth; and (iii) to improve the use of tools, maybe by adopting tutorials. 5. Discussion In the beginning of this study, we discussed the relevance of the software evolution area. This fact was supported by our survey results, where all graduate students participating in this study that had already worked at industry used at least 50% from their work time with maintenance activities. Despite this involvement, 75% of the students changed their opinion about the importance of teaching software maintenance and evolution, only after studying it in our course. One possible cause for this lack of awareness could be the

relative lack of knowledge about the techniques to evolve or maintain software systems. Perhaps, this is because teaching maintenance in college, most of the times is focused on the maintenance process, prioritization and impact analysis. And, as we mentioned, lots of concepts, processes and techniques must be covered. Regarding computing curricula, starting with ACM recommendations, we noticed the absence of coverage of the subject in the IS Curriculum, regardless of the large number of legacy systems existing in companies that must be maintained. For the CS and CE Curricula, the minimum core coverage time specified only allows a shallow learning of concepts. To practice the techniques, extra time is needed. Considering the SBC and MEC recommendations and the relevance of the theme, from our point of view, the matter has been neglected. In practice, using three Brazilian universities as examples, we identified that, in present, the coverage of this area depends on faculty staff s interests, even though this interest has increased in recent years. In our opinion, to introduce SME in the curriculum by a more traditional approach, there should exist one core course for SE and IS undergraduate programs, and one elective course for CS and CE programs. In a more revolutionary approach, following recently ideas [Rajlich 2013], the SE discipline could be introduced using a software evolution approach. Since SME covers important aspects of the software engineering area and corresponds to majority of activities which graduates will perform in their future jobs. 6. Conclusion In this study, we aimed to highlight the relevance of teaching and learning software evolution, so that universities can better prepare computing professionals to the challenges of industry of maintaining important software systems. Therefore, craft activities need to be replaced by proper methods. However, we noticed some failed coverage of SME in university-level, on the examples studied, possibly reflecting the importance found in curricula references. One solution would be raising faculty awareness about the relevance of evolving and maintaining a software system, activities as important as developing new systems from scratch, as it typically happens in most colleges and universities. Studies like this can help in this task. Teaching software maintenance and evolution is not an easy task, but there are some interesting initiatives that can be followed and adapted for different contexts. In this study, we reported our experience in teaching software evolution in a graduate course. The methodology used critical reviews, seminars and a practical project and was deemed satisfactory in this case. Beyond the adoption of the improvements suggested in the next graduate course, we are planning one undergraduate course that will be offered in the second term of the present academic year. The approach will be different, because, in this case, the practical project should reflect activities found in industry. References ACM and AIS (2010). IS 2010: Curriculum Guidelines for Undergraduate Degree Programs in Information Systems. ACM and IEEE (2004a). Computer Engineering 2004: Curriculum Guidelines for Undergraduate Degree Programs in Computer Engineering.

ACM and IEEE (2004b). Software Engineering 2004: Curriculum Guidelines for Undergraduate Degree Programs in Software Engineering. ACM and IEEE (2006). Computing Curricula 2005: The Overview Report. ACM and IEEE (2008a). Computer Science Curriculum 2008: An Interim Revision of CS 2001. ACM and IEEE (2008b). Information Technology 2008: Curriculum Guidelines for Undergraduate Degree Programs in Information Technology. El-Ramly, M. (2006). Experience in Teaching a Software Reengineering Course. In Proc. of the 28th Int. Conf. on Software engineering - ICSE 06, page 699. IEEE (2013). SWEBOK V3 Review: Guide to the Software Engineering Body of Knowledge. issec Project (2009). Graduate Software Engineering 2009: Curriculum Guidelines for Graduate Degree Programs in Software Engineering. Koskinen, J. (2011). Seminars on Software Maintenance and Evolution An Empirical Study of the Background Factors Affecting Student Success. McCartney, R., Gokhale, S. S., and Smith, T. M. (2012). Evaluating an Early Software Engineering Course with Projects and Tools from Open Source Software. In Proc. 9th International Computing Education Research - ICER 12, page 5. MEC (2012). Parecer CNE/CES N o :136/2012 - Diretrizes Curriculares Nacionais para os cursos de graduação em Computação. Mens, T., Wermelinger, M., Ducasse, S., Demeyer, S., Hirschfeld, R., and Jazayeri, M. (2005). Challenges in Software Evolution. In Proc. of 8th Int. Workshop on Principles of Software Evolution, page 10. Petrenko, M., Poshyvanyk, D., Rajlich, V., and Buchta, J. (2007). Teaching Software Evolution in Open Source. Computer, 40(11):25 31. Rajlich, V. (2013). Teaching Developer Skills in the First Software Engineering Course. In Proc. of the 35th Int. Conf. on Software engineering - ICSE 2013, page 8. SBC (2002). Currículo de Referência para Cursos de Licenciatura em Computação. SBC (2003). Currículo de Referência da SBC para Cursos de Bacharelado em Sistemas de Informação - Versão 2003. SBC (2005). Currículo de Referência para Cursos de Graduação em Bacharelado em Ciência da Computação e Engenharia da Computação. UEFS (2012). Resolução CONSEPE 206/2012. UFBA (2013). Departamento de Ciência da Computação - UFBA - Graduação. http://wiki.dcc.ufba.br/dcc/ensinograduacao. UFS (2008). Resolução n. 50/2008/CONEPE. van Deursen, A., Favre, J.-M., Koschke, R., and Rilling, J. (2003). Experiences in Teaching Software Evolution and Program Comprehension. In Proc. Int. Symposium on Micromechatronics and Human Science - MHS2003., pages 283 284.