ADVANCED SOFTWARE ENGINEERING COMP 3705 Exercise Software process simulation Department of Computer Science Sada Narayanappa Department of Communication Systems Carina Andersson, Thomas Thelin
1. Introduction Software process simulations are increasingly being used in a variety of areas in software engineering, e.g. to support process improvement, in software project management training, and as a tool for decision support in project planning. Models are always simplification of real systems, though simulations of these are useful means for observing and understanding dynamic phenomena in the system. 2. Learning Objectives The exercise aims at giving an understanding of software process simulation and project planning. The specific learning goal is to use a simulation model as decision support in the project planning, with a special focus on the verification and validation activities. 3. Preparation The preparation assignments should be done before the exercise in order to get input for the exercise. Start by reading appendix A. Assignment 1: Read chapter 9 in [Burnstein03] and the slides from lecture 4. Assignment 2: Specify the factors you believe have an impact on the progress of a test project. While specifying the factors consider the following questions: What do project managers take into account when planning for a new test project? Are there any differences compared to replanning in the middle of a project? Try to estimate the quantitative impact the specified factors could have on the test project. Assignment 3: Specify a module for the activity design inspection. Do a subjective estimate of how large proportion of the existing faults in the design documents that could be detected by a design inspection. Model the module on paper (tip: use the model template in Figure 1) and include variables that have impact on the fault detection. 4. Exercise Assignment 4: Use existing template model (download [model1]). Change the parameter values to reach the deadline. You are not allowed to reduce the product size of the product. Discuss advantages and disadvantages with the different solutions, and reflect on real-world approaches for reaching given deadlines. Assignment 5: Modify the model according to preparation assignment 2. Include the factors and variables that you have specified and relate their quantitative impact on the model to existing variables. Compare to previous runs. Discuss how the included activities have affected the process and the product s final result. Assignment 6: Modify and extend the model according to preparation assignment 3. How is the model behaviour changed? Assignment 7: Download [model2] and compare to the model you recently extended with the activites from the preparation tasks. 2
5. Report The purpose of the report is to discuss the result of the exercise and related topics. Write the name of the exercise, your names, group number and email addresses on the first page of the report. The size of the report should be 4-5 Letter size pages (including the first page). Two parts should be included, 1) conclusions from the lab session, and 2) discussion of exercises from the course literature: 1. Write a report (1-2 Letter size pages) and describe the outcome of the lab session. Discuss the assignments and include the following: Advantages and disadvantages with software process simulation models. Compare to other means that project managers might use for decision support. Improvement possibilities. Consider how the model could be improved, in terms of being more realistic, or give more accurate output. How the model could be extended to be used as a prediction model. What decisions during software development can be taken based on the results. The problems you discovered, in terms of origin and possible solutions. In addition, discuss the following related topics in the report (1-2 Letter size pages): [Burnstein03, chapter 9, pages 300-301] Exercise 9.6 [Burnstein03, chapter 9, pages 300-301] Exercise 9.14 6. References [Burnstein03] Burnstein, I., Practical Software Testing A Process-Oriented Approach, Springer-Verlag, 2003. [Kellner99] Kellner, M., Madachy, R., Raffo, D., Software Process Simulation Modeling: Why? What? How?, Journal of Systems and Software, 46(2-3):91-105, 1999. Appendix Background information Simulation is an approach for creating an understanding of the mechanisms affecting the development process. An implemented computer model, calibrated with empirical data, can be executed, simulating the real process behaviour. Empirical issues related to software process simulations concern the analysis of the process data, as direct input to the model or used for the model building. Other areas of concern are the model output data, used as support for planning and management decisions, and the model structure in the context of evaluating the efficiency of a process. Hence, to use a simulation model, the model should be customized to an organization, to ensure that the right metrics are used as input. The models do often follow a generic waterfall life cycle, presenting either the whole development process, from requirements to system testing, or chosen parts of it, like the unit test phase. One benefit of simulation modelling is that to get an accurate model the activities have to be clearly identified. Furthermore, dependencies, as well as the feedback between various activities, have to be defined. 3
The above discussion illustrates that there are many variables that affect the software development process. Furthermore, these variables are not independent but are related to one another in complex ways. Understanding the behaviour of such systems could be complex far beyond the capacity of human intution, and simulations of a modelled system could be of great assistance. We will use simulations as an analysis tool, an approach often used since experiments and realworld trials often are too expensive and difficult to conduct. A model of the essential structure of the situation of interest is designed and computer simulated, though it is still only a model of the real world. Simulation of software development processes can be used for extending the understanding of the interdependencies between different activities in the process, or be used as means for an organization to evaluate process changes, etc. Simulations are classified in two different types, discrete-event and continuous [Kellner99]. In discrete-event simulation, the state of the system is changed only when certain events occur. In continuous simulations, also referred to as system dynamics, the state changes continuously over time, and the simulation model is designed by differential equations. The system dynamics paradigm will be used in this exercise, mainly because this paradigm is better for representing the project environment and the feedback loops that may exist in the project environment. Terminology A simulation tool named Vensim (http://www.vensim.com) will be used for the exercise. It has a graphical interface with only a few generic building block to use in the model diagrams. The main building blocks are: Table 1: Building blocks Symbol Name Description Level Rate Also called accumulators, stocks or reservoirs. Elements for storage. The elements that determine the flow rates in-between level component. Auxiliary Auxiliary Source/Sink Variables, or auxiliary equations allow to structure the decision processes in order to resemble real systems or to make a model more readable. A limitless destination or origin, indicating a reservoir which is outside the system scope or which is not modelled in more detail. In Figure 1 a Vensim model is shown. The model is a simplified reflection of a single test phase, where most of the affecting factors are peeled off. The model is based on a flow of tasks, from Code to be tested to Tested code. In the test phase there is a transformation from uncompleted tasks to completed tasks by the Completion rate, which in this case reflects the testing. 4
Rework Code to be tested Uncompleted tasks Completion rate Completed tasks Tested code Resources Figure 1. Basic Vensim model A fraction of the tasks are not acceptable and needs to be taken care of in the rework loop. One variable is included in the model as a constant, Resources, along with an arrow representing dependency. 5