A s Perspective on Agile Software Development Methods Peter Wendorff ASSET GmbH, Oberhausen, Germany P.Wendorff@t-online.de Abdul Rashid Subhani Pro-Net Internet Services Ltd., London, UK Rashid@pro-net.co.uk ABSTRACT Since the late 1990s a number of so-called "agile" software development methods have been proposed to overcome problems experienced with more traditional approaches. This has led to a controversial discussion within the software engineering community illustrating the need for an integrating theoretical framework to clarify the concept of agile software development methods. In this paper we use general systems theory to characterise software development processes. Using the Unified Software Development Process and Extreme Programming as examples we demonstrate that general systems theory allows a clear and meaningful characterisation of agile methods. Keywords: Agile Software Development Methods 1.0 Introduction A software development method (SDM) describes a systematic way to produce software, for example, executable computer programs. Usually such a method comprises a software development process model (SDPM) that provides an abstraction of the SDM's underlying process. Obviously, SDPMs are of paramount economic relevance, because they govern the way organisations spend expensive resources on the development of software. There are many differing proposals for SDPMs available, for example, recently Jacobson et al. (1999) published the "Unified Software Development Process" (USDP), and Beck (2000) published "Extreme Programming" (XP). Recently, a number of so-called "agile" SDPMs have been proposed. Agile SDPMs share a core of values published in Cockburn (2001) as "The Agile Software Development Manifesto". Agile SDPMs claim to be superior to other processes in some situations where requirements are volatile, early user feedback is essential, and time-to-market cycles are compressed. The USDP is a very successful SDPM that is usually not regarded as agile. It has been developed by some of the most recognised authorities in the software engineering community and clearly epitomises a very sophisticated engineering approach to the development of software systems. XP is by far the most popular agile SDPM, and even the first ever dynabook IEEE (2000) of the Institute of Electrical and Electronics Engineers has been devoted to it. The application domains of XP and the USDP show a large overlap, and therefore, these two methods compete in many areas of software engineering. This has led to a vivid and often controversial exchange of opinions, for example, published in the dynabook IEEE (2000). The observation by Jawed Siddiqi of "both enthusiastic support and equally vigorous criticism" of XP in IEEE (2000) indicates the need for an integrative theoretical framework for SDPMs in software engineering. In this paper we use general systems theory as an integrative theoretical framework to analyse SDPMs and use it to develop a characterisation of agile SDPMs.
2.0 Modelling Software Development Processes A SDPM is a prescriptive model of a SDM's underlying process. Pressman (1997) gives a comprehensive overview of published SDPMs since the early 1970s. Pressman does not include XP or the USDP, and he does not discuss agile SDPMs in general, because these methods were not published at the time of his writing. Therefore, Pressman (1997) can be regarded as a comprehensive overview of published SDPMs until the mid 1990s. Weinberg (1992) uses the cybernetic system shown in Figure 1 to represent SDPMs. This system consists of three subsystems, namely Controller, Software Development, and Customer. Almost all of the models discussed by Pressman (1997) can be subsumed under the basic model given in Figure 1. Controller Requirements Resources Randomness Software Development Software Other Outputs Customer Figure 1: A s Perspective on SDPMs (Weinberg, 1992, Figure 11-4) The Software Development has Requirements and Resources as major inputs, and software as major output. Most of the SDPMs discussed by Pressman (1997) focus on this system and define in detail its static properties and dynamic behaviour. According to Weinberg (1992) the Controller represents all efforts to keep the software development on track. This comprises the classical controlling function of project leaders. Weinberg argues that practical Controller s frequently do not have direct access to the state of the Software Development, instead they rely on its outputs and inputs. The Customer is a crucial element of the software development process, and Weinberg (1992) devotes chapter 11 in his book to a discussion of this system. The customer is vital, because he is the reason for the operation of the Software Development, but the Customer can also have a destabilising effect on the Software Development Process. Essentially, the model of a software development process shown in Figure 1 consists of a Software Development that is controlled by the Controller and the Customer that operate as feedback loops. The Controller and the Customer do not have direct access to the state of the Software Development, instead they monitor the outputs and contribute to the inputs of this system.
In the real world software development is a human activity that takes place in complex social systems. As Gharajedaghi (1999) notes, these social systems can exhibit counterintuitive behaviour caused by time delays, non-linearities, multidimensionality, etc. These phenomena make it extremely difficult for the Controller to find an appropriate corrective action. Therefore, Weinberg (1992) points out, exclusive reliance on outputs and inputs can severely limit the effectiveness of the Controller for two reasons. First, it is often impossible to infer the significant things that are really happening inside the Software Development from its outputs, for example, progress reports may be misleading. Second, the inputs provided by the Controller to the Software Development are based on a partial view of the latter, for example, this may lead to counterproductive appraisal schemes. The problems encountered with SDPMs corresponding to Figure 1 have become the subject of many popular books on software engineering management, e.g. Brooks (1995), DeMarco and Lister (1999), Weinberg (1992), and Weinberg (1998). Probably the best-known example of counterintuitive system behaviour is the infamous "Brooks's Law": "Adding manpower to a late software project makes it later." (Brooks, 1995, p. 25) 3.0 The Unified Software Development Process Three of the most recognised authorities in the software engineering community, Ivar Jacobson, Grady Booch, and James Rumbaugh (1999) invented the USDP in the 1990s. The USDP is a process framework, rather than a single process model, and it clearly epitomises a very sophisticated engineering approach to the development of information systems. In the preface of their book Jacobson et al. note the need for a software development process "that will be the best process the industry is capable of putting together at this point in its history." (Jacobson et al., 1999, p. xviii). Jacobson et al. argue that "the real distinguishing aspects of the Unified Process are captured in the three key words - use-case driven, architecture-centric, and iterative and incremental. This is what makes the Unified Process unique." (Jacobson et al., 1999, p. 4) Use cases are a simple tool to elicit, document, and manage user requirements using a mixture of diagrams and textual representations. Software architecture is an emerging field in software engineering, and the USDP probably puts more emphasis on this issue than most other SDPMs. Iterative and incremental SDPMs have been proposed long before the publication of the USDP (cf. Pressman, 1997), and therefore these ideas are not innovative at all. Apart from these three "distinguishing aspects" the USDP is an eclectic approach that relies on ideas and elements that are part of many SDPMs discussed by Pressman (1997). The USDP defines phases, iterations, workflows, activities, roles, artefacts, deliverables, controls, etc. in much the same way as these other approaches do. 4.0 Extreme Programming Beck (2000) invented XP in the late 1990s during a small software project. XP is the most prominent "agile" SDPM. XP is based on four values, and Beck argues: "Short-term individual goals often conflict with long term social goals. Societies have learned to deal with this problem by developing shared sets of values, backed up by myths, rituals, punishments, and rewards. Without these values, humans tend to revert to their own short-term best interest. The four values of XP are: communication, simplicity, feedback, and courage" (Beck, 2000, p. 29). Later Beck (2000) defines four basic activities (coding, testing, listening, and designing), twelve core practices that embody these activities, and a number of roles. Many of these activities, practices, and roles
are similar to those found in other SDPMs discussed by Pressman (1997), or the USDP, but three elements of XP mark a clear departure from these approaches: - the role "coach" - the role (and practice) "on-site customer" - the practice "pair programming" 5.0 Comparison of XP and the USDP In the preceding section we have identified three elements where XP differs noticeably from other SDPMs. In this section we will analyse what the particular contributions of these additions are. 5.1 Coach A coach is an integral part of every XP team. Beck gives this characterisation of coaching: "Coaching is primarily concerned with the technical execution (and evolution) of the process." (Beck, 2000, p. 73) "As coach, you are responsible for the process as a whole. You notice when people are deviating from the team's process and bring this to the team's attention." (Beck, 2000, p. 145) Jacobson et al. (1999) do not define a role comparable to the coach in XP. 5.2 On-Site Customer Beck (2000) states: "XP calls for the customer to be an integral part of the team. The specification of the project is continuously refined during development, so learning by the customer and the team can be reflected in the software." (Beck, 2000, p. 4) This on-site customer is a mandatory element of any XP project team. The USDP does clearly not regard the customer as integral part of the development team. Instead, the two parties are separated and do mainly communicate through documents at predefined stages during the development process. For example, requirements are mainly communicated by use-cases. These use cases are elaborated by analysts during workshops with the users and must be reasonably complete and consistent before subsequent software development acitivities can start. 5.3 Pair Programming One of the most controversial practices of XP is pair programming: "All production code is written with two people looking at one machine, with one keyboard and one mouse." (Beck 2000, p. 58) Beck regards pair programming as the ongoing execution of code reviews during programming. He draws on the generally accepted opinion that reviews are a very effective tool for software quality assurance, stating: "If code reviews are good, we'll review code all the time (pair programming)." (Beck 2000, p. xv) Jacobson et al. (1999) do not define a practice similar to pair programming. 6.0 Modelling Agile Software Development A closer analysis of the three distinguishing elements of XP given in section 5.0 shows that they all relate to the cybernetic system depicted in Figure 1. The intention in all three cases is to include parts of the feedback mechanisms shown in Figure 1 as Controller and Customer into the Software Development itself. This does not only apply to XP but also to other agile SDPMs (cf. Cockburn, 2001). The resulting system is depicted in Figure 2.
External Controller Requirements Resources Randomness Software Development Internal Controller Core Development Internal Customer Software Other Outputs External Customer Figure 2: A s Perspective on Agile SDPMs Figure 2 shows that the Internal Controller and the Internal Customer are integral parts of the Software Development, and as a result they have direct access to the state of the Software Development. These two systems are therefore able to exert control over the Software Development based on its state, and according to Weinberg (1992) this enables faster and more accurate control than in the situation given in Figure 1. In the case of XP coaching and pair programming constitute an Internal Controller. XP's on-site customer obviously constitutes an Internal Customer. The Internal Controller and the Internal Customer do not necessarily rely on externally defined targets for their operation only, instead they may define control policies of their own. This corresponds to Cockburn (2001) who regards the gradual refinement of the software development process by the team members during their work as a distinguishing feature of agile SDPMs. We conclude that agile SDPMs have the potential for self-maintenance. For example, they may modify or enhance their own organisation, and therefore, they can evolve during their own operation to change their complexity, structure, or behaviour. To accomplish this agile SDPMs may spend resources on their own maintenance. This characterisation of agile SDPMs corresponds exactly to the characterisation of open systems given by Scott (1998), who notes that open systems are capable of self-maintenance, thereby exceeding the capabilities of classical cybernetic systems.
7.0 Conclusion General systems theory can be used as an integrating theoretical framework for the analysis of SDPMs. On this basis we have proposed a clear and objective characterisation of agile SDPMs. SDPMs that are generally not regarded as agile can usually be described as cybernetic systems. They rely on separate control systems that do not have access to the state of the software development system. Therefore, control is based on inputs and outputs of the software development system only. This partial view can possibly prevent effective control of the software development process. Agile SDPMs integrate some control systems into the software development system, and therefore these control systems have access to the state of the software development system. This can enable faster and more accurate control of the software development process. In agile SDPMs control mechanisms do not necessarily rely on targets defined outside the software development system only. Instead, control policies may emerge within the software development system itself. This enables the gradual refinement of the software process during its execution. This capability of self-maintenance is characteristic of agile SDPMs and shows that they correspond to an open-systems view of the software development process. The fundamental distinction in general systems theory between cybernetic systems and open systems can therefore be used to distinguish agile SDPMs. SPDMs that can appropriately be modelled as cybernetic systems are generally not agile. Agile SDPMs, on the other hand, are fundamentally based on an opensystems perspective on the software development process. In this respect the emergence of agile SDPMs indicates the inception of a new paradigm in software engineering management. References Beck, K. (2000) "Extreme Programming Explained", Addison-Wesley. Brooks, F. P. (1995) "The Mythical Man-Month", Addison Wesley Longman. Cockburn, A. (2001) "Agile Software Development", Addison-Wesley. DeMarco, T; Lister, T. (1999) "Peopleware: Productive Projects and Teams", Dorset House Publishing. Gharajedaghi, J. (1999) "s Thinking: Managing Chaos and Complexity", Butterworth-Heinemann. IEEE [Institute of Electrical and Electronics Engineers, USA] (2000): Dynabook on Extreme Programming. http://computer.org/seweb/dynabook/index.htm (Copyright IEEE 2000, last visited on 02/01/2002). Jacobson, I.; Booch, G.; Rumbaugh, J. (1999) "The Unified Software Development Process", Addison Wesley Longman. Pressman, R.S. (1997) "Software Engineering: A Practitioner's Approach", McGraw-Hill. Scott, W.R. (1998) "Organisations: Rational, Natural, and Open s", Prentice-Hall. Weinberg, G.M. (1992) "Quality Software Management (Volume 1): s Thinking", Dorset House Publishing. Weinberg, G.M. (1998) "The Psychology of Computer Programming", Dorset House Publishing. Weinberg, G.M. (2001) "An Introduction to General s Thinking", Dorset House Publishing.