Introduction "We cannot solve our problems with the same thinking we used when we created them." - Albert Einstein Who This Book Is For This book is designed for anyone attempting to gain insights into how to achieve Agility at scale. If you are one of the following, you will find this book of value: We re just considering beginning an Agile transition and are concerned about how to use it beyond the team We ve not had much success with Agile, even at the team level, either because we can t figure out how to create the cross-functional teams that Scrum calls for or we can t get our business stakeholders to limit the amount of work they are giving the teams We ve been using Agile successfully at various teams, but now aren t sure how to take it to the enterprise We ve been using Agile successfully but are now struggling in an Agile-wide adoption Since Scrum is the most popular Agile method being followed today, we write this book assuming the reader is familiar with Scrum and has been using that as their basic method. Those already using Kanban or Lean will find this book of great value as we ll go beyond principles and discuss practices we ve seen that can assist multiple teams, whether using Scrum and/or Kanban, to work together. Net Objectives has worked with hundreds of companies since our inception in 1999. We started out with technical training but within a year or two moved into Agile methods because it was clear technical practices were not always the problems teams were facing. We first started helping teams with extreme Programming (XP) and Scrum. When we saw challenges in scaling these methods we turned to Lean. When Kanban came on the scene we expanded our offerings to include that so that we could help even more organizations where Scrum seemed difficult or ineffective. In many ways we ve come full circle because much of our work now is helping companies use Scrum to get off the plateau they ve found themselves on. And, of course, we re still helping people with better design and programming methods. This book addresses many of the challenges we ve seen most companies starting with Scrum encounter when they attempt to go beyond a few teams. It also will provide insights into how to start at a larger scale. To us, Scrum is a good team framework that many people have attempted to take beyond its sweet spot. If you are a fan of Scrum, please don t take this a negative statement about it. Scrum is great to take an organization to a certain level, but if you want to take it throughout the enterprise, more is needed. That s what this book is about. Copyright, 2012, Net Objectives 1
The Common Approach to Scaling Agile and the Problems Encountered Most companies begin their transition to Agile by starting with a pilot project. Pilots are good, and have a track record of success. Unfortunately, many times the pilots are selected on the basis of making the pilot work without looking at what needs to be learned to make the transition to Agile be successful. These are often not the same. We ve seen many teams having initial success with Agile often describe their experience as one of the following: The first few Scrum pilots were successful, but now they can t extend it to other teams (see the Chapter on How Successful Pilots Can Actually Hurt an Organization ) They had initial success with Scrum but their teams have stopped improving and are even sliding back Their first few teams wanted to become Agile but they are now finding resistance from other teams to go to Agile Their pilot was successful but those not involved in it are claiming Agile has hurt them. Their teams individually work well but they can t release quickly when a project requires the collaboration of multiple teams (see the chapter on Coordinating Backlogs to Coordinate Teams ) Your teams can complete code every sprint, but it still seems as if they can t release it on a regular cadence (see chapter on Coordinating Backlogs to Coordinate Teams Your business stakeholders have to talk to several product owners to see what is happening to their projects and your product owners have to do project management as well (see chapter on Managing Work Flow from Multiple Business Stakeholders to Multiple Teams ) Do any of these sound familiar? If so, you will find this book of great value. If not, please get in touch with us as we d like to know what you are doing! (seriously). You ll discover that each of these challenges stems from either the company not taking an holistic approach in their transition to Agile methods or due to a lack of understanding the basic principles of product development flow. One of the intents of this book is to make the reason that these patterns exist. By the end of this book you ll both understand why these patterns occur and what to do about them. Let s describe each of these in a little more detail. The first few Scrum pilots were successful, but now they can t extend it to other teams. This happens for two reasons. The first, of course, is that Scrum works. If an organization already has teams in place, getting them to develop software in iterations with Scrum will improve their productivity. If crossfunctional teams do not already exist, creating them has two great advantages, even before any Agile methods are used. These are that the people on the teams can focus on working on one project at a time and have the people they need available when they need them. We have found that projects developed by a cross-functional team working only on the one project will out-perform people working on multiple projects, coming and going as available, by a factor of 3 to 8 times. Copyright, 2012, Net Objectives 2
The challenge is that while one or two such teams can be created, if key people are needed across the organization or if too many projects are being initiated, at some point it will no longer be possible to keep making cross-functional teams. The real challenge of managing key people across projects and limiting the number of projects in play has not been addressed by the pilots. They had initial success with Scrum but their teams have stopped improving and are even sliding back. There are many reasons for this. Mostly we ve seen it occur for a combination of the following: 1) the teams have made enough progress that they are no longer in the pain that motivated their transition to Scrum 2) they are getting bogged down by using less efficient methods than are possible 3) they have improved their teams but haven t figured out how to get the teams to work together better We often find that teams that retrospections for teams having done Scrum for a while have gotten to be pretty mundane and not very useful. The serious pain they had been experiencing that got them into Scrum is gone and at the end of a sprint they just want to get on to the next one. We also see that they haven t solved their problems but have become cynical about doing so because Scrum hasn t provided much guidance on how to solve the impediments they are currently facing. Many of these can only be solved by their management. While Scrum doesn t tell you not to collaborate well with management, it ignores management to such a great extent that it s hard to fault Scrum teams for not knowing how to interact with them when they need help. I ll talk more about how Lean provides insights here. While Scrum is not intended to be a prescriptive framework, because it starts out with a prescriptive structure, roles and sets of practices, we find many teams tend to stick with other practices their Scrum coach/trainer has presented to them without looking for alternatives. In particular, there are many estimation methods out there but we see Mike Cohn s planning poker as the most commonly used one not coincidentally because he is such a strong fixture in the Scrum Alliance which is the largest Scrum certification body in the world. Unfortunately, his estimation method is the most time-intensive of the ones out there and this causes a significant amount of waste. This is, unfortunately, only one example of what happens when teams start their road to Agile down a prescribed path they don t know when to start thinking for themselves. Another reason for this stagnation is that intra-team collaboration (that is, team members working together) is quite different from inter-team collaboration. Teams are often motivated by and/or measured differently. This can make cross-team collaboration a challenge. While some teams figure out how to solve this problem, we ve found that the principles of Lean-flow to be extremely useful in helping figure out how teams can better work together. Their first few teams wanted to become Agile but they are now finding resistance from other teams to go to Agile. Geoffrey Moore s crossing the chasm is often quoted and almost as often misunderstood. You often hear talk about companies as being early adopters and that those doing Agile the last few years are early adopters. However, it is actually people that are early or late adopters. What often happens in a company is that the early adopters decide to go Agile, get success and then have Copyright, 2012, Net Objectives 3
management try to spread it to the others. Unfortunately, these others are often not early adopters but are treated as if they were. This has a couple of problems. First, the people now doing Agile aren t necessarily comfortable with it. Second, Agile cannot really be imposed on teams very well. If one tries to impose a method that does not have an explicit model about why it works (e.g., Scrum) this will be particularly difficult. Their pilot was successful but those not involved in it are claiming Agile has hurt them. We came across a company that had a very successful pilot project by putting their best people together to work on one project. Not surprisingly, it was a big success. Unfortunately, some of these people had deep domain and/or legacy code knowledge or were just plan smart. By putting them on the pilot projects they were less available to other projects, thereby hurting these projects. It wasn t jealousy that had them complain about the pilots, it was setting up those on the pilots in a way that hurt the non-pilot teams that upset them. Their teams individually work well but they can t release quickly when a project requires the collaboration of multiple teams. Getting a cross-functional team to work together well is not that difficult. Unfortunately, getting several teams with different motivations to work together is another issue altogether. Releasing cross-team products on a regular basis requires a different skill set than just building stories at a team level. If any of the cases seem familiar to you in your organization this book will definitely be of help to you. Many in the Agile space are convinced that a bottom-up only, team centric approach can solve the impediments individual teams face. We have seen that sometimes organizations solve these. But more often they have little idea how to do so. In our own consulting practices we ve often gone into organizations and in a manner of days, provided them key insights based on the theories of flow based development that helped them get past the challenges that had stymied them. We ve come to the conclusion that there are many dynamics at play in an organization that is moving to more effective Agile methods and that merely learning team frameworks leaves on at risk of not having access to the insights they will need to continue learning and moving forward. This book will provide many insights and case studies that illustrate how to take an Agile endeavor to the next level. The Challenge of Scaling There is a dilemma organizations face. They have been told the essence of Agile is self-organizing crossfunctional teams, that management s role is to remove impediments to these teams. However, it now appears that, while these teams may be able to organize at the team level, they are having severe challenges organizing across the teams at the enterprise level. The methods that worked well to this point seem to fall short now. Why this happens should not really be a big surprise. Look at figure 1, 2 and 3. Copyright, 2012, Net Objectives 4
Figure 1: The Scrum Model for one team (SM= ScrumMaster) Figure 2: The Scrum Model with multiple teams. Copyright, 2012, Net Objectives 5
Figure 3:The Scrum model in large organizations. The model that works well in figure 1 is clearly stretched by the time an organization tries to apply it as illustrated in Figure 3. The attempt to do this is something called Scrum-of-Scrums. Unfortunately, Scrum-of-Scrums is still usually espoused as a peer-to-peer method, and because of that, does not work well except in the most culturally functional organizations. Even there, however, it is often a challenge. There are two reasons for this. First, the intra-team (within a team) dynamics of a Scrum team are considerably different than the inter-team (between teams) dynamics. People are much more likely to identify and work with the people they work with on a daily basis (their own team members) than they are to work with other teams. Particularly when monetary evaluations and incentives have them focus on their own team. Even then, some higher level view is needed. The only times we ve seen Scrum-of- Scrums work to coordinate multiple teams is when they have been comprised of the product owners for the teams where the product owners could create a shared vision to give all of the teams. Even here, however, how this team of Product Owners gives the work to the team must be different than the normal Sprint backlog (more on this in case study 3 Using Shared Backlogs. One way to see why the Scrum-of-Scrums, bottom-up approach is flawed is to look at how well (or rather poorly) it would work for forest fighting teams. When teams go in to fight a fire, they do not decide what to do based on their own, local, situation. Rather, there is a commander that gives them objectives based on where they are. Each team will, of course, figure out how to best handle their objective, but the overall goal, or context within which they work, is set by someone who is focused on the bigger picture. If the team members got together to figure out what to do it is likely people would die. A New Approach Is Needed Scrum is a framework that has cross-functional teams self-organize and deliver software incrementally. There are many, good, power concepts here. However, these are all based around a close-knit team working on a reasonably defined problem adding specific value to a known customer. Software development at the enterprise level is both much more complex and much less defined. To solve this more complex problems new understanding is needed in the areas of organizational people dynamics and how to coordinate work across teams and how to create. Furthermore, a more holistic view of the problem other than a bottom-up team driven approach is needed. It is not Scrum s fault it will often not scale any more than it is the fault of a hammer that it won t turn a screw well. Scrum, used properly, can be a very good team solution. But if you are trying to get to agility at scale, a larger framework to work from in order to avoid the risk of your Agile efforts stagnating is almost always useful. We must acknowledge the limitations of our good tools in order to find better ones for different situations. This book explains both what is needed for implementing Agile at scale and how one can take advantage of any effort they ve already expended in achieving Agile teams. The only thing that really needs to be abandoned is a limiting team-centric viewpoint. Copyright, 2012, Net Objectives 6
In the process, we must understand why achieving Agile at the team level, is different than across the organization. In fact, we must ensure we understand what true agility at the team level is. Many companies have greatly improved their software development methods over waterfall, but are still not truly Agile at the team. This improvement masks additional value to be achieved. We must truly understand team-agile before trying to implement enterprise Agility. We have found that three things are needed in order to achieve Agility at scale: 1) Enterprise solutions need an enterprise view one that a team-based framework does not provide this can be provided by systems thinking 2) Enterprise solutions need a management component beyond merely removing impediments for teams this can be provided by lean-management 3) A deeper understanding of how to produce value at scale is required to create a vision for multiple teams to work together this can be provided by lean-flow This book is therefore about how to use systems thinking, lean-management and the theory of lean-flow to create a mindset that will enable an organization to achieve enterprise agility with teams collaborating together effectively. This information will be provided through a series of five case studies so that we can provide proven methods with the theory that backs it up. These case studies will present cases where: 1) The ideal Scrum team can be formed where truly cross-functional teams exist 2) Stable feature teams can be created that can repeatedly develop features independently from each other 3) Dynamic feature teams can created so that features can be built and deployed quickly 4) Component teams can use shared backlogs so that they can work together effectively without the thrashing that often occurs during integration 5) Multiple stakeholders need features built by multiple development teams The Pre-requisites for This Book This book assumes a basic understanding of Lean-Agile Methods. This background is provided in the Essentials for the Lean-Agile Roadmap: Fundamentals of Lean-Agile Software Development. One doesn t need to read that entire book but should read/watch at least one entry from each of the following chapters: Introduction to Lean-Agile Methods The Business Case for Agility Shifting our View from Utilization to Throughput Copyright, 2012, Net Objectives 7