Improving Performance with Enhanced Measures and Causal Analysis in an Agile Organization

Size: px

Start display at page:

Download "Improving Performance with Enhanced Measures and Causal Analysis in an Agile Organization"

Cynthia Miles
6 years ago
Views:

1 Aligned with your needs. Improving Performance with Enhanced Measures and Causal Analysis in an Agile Organization Paul E. McMahon CMMI/Agile Consultant, PEM Systems Dr. Bela Joshi Sr. Principal, Alion Science and Technology

2 Slide 2 Topics Overview Background Motivation for 2013 Improvement Effort How Improvement Effort was Conducted Defect Analysis Results Achieved Validation Implementation of Improvements Plan Forward/Questions

3 Slide 3 Overview The Advanced Modeling and Simulation Technology Operation (AMSTO) of Alion Science and Technology is a CMMI-DEV Maturity Level 3 organization In 2013 AMSTO undertook an initiative to improve performance through enhanced measures and causal analysis In this presentation we discuss the analysis conducted and the resulting improvements

process journey began in 2003 CMMI-DEV Maturity Level (ML) 3 in 2007 Maintained ML 3 with

4 Slide 4 Background AMSTO provides technical solutions to the Department of Defense mainly in modeling and simulation arena Majority software projects use a tailored version of Scrum AMSTO s process journey began in 2003 CMMI-DEV Maturity Level (ML) 3 in 2007 Maintained ML 3 with re-appraisals in 2010 and 2013 Lightweight processes, minimal essential must-dos supporting agile approach

re-appraisal CMMI/Agile consultant encouraged maturing Measurement and Analysis (MA)

5 Slide 5 Motivation for 2013 Improvement Effort AMSTO successful with happy customers providing repeat business Committed to CMMI ML 4/5 since 2010 In preparation for 2013 re-appraisal CMMI/Agile consultant encouraged maturing Measurement and Analysis (MA) process area While MA was institutionalized, significant potential benefits believed to exist

estimation improvements Defect analysis improvements Multiple improvements

6 Slide 6 How the Improvement Effort was Conducted Initial brainstorming (EPG, Project personnel, consultant) considered: Enhancing retrospectives Task estimation improvements Defect analysis improvements Multiple improvements ensued This report focuses on defect analysis and improvements focused on reducing escapes

7 Slide 7 Defect Analysis Defect analysis performed on successful on-going project with well established processes and mix of personnel experience level Software product undergoes joint verification event with government customer before release in end-user environment

Slide 8 Defect Analysis (continued) Step 1: Extract

developers Goal: Determine root cause using 5 Why

root cause Step 3: Use Pareto principal to apply

step for management buy-in Initial results: Pareto

8 Slide 8 Defect Analysis (continued) Step 1: Extract key information about each escape by interviewing developers Goal: Determine root cause using 5 Why Technique Step 2: Categorize escapes according to root cause Step 3: Use Pareto principal to apply improvements to category with most escapes Crucial step for management buy-in Initial results: Pareto principal did not hold Close analysis: Requirements most often underlying cause

9 Slide 9 Defect Analysis (continued) Step 4: Search for assignable causes, and ask developers for suggestions to keep escapes from occurring in future Step 5: Report result to Senior Management with time effort took Crucial to selling approach to management as part of AMSTO MA process

10 Slide 10 Data Analysis Defects count by source prior to 5 Why analysis Defects count by source after 5 Why analysis

11 Slide 11 Results Achieved Two most common patterns: Inadequate requirements Inadequate regression testing Root cause: Weakness in Requirements Development process Poor requirements lead to inferior product and inferior tests

12 Slide 12 Validation Needed to validate findings before making significant changes to process that had worked successfully for many years Compared study results with observations from periodic gap analyses and training workshops conducted by CMMI/Agile consultant AMSTO teams frequently work on cutting edge solutions in research and development environment Incomplete, ambiguous, vague requirements often surfaced One reason why iterative agile/scrum cycle worked well

13 Slide 13 Validation (cont.) While good collaborative relationship with customers existed, due to busy schedules and priorities, customers not always available for clarifications Thus project teams often had to make assumptions when producing test scripts The results were thus validated in this manner

testable requirements Improved requirements development guidelines were developed Included

14 Slide 14 Implementation of Improvements No assignable causes or quick fixes Practitioners would need to be trained and mentored on how to develop complete, unambiguous, and testable requirements Improved requirements development guidelines were developed Included concrete and practical methods to develop requirements under the common conditions teams often faced

to continue monitoring escapes to determine if changes

15 Slide 15 Plan Forward Improved requirements development guidelines being deployed and trained Plan to continue monitoring escapes to determine if changes lead to measurable improvements, or if further improvements needed

16 Slide 16 Questions and contact information Paul E. McMahon Ph: Web: Dr. Bela Joshi Ph: Web:

IT4305: Rapid Software Development Part 2: Structured Question Paper

IT4305: Rapid Software Development Part 2: Structured Question Paper UNIVERSITY OF COLOMBO, SRI LANKA UNIVERSITY OF COLOMBO SCHOOL OF COMPUTING DEGREE OF BACHELOR OF INFORMATION TECHNOLOGY (EXTERNAL) Academic Year 2014/2015 2 nd Year Examination Semester 4 IT4305: Rapid