Frameworks and Approaches for Working with Complex Systems: Views from Other Endeavors

Transcription

1 Complex Aerospace Systems Exchange 2012: New Paradigms for Complex Systems Development Frameworks and Approaches for Working with Complex Systems: Views from Other Endeavors Dr. Jimmie McEver Senior Scientist September 12, 2012

2 Motivation: The Evolving World Complex, volatile and uncertain environments OR EVEN Increasing systems complexity, scale and performance/funct ionality Today 2

3 Complexity in Systems and Behavior Many entities, many interactions, collective behavior Qualitative and quantitative aspects Scale, heterogeneity of entities and interdependencies, and their nature Definition or characteristics Emergence, self-organization, self-similarity, chaos, etc. Complexity may refer to systems or systems behaviors Simple systems can exhibit complex behavior Complex systems can exhibit simple behavior Various complexity measures Computational complexity (of a problem) Algorithmic information content (of a string) Structural complexity Dynamic complexity Sources of complexity The system itself The environment The people/organizations acquiring, developing and using the system 3 Adapted from: Michael Bell, Implications of Complexity Research for C2, Briefing for CCRP Focus and Convergence Team, July 29, 2009.

4 Interdependencies are one reason we are in a different problem domain 4 Network size Configurations Number of options 2 nodes 4 3 nodes et al 64 4 nodes nodes nodes For reference: estimated number of atoms in the universe on the order of Complex systems: more interactions, and more of the interaction options space is relevant to our development activities But there are limits to our ability to explicitly deal with that space So what do we do?

5 Hallmarks of Complexity and Impacts on Decision Making How complexity makes it difficult UNCLASSIFIED Hallmarks of complexity Interdependence Nonlinearities Open boundaries Multi-scalarity Causal & influence networks Emergence Complex goals Adaptation & innovation Opaqueness Impact on Decision Maker Cannot treat by decomposition Extrapolation of current conditions error Cannot focus only on processes inside boundary Have to address all relevant scales Challenge: develop requisite conceptual model within time and information resource constraints Unknown risks and unrecognised opportunities Goals may change, be unrealistic, vague Rules change, interventions stimulate adaptation Many possible hypotheses about causal paths, insufficient evidence to discriminate Adapted from Grisogono, Anne-Marie and Vanja Radenovic, The Adaptive Stance Steps towards Teaching more Effective Complex Decision-Making, International Conference on Complex Systems, June UNCLASSIFIED

6 Ten Attributes of a Wicked Problem Horst, Webber: Dilemmas in a General Theory of Planning Policy Sciences, 1973 There is no definitive formulation of a wicked problem Wicked problems have no stopping rule Solutions to wicked problems are not true-or-false, but good-or-bad There is no immediate and no ultimate test of a solution to a wicked problem Every solution to a wicked problem is a "one-shot operation"; because there is no opportunity to learn by trialand-error, every attempt counts significantly Wicked problems do not have an enumerable (or an exhaustively describable) set of potential solutions, nor is there a well-described set of permissible operations that may be incorporated into the plan Every wicked problem is essentially unique Every wicked problem can be considered to be a symptom of another problem The existence of a discrepancy representing a wicked problem can be explained in numerous ways. The choice of explanation determines the nature of the problem's resolution The planner has no right to be wrong

7 Complex Decision Making Emphasizes How Decisions are Made Assessment of Decision Quality Varies with Complexity For Simple Decisions Determine quality by comparing the recorded decisions with the logically or doctrinally correct decision Complicated Decisions Determine quality by evaluating the degree to which the decision makers considered trade-offs and priorities Adapted from Network Centric Operations Conceptual Framework, Office of Force Transformation Applied to metrics/assessment approach development for 2009 Decision Ranges Study for Singaporean Armed Forces by Evidence Based Research, Inc. Complex Decisions Quality determined by evaluating the degree to which the decision makers had an understanding of the relevant factors involved, but acknowledged: The impossibility of predicting the consequences of an action, The need to take actions to generate more time and information The need to retrain reserve assets for use as alternatives 7

8 Selected Complex Systems Efforts Network Centric Operations, Network-Enabled Capability Command and Control Maturity Model Approaches to deal with complexity, volatility and uncertainty in C2 A value chain and metrics for how information and collaboration enhance the agility of individual and collective decision making processes What it takes to be more agile, to be more capable of dealing with complexity, volatility and uncertainty TTCP Action Group 14: Complex Adaptive Systems for Defence International effort to explore implications of emerging insights from complex adaptive systems research for national security Topics addressed included complex causality, approaches to enhancing adaptiveness/agility, and the considerations for SE of complex systems DoD Cyberspace Operations Capability Development/Acquisition NDAA 2011 called on DoD to develop a strategy for responsive and accountable acquisition of cyber warfare capability, addressing requirements, acquisition, and test/evaluation aspects of capability generation Identified principles, tenets, and cross-cutting enablers 8

9 The C2 Problem Context: Complex Endeavors 9 Source: Hayes, Richard E., It s an Endeavor, Not a Force, International C2 Journal, Vol. 1, No. 1 (2007). Complex endeavors* reflect nature of many current and emerging operations Have a purpose or set of related purposes Large number of disparate entities whose activities are related to a broad range of effects No single leader or commander Individual participants may be working toward different purposes No subset of participants is capable of achieving its relevant goals absent contributions of others Participants may have a variety of relationships with one another *Complex endeavors are introduced in Hayes and Alberts, Planning: Complex Endeavors, CCRP Publication Series, 2007.

10 The Network Centric Value Chain Information Domain Quality of Information Awareness Cognitive Domain Understanding Intent Physical Domain Agility Robustly Networked Force Information Sharing Shared Awareness Shared Understanding Shared Intent Decision Quality Adapted from Network Centric Operations Conceptual Framework, Office of Force Transformation Collaboration Synchronization Mission Effectiveness Social Domain 10

11 Toward Agile C2 in Complex Endeavors: NATO NEC C2 Maturity Levels C2 Objectives C2 Maturity Implications ID of additional C2 approach options Ability to ID and implement appropriate approach given the situation Establishment of shared intent Configure/reconfigure roles Rich sharing of non-organic resources Some pooling of organic resources Mutual support for individual intent Links among plans to enhance effects Initial pooling of non-organic resources Agile C2 Collaborative C2 Coordinated C2 High shared understanding of common intent Rich continuous interactions Required for situations with high dynamics, uncertainty, complexity Sharing of resources, interdependence More information sharing/interactions Planning in parallel Effectiveness >> sum of parts Actions may reinforce other actions Planning time may increase Effectiveness > sum of parts Avoidance of adverse cross-impacts. De-conflicted C2 Willingness to accept constraints Limited information interactions Effectiveness approaches sum of parts None. Only individual objectives present. Conflicted C2 No collective C2 No avoidance of negative cross-impacts Effectiveness < sum of parts 11

12 So What Do We Need to be Able to Do? C2 Approach Allocation of Decision Rights to the Collective Inter-Entity Information Sharing Behaviors Distribution of Information (Entity Information Positions) Edge C2 Not Explicit, Self- Allocated (Emergent, Tailored, and Dynamic) Unlimited Sharing as Required All Available and Relevant Information Accessible Collaborative C2 Collaborative Process and Shared Plan Significant Broad Sharing Additional Information Across Collaborative Areas/Functions Coordinated C2 Coordination Process and Linked Plans Limited Focused Sharing Additional Information About Coordinated Areas/Functions De-Conflicted C2 Establish Constraints Very Limited Sharply Focused Sharing Additional Information About Constraints and Seams Conflicted C2 None No Sharing of Information Organic Information 12

13 TTCP AG14 Overview In 2007, the Technical Cooperation Programme (TTCP) stood up an Action Group to explore applications of complex adaptive systems science for defense AG14 Lead: Dr. Anne-Marie Grisogono, DSTO, Australia Objective: Explore the implications of complexity theory in particular the science of complex adaptive systems (CAS), for major related defence challenges, and translate those implications into practical applications The scope of the scientific issues which this Action Group addressed included: Fundamental theory and processes, especially adaptation Causality and influence-ability within complex systems; Design and management principles for complex systems; and Approaches and tools for applying complex systems science insights and concepts to real world problems in defence and security One major AG14 thrust: Complex Systems Engineering 13

14 Complex Systems Engineering Increasingly, complexity has stretched or reached classical systems engineering process limits Results: cost escalation, schedule overruns, performance issues Even greater challenge: Cross-project integration to create systems-of-systems Capable of emergent behavior Subject to co-adaptation with environment Need to get beyond traditional systems engineering techniques, which depend upon: Desired outcomes and problem definitions known a priori Single manager able to make decisions about resource allocation Change introduced and managed centrally Fungible resources that can be applied and reallocated as needed BUT: Systems we are building are complex They operate in complex, volatile and uncertain environments and SoS contexts Our engineering endeavors are complex and dynamic 14

15 Adaptation Improves fit of system to its environment General Model of Adaptation What is needed to be adaptive: To retain successful, & discard unsuccessful, variation need a. Ways to judge success-value of variations b. Ways to encode information (successful outcomes) into the system In addition adaptation requires the ability to c. Produce potentially useful variation d. Produce success-relevant feedback e. Select and implement a successful variation f. Continuously iterate Because it is not obvious how to be successful, and The situation and environment will keep changing Adaptation may be triggered (reactive), anticipatory (pro-active), or neither UNCLASSIFIED Six key aspects of adaptation =targets for improvement UNCLASSIFIED Valuebased Incremental Grounded in reality Cyclic Conceptual Framework for Adaptation identifies the space of possibilities for being adaptive 5 classes: continuous improvement of functions + resilience, responsiveness, flexibility, agility 5 levels adapting: action, capability, adaptivity, proxies and relationships (co-adaptation) At multiple scales From Grisogono & Radenovic, ICCS 2011 with permission.

16 Levels of Adaptation Adaptive action: changing the use of existing sensing, decision and action capabilities Learning: changing the sensing, decision and action capabilities themselves Learning to learn: changing the system s adaptive processes (how changes in action and learning are generated and adopted) Defining success: changing the internalized proxies for goodness or success used in adaptive processes Co-adaptation: determining how the system allocates resources, responsibilities and decision rights to its components and allied systems 16

17 Key Concepts for CxSE (1/2) Understand the fundamental unique value and possible future needs given a changing environment Utility and fitness can only be measured through use and interaction with the environment Actual future needs may not be anticipated a priori but become clearer when we ask why does the system add value Acknowledge and understand limits of decomposability Parts may be understandable but composition may lead to emergent and novel behaviors both desirable and undesirable Focus on creating a development environment that includes adaptive processes up front as part of system design strategies to enable future adaptation of products Consider design patterns and a pattern language to help evolve systems design Ensure most important system elements are composable with clear and accessible interfaces 17

18 Key Concepts for CxSE (2/2) Recognize that adaptation can occur at different levels, considering and requiring various degrees of freedom in the design space The platform concept can provide tools for creative use and re-use of systems and systems elements depending on context Enables architecting aspects of the system without a centrallycontrolling architect Robustness, resilience and more broadly adaptivity are strategic concepts that should be a significant consideration in systems design These attributes often come at the expense of local efficiency and performance Employ evolutionary principles for enterprise systems engineering in a deliberate way 18

19 NDAA Section 933: Cyber Capability Acquisition Strategy In Jan 2011, Congress directed SECDEF to develop a strategy/approach for the acquisition/development of DoD cyber warfighting capability Improve responsiveness, efficiency, rigor Study team identified approaches to requirements, acquisition/development, and test & evaluation aimed at enhancing the Department s ability to respond to the particular demands of the cyber domain Also focused on questions of governance and oversight how to enable agility while maintaining accountability 19

20 Key Elements of Cyber Capability Acquisition Strategy Governance Oversight must allow for more timely and less formal approaches Governance roles emphasize integration, alignment, and collaboration across communities and processes Requirements Greater flexibility, faster responses from requirements validation process Flexibility to address both rapidly emerging and longer-term needs Acquisition Alternative acquisition processes, including oversight, tailored to the complexity, cost, urgency of need and fielding timeline Management/authority delegated as appropriate Test and Evaluation Ability to evaluate operational performance and risk is key Need to ID and address risk collaboratively throughout process T&E infrastructure and capabilities will be needed 20