Economic evaluation of complex intervention using simulation modelling techniques

Transcription

1 Leeds Institute of Health Sciences Economic evaluation of complex intervention using simulation modelling techniques Armando Vargas-Palacios Research Fellow Academic Unit of Health Economics 30 th April 2015

2 Outline Brief introduction to decision modelling Simple vs complex systems for health care interventions Overview of simulation methods Traditional methods vs simulation methods System dynamics (SD) Agent based models (ABM) Discrete event simulation (DES) Why dynamic approaches are relevant? SIMULATE checklist

3 Decision analysis The main purpose of economic evaluation is to inform decisions about alternative uses of scarce resources Decision analysis specifically looks at decision making under uncertainty, using theories of probability and of expected utility Usually we construct a model to represent the clinical pathway/health intervention

4 Decision analysis Economic evaluation is more commonly used to determine the economic value (cost-effectiveness, cost-utility) of two or more treatment strategies Treatment with Drug A vs Drug B: which is cost-effective in the treatment of a particular disease? (costs per-qaly gained or cost per life years gained)

5 Modelling A good health economic model should: Be populated with the most appropriate and good quality clinical data (e.g. from meta-analyses) Reflect a realistic picture of: current clinical practice, health care intervention, health care systems, etc Use the appropriate comparator(s) Be run for an appropriate time period Be valid, transparent and reproducible Explore uncertainty Be easily interpreted

6 Modelling A good health economic model should: Be populated with the most appropriate and good quality clinical data (e.g. from meta-analyses) Reflect a realistic picture: current clinical practice, health care intervention, health care systems Use the appropriate comparator(s) Be run for an appropriate time period Be valid, transparent and reproducible Explore uncertainty Be easily interpreted

7 Methodologies used in decision modelling There are many types of models to choose from Different ways to classify them: Cohort/aggregate level or Individual level; Measure of time: untimed, timed, discrete time or continuous time Static or dynamic; Deterministic or stochastic; Most common are those based on a cohort/aggregate level models, untimed/timed, static and deterministic: Decision trees (DTM), Markov models (MM), Hybrids (combination of both)

8 Decision modelling In general terms when evaluating different interventions (treatments options, drugs, medical devices, etc.) traditional methods such as DTM and/or MM are enough to determine the cost-effectiveness of an intervention. Usually because they follow a linear process (clinical pathway) There are however interventions that require more complex approaches as the process is not linear: infections diseases: transmission is a function of the total number of infectious and susceptible individuals (dynamic process) How about health care delivery systems or health care interventions in complex health care systems?

9 Complex vs simple systems* Health care systems: inherently complex and fragmented social systems consisting of governments, payers, and multiple providers responsible for delivering health care services Social systems are different from other systems as people make decisions, interact among themselves, and also interact with other parts of the system in an interdependent nature. Health care delivery systems are dynamic by nature (interaction between different actors are adaptive; decisions, choices, alternatives, availability change over time)

10 Complex vs simple systems* A complex system: is adaptive to changes in its local environment, is composed of other complex systems, behaves in a nonlinear fashion, the behaviour of the system as a whole is different from that of its parts or components Simple systems consist of tasks that can be answered as yes or no, while complicated systems consist of tasks that are based on if-then. *Marshall DA, Burgos-Liz L, IJzerman MJ, et al. Applying dynamic simulation modeling methods in health care delivery research The SIMULATE checklist: Report of the ISPOR Simulation Modeling Emerging Good Practices Task Force. Value Health 2015;18:5-16.

11 Overview of simulation methods Simulation models are intrinsically dynamic as they allow interaction between individuals or other actors involved in the intervention. These interactions can determine future pathways or processes. Originally from operational research field, simulation models are used to design and develop mathematical representations of operational process and systems Test interventions, scenarios, consequences over time

12 Overview of simulation methods These types of models are: Cohort/aggregate level or Individual level Timed: discrete time or continuous time Dynamic; Deterministic or stochastic; Three methods can be highlighted: System dynamics (compartmental models) Discrete event simulations Agent-based model

13 Traditional vs simulation models Traditional methods (DTM and MM) Simulation Models DTM Capture what happens at a point in time: i.e. time is not explicitly measured MM Time measured in cycles (days, months, years) Continuous time; Discrete time Recursion or looping is not allowed Recursion or looping allowed Recursion or looping allowed Memory less assumptions No interaction between individuals or actors Memory less assumptions No interaction between individuals or actors Patient history, background information, previous health events Interaction between individuals or different actors allowed

14 Simulation models System dynamics Aggregate level model, Continuous time Deterministic Discrete event simulation Individual level model Continuous time but examine and updates when an event occur Stochastic Agent-based model Similar characteristics of a DES, but individuals are autonomous

15 System dynamics Core elements: feedback, stocks, rates, time delays Stocks: accumulation of something (individuals, hospital beds, etc) Rates are flows (determine the rate at which individuals move through the system i.e. rate of transmission in the context of an infectious disease) Work with compartments, each compartment holds a subpopulation of individuals with a particular characteristic and transit to other compartments at a rate Constructed using differential, ordinal differential or partial differential equations Software allow visual representation of the flowchart and aid in the construction of the differential equation model Berkeley Madonna ($80-$100 licence)

16 Discrete event simulation Is a continuous time model that employs a next event technique (only examines and updates when a change of state occurs) The core concepts in DES are events, entities, attributes, and resources Main elements are the individual entities : components being simulated and tracked. Entities can be a representation of individuals, patients, resources etc. Attributes can be assigned to each entity, via labels (to differentiate entities at the individual level) Based on the value of the attributes, entities transit through different activities that alter their characteristics and influence future events. Depending on the progressive order and conditions of these activities, the entities can be held in queues until it is time to engage in another activity or expire Software: SIMUL8 (Cost $1,900-$5,000 USD)

17 Agent-based models There are autonomous and interacting objects called agents. Agents interact with others within an environment Their next actions are based on the current state of the environment as they sense its environment and behaves accordingly on the basis of defined decision rules. Agents may have explicit goals to maximize or minimize and may learn and adapt themselves on the basis of experience Free software: NetLogo

18 Why dynamic approaches are relevant? Health care delivery systems are inherently complex, characterized by nonlinearities, feedback loops, and a large number of variables that evolve dynamically over time. Simulation models can help identify the critical functional and relational aspects of a system. Thus, dynamic simulation modelling allows us to understand why a system behaves the way it does as a function of its organization and relationships among components of the system

19 Simulation models Open the possibility to evaluate more aspects or characteristics of the system Considers how different actors can influence the outcome of a programme or intervention Not only useful for complex systems but also for interventions or health conditions where time or patient history are important Even when a MM is enough a DES can offer a more clear vision of the pathway (visual representation of the model) DES models are easier to modify (add, change or delete a health state)

20 SIMULATE Checklist* System Interactions Multilevel Understanding Loops Agents Time Emergence Modelling multiple events, relationships, and stakeholders representing health care delivery processes? Including nonlinear or spatial relationships among stakeholders and their context that influence behaviours and make outcomes in the system difficult to anticipate? Modelling a health care delivery problem from strategic, tactical, or operational perspectives? Modelling a complex problem to improve patient-centered care that cannot be solved analytically? Modelling feedback loops that change the behaviour of future interactions and the consequences for the delivery system? Modelling multiple stakeholders with behavioural properties that interact and change the performance of the system? Time-dependent and dynamic transitions in a health care delivery system, either between or within health care system levels or in health status change? Considering the intended and unintended consequences of health system interventions to address policy resistance and achieve target outcomes? *Marshall DA, Burgos-Liz L, IJzerman MJ, et al. Applying dynamic simulation modeling methods in health care delivery research The SIMULATE checklist: Report of the ISPOR Simulation Modeling Emerging Good Practices Task Force. Value Health 2015;18:5-16.

21 References Marshall DA, Burgos-Liz L, IJzerman MJ, et al. Applying dynamic simulation modeling methods in health care delivery research The SIMULATE checklist: Report of the ISPOR Simulation Modeling Emerging Good Practices Task Force. Value Health 2015;18:5-16. Marshall DA, Burgos-Liz L, IJzerman MJ, et al. Selecting a dynamic simulation modeling method for health care delivery research Part 2: Report of the ISPOR Simulation Modeling Emerging Good Practices Task Force. Value Health 2015;18: Berkeley Madonna: SIMUL8: NetLogo: