G54SIM (2011) Lecture 02 Simulation Studies - An Overview Peer-Olaf Siebers pos@cs.nott.ac.uk
Lecture Outline 1. Life Cycle of a Simulation Study Key Processes Verification and Validation Simulation Project Time-Scales 2. Case Study Modelling the Cargo Screening Process at the Ferry Port in Calais G54SIM (http://www.cs.nott.ac.uk/~pos/g54sim/) 2
Life cycle of a simulation study Robinson (2004) Balci (1990) Andersson and Karlsson (2001) G54SIM (http://www.cs.nott.ac.uk/~pos/g54sim/) 3
Oval symbols: Phases Dashed arrows: Processes Solid arrows: Credibility assessment stages G54SIM (http://www.cs.nott.ac.uk/~pos/g54sim/) 4
G54SIM (http://www.cs.nott.ac.uk/~pos/g54sim/) 5
Processes Problem formulation (problem structuring / definition) Communicated problem is rarely clear, specific, or organized Initially communicated problem is translated into a formulated problem sufficiently well defined to enable specific research action Investigation of Solution Techniques Sometimes communicated problem is formulated under the influence of a solution technique in mind Important to identify all alternative techniques that can be used in solving the formulated problem Chosen technique needs to be a sufficiently credible one which will be accepted and used by the decision maker(s) G54SIM (http://www.cs.nott.ac.uk/~pos/g54sim/) 6
G54SIM (http://www.cs.nott.ac.uk/~pos/g54sim/) 7
Processes System investigation Process of investigating the characteristics of the system that contains the formulated problem (for consideration in system definition and modelling) Change characteristics: How often and how much the real system will change during the course of a simulation study Environment characteristics: Consists of all input variables that can significantly affect its state Counterintuitive behaviour characteristics: Some systems may show counterintuitive behaviour which we should try to identify for consideration in the model G54SIM (http://www.cs.nott.ac.uk/~pos/g54sim/) 8
Processes System investigation (cont.) Drift to low performance characteristics: A system may show a drift to low performance due to the deterioration of its components over a period of time Interdependency and organisation characteristics: In a complex system, many activities or events take place simultaneously and influence each other; needs to be examined before abstracting the real system for the purpose of modelling; decomposing the system into subsystems and subsystems into sub-subsystems G54SIM (http://www.cs.nott.ac.uk/~pos/g54sim/) 9
G54SIM (http://www.cs.nott.ac.uk/~pos/g54sim/) 10
Processes Model formulation Process by which a conceptual model is envisioned to represent the system under study Robinson (2004): "The conceptual model is a non-software specific description of a simulation model describing the objectives, inputs, outputs, content, assumptions and simplification of the model" Input data analysis and modelling SDS: Model driven by rate changes defined through differential equations DES and ABS: Model driven by input values obtained via sampling from probability distributions G54SIM (http://www.cs.nott.ac.uk/~pos/g54sim/) 11
Processes Model Representation Process of translating the conceptual model into a communicative model (representation which can be communicated to other humans) Typical representation formats: SDM: Causal loop diagrams, stock and flow diagrams DEM: Flow charts, activity cycle diagrams ABM: UML (14 types of diagrams) + AgentUML Specific mechanisms: Pseudo-code Criteria for selection: Applicability for describing the system under study Technical background of the people to whom the model is communicated Translatability into a programmed model G54SIM (http://www.cs.nott.ac.uk/~pos/g54sim/) 12
Resting *** Processes STORE Entering CUSTOMERS Customer #3 State-Chart Customer #2 State-Chart Customer #1 State-Chart SIGNALS STAFF Rota Resting *** Queuing at till (for refund) Being served at till (refund decision) Want to buy Staff #3 State-Chart Staff #2 State-Chart Staff #1 State-Chart Want help Serving Seeking refund Browsing Want refund Contemplating (dummy state) Waiting Seeking help Queuing at till (to buy) Invite Evaluating (system state) Queuing for help Being helped Being served at till (buying) Leaving Spreading word of mouth Evaluating (shopping experience) *** = initialisation state G54SIM (http://www.cs.nott.ac.uk/~pos/g54sim/) 13
Processes Input data modelling situation min mode max leave browse state after 1 7 15 leave help state after 3 15 30 leave pay queue (no patience) after 5 12 20 PDF 1 7 15 x event someone makes a purchase after browsing someone requires help someone makes a purchase after getting help probability of event 0.37 0.38 0.56 G54SIM (http://www.cs.nott.ac.uk/~pos/g54sim/) 14
G54SIM (http://www.cs.nott.ac.uk/~pos/g54sim/) 15
Processes Programming Nowadays mainly Visual Interactive Modelling Systems (VIMS) Software: SDS: Dynamo, ithink/stella, PowerSim, Vensim,... DES: Arena, SimIO, Simul8, Witness, ProModel, Extend, FlexSim,... ABS: AnyLogic, many academic tools focusing on specific research areas Programming languages: GPSS, SIMAN, SIMSCRIPT, SIMULA, SLAM,... Survey available at OR/MS website (latest version from October 2011) http://www.lionhrtpub.com/orms/surveys/simulation/simulation.html G54SIM (http://www.cs.nott.ac.uk/~pos/g54sim/) 16
G54SIM (http://www.cs.nott.ac.uk/~pos/g54sim/) 17
Processes Design of experiment Process of formulating a plan to gather the desired information at minimal cost and to enable the analyst to draw valid inferences Obtaining accurate results Run conditions: Warm up period, number of replications, run length Variance reduction techniques: Obtain greater statistical accuracy for the same amount of simulation runs Searching the solution space Response-surface methodologies: Find the optimal combination of parameter values which maximize or minimize the value of a response variable Factorial designs: Determine the effect of various input variables on a response variable Ranking and selection techniques: comparing alternative systems G54SIM (http://www.cs.nott.ac.uk/~pos/g54sim/) 18
Processes Experimentation What-if analysis Making changes to the model s inputs, running the model, inspecting the results, learning from the results, making changes to the model s inputs... Different purposes of experimentation Comparison of different operating policies, evaluation of system behaviour, sensitivity analysis, forecasting, optimisation, determination of functional relations Output analysis (for stochastic simulation) Analysis of results from single scenario (mean and standard deviation) Comparing alternative scenarios (using confidence intervals to test difference between results from two scenarios) G54SIM (http://www.cs.nott.ac.uk/~pos/g54sim/) 19
Processes Histograms of the same mean but different levels of variability Robinson (2004) G54SIM (http://www.cs.nott.ac.uk/~pos/g54sim/) 20
G54SIM (http://www.cs.nott.ac.uk/~pos/g54sim/) 21
Processes Redefinition Maintaining the model for further use Updating the model so that it represents the current form of the system Altering it for obtaining another set of results Presentation of simulation results Process of interpreting simulation results and presenting them to the decision makers for their acceptance and implementation Implementation: Putting the solution into practice Implementing the model Implementation as a learning aid G54SIM (http://www.cs.nott.ac.uk/~pos/g54sim/) 22
Verification and Validation Verification and validation are continuous processes that are performed throughout the life cycle of the simulation study Verification is substantiating that the simulation model has been transformed from one form into another as intended with sufficient accuracy Validation is substantiating that the simulation model, within its domain of applicability, behaves with satisfactory accuracy consistent with the study objectives G54SIM (http://www.cs.nott.ac.uk/~pos/g54sim/) 23
Break See you in 10 minutes... G54SIM (http://www.cs.nott.ac.uk/~pos/g54sim/) 24
Simulation Project Time-Scales Cochran et al. (1995): Surveyed: Simulation users in industrial settings more than 6 month 29 3 to 6 month 16 1 to 3 month 31 1 week to 1 month 20 1 week or less 4 0 5 10 15 20 25 30 35 G54SIM (http://www.cs.nott.ac.uk/~pos/g54sim/) 25
Case Study Modelling the Cargo Screening Process at the Ferry Port in Calais
Problem Formulation Location: Calais Ferry Port (France) Problem: Illegal immigration 900.000 lorries/year 3500 positive lorries ~ 0.4% G54SIM (http://www.cs.nott.ac.uk/~pos/g54sim/) 27
Problem Formulation G54SIM (http://www.cs.nott.ac.uk/~pos/g54sim/) 28
System Investigation
System Investigation French Border Control Offices and Detention Facilities UK Border Control Offices and Detention Facilities French Passport Check French Screening Facilities Tickets UK Passport Check UK Search Facilities Berth Parking Space French Deep Search Facilities UK Deep Search Facilities Controlled by Calais Chamber of Commerce (CCI) Controlled by UK Border Agency G54SIM (http://www.cs.nott.ac.uk/~pos/g54sim/) 30
System Investigation Inspection Sheds Heartbeat Detector CO2 Probe Visual Inspection Canine Sniffers Drive Through Passive Millimetre Wave Scanner G54SIM (http://www.cs.nott.ac.uk/~pos/g54sim/) 31
Model Representation (French Side) Arrive at Calais S=44%; H=56% 100% PassportCheck_F 1.0 Screening_F@ BorderAgency_F S PMMWCheck_F x.x DeepSearch_F@ Tickets@ @Screening_F Red font = example H HBCheck_F 0.6 4.90% 95.10% OpenCheck_H Tickets@ RemovePeople_H Tickets@ 100% BorderAgency_F OpenCheck_S x.x RemovePeople_S Tickets@ 100% BorderAgency_F @DeepSearch_F CO2Check_F x.x OpenCheck_S 0.99 Tickets@ RemovePeople_S Tickets@ 100% BorderAgency_F @Ticket 100% TicketFerry 100% PassportCheck_UK 1.0 Routing@ BorderAgency_UK G54SIM (http://www.cs.nott.ac.uk/~pos/g54sim/) 32
Model Representation (UK Side) @Routing Berth@ Search_UK@ CO2Check_UK x.x DeepSearch_UK@ Berth@ @Search_UK S HBCheck_UK x.x OpenCheck_H x.x Berth@ RemovePeople_H Berth@ 100% BorderAgency_UK H OpenCheck_H x.x RemovePeople_H Berth@ 100% BorderAgency_UK @DeepSearch_UK 100% OpenCheck_S x.x RemovePeople_S Berth@ 100% BorderAgency_UK S random checks CO2Check_F x.x OpenCheck_S x.x @Berth RemovePeople_S @Berth 100% BorderAgency_UK @Berth 100% WaitFerryArrival a maximum of 175 lorries can leave Calais once per hour Leave to Dover H random checks OpenCheck_H RemovePeople_H 100% BorderAgency_UK G54SIM (http://www.cs.nott.ac.uk/~pos/g54sim/) 33 x.x @Berth
Programming G54SIM (http://www.cs.nott.ac.uk/~pos/g54sim/) 34
Programming Inspection sheds and berth activities G54SIM (http://www.cs.nott.ac.uk/~pos/g54sim/) 35
Experimentation G54SIM (http://www.cs.nott.ac.uk/~pos/g54sim/) 36
proportion of clandestines detected Experimentation Detection Rate vs. Clandestines Detected 1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% detection rate G54SIM (http://www.cs.nott.ac.uk/~pos/g54sim/) 37
Further Reading Balci (1990) Robinson (2004) - Chapter 4 Siebers et al. (2009)
Comments or Questions? G54SIM (http://www.cs.nott.ac.uk/~pos/g54sim/) 39
References Andersson C and Karlsson L (2001). A system dynamics simulation study of a software development process. CODE:LUTEDX(TETS-5419)/1-83/(2001)&local 3, Department of Communication Systems, Lund Institute of Technology. Balci O (1990). Guidelines for successful simulation studies. In: Proceedings of the 1990 Winter Simulation Conference, IEEE: Piscataway, NJ, pp. 25 32. Cochran JK, Mackulak GT and Savory PA (1995). Simulation project characteristics in industrial settings, Interfaces 25(4), pp. 104 113. Robinson (2004). Simulation: The practice of model development and use. Wiley, Chichester, UK. Siebers PO, Aickelin U and Sherman G (2009) Development of a cargo screening process simulator: A first approach. In: Proceedings of the 21st European Modeling and Simulation Symposium, 23-25 September, Tenerife, Spain.