Geospatial Visual Analytics Tutorial. Gennady Andrienko & Natalia Andrienko

Geospatial Visual Analytics Tutorial Gennady Andrienko & Natalia Andrienko http://geoanalytics.net

Outline Visual Analytics Introduction - Definition of Visual Analytics - Roots - What is new? Where are we now? - Live demo of current visualisation techniques - Example of visual analytics: evacuation scheduling - Example of visual analytics: analysis of movement Where should we go? - The visual analytics example: what is lacking? - Visual analytics for spatial decision support Conclusion 2

Definition of Visual Analytics Visual analytics. Introduction Visual Analytics is the science of analytical reasoning facilitated by interactive visual interfaces. People use visual analytics tools and techniques to Synthesize information and derive insight from massive, dynamic, ambiguous, and often conflicting data Detect the expected and discover the unexpected Provide timely, defensible, and understandable assessments Communicate assessment effectively for action The book (IEEE Computer Society 2005) is available at http://nvac.pnl.gov/ in PDF form 4

Emergence of Visual Analytics Visual analytics. Introduction Initially driven by the USA Homeland Security but now has a much broader scope and impact 5

Conferences, symposia, workshops University courses and seminars Research programs 6

Grand Challenges Visual analytics. Introduction The need to analyse and make sense of overwhelming amounts of complex, disparate, conflicting, and dynamic data and information - Requires human judgement - People must collaborate and share information and knowledge The scalability challenge - Information scalability: the capability to extract relevant information from massive data streams - Visual scalability: the capability to effectively display massive data sets - Display scalability: effective use of everything from wall-sized to phone-sized displays - Human scalability: scale from single users to collaborative environments - Software scalability: the capability of a software system to interactively manipulate large data sets 7

Focus of Visual Analytics Visual analytics. Introduction The science of analytical reasoning but what kind of reasoning? Given: massive, dynamic, ambiguous, and often conflicting data Required: - Synthesise information and derive insight - Detect the expected and discover the unexpected - Provide timely, defensible, and understandable assessments Analytical reasoning = data information knowledge explanation (interpreted data) (for myself) (for others) Focus: interactive visual interfaces that enable this process 8

Visual analytics. Introduction Roots of Visual Analytics Visualisation - information visualisation - scientific visualisation - geographic visualisation use of graphics and images for decoration for illustration for communication for exploration, discovery, and ideation Current standard: highly interactive interfaces Recent trends: Combine visualisation with data mining Incorporate database and data warehousing technologies in order to handle very large and complex data 9

Visual analytics. Introduction Visualisation vs. Visual Analytics Visualisation: Visual analytics: Interactive visual interfaces Interactive visual interfaces o to stimulate insight into data: data information knowledge detect the expected and discover the unexpected in massive, dynamic, ambiguous, and often conflicting data Reasoning and synthesis of information and knowledge are implicit o to facilitate analytical reasoning: data information knowledge explanation detect the expected and discover the unexpected in massive, dynamic, ambiguous, and often conflicting data - provide understandable assessments - communicate assessments effectively Information, knowledge, and reasoning must be made explicit 10

A Field for Multidisciplinary Research Visual analytics. Introduction Visual Analytics Analytical reasoning - How to maximise human capacity to perceive, understand, Cognitive science and reason about complex and dynamic data and situations? Knowledge Management Artificial intelligence (?) Visual representations and interaction techniques - How to augment cognitive reasoning with perceptual reasoning through visual representations and interaction? Data representations and transformations Data Mining: numeric data, texts, images, video - How to transform data into a representation that is Database Science appropriate to the analytical task and effectively conveys the important content? GI Science Production, presentation, and dissemination - How to convey analytical results in meaningful ways to various audiences? Visualisation Human-Computer Interaction Knowledge Management Knowledge Visualisation Communication Sciences Artificial Intelligence (?) 11

Visual analytics. Introduction Artificial Intelligence and Visual Analytics Artificial Intelligence Typology of knowledge - Ontologies: kinds of entities and how they are related - Actions, situations, events - Procedural knowledge Knowledge representation Knowledge engineering Reasoning Modelling of uncertainty and confidences Explanation Learning Visual Analytics Must build upon an understanding of - the reasoning process - analytical artefacts (e.g. patterns) Requires knowledge capture (tacit knowledge explicit knowledge) - Pre-existing knowledge involved in analysis and reasoning - New knowledge resulting from analysis Requires knowledge representation Requires modelling of uncertainty and confidences Requires building of explanations 12

Current visualisation techniques 1) Interactive and dynamic data displays Visual Analytics: Where are we now? Visual comparison Access to data 14 Time controls and animation

Current visualisation techniques 2) Coordinated complementary views Visual Analytics: Where are we now? 15

Current visualisation techniques 3) Interactive dynamic querying Visual Analytics: Where are we now? 16

Current visualisation techniques 4) Interactive data transformations Visual Analytics: Where are we now? 17

Visual Analytics: Where are we now? Current visualisation techniques 5) Integration of visual and computational analysis methods 18

Details: Natalia and Gennady Andrienko Exploratory Analysis of Spatial and Temporal Data A Systematic Approach Approx. 715 p. 282 illus., 37 in colour., Hardcover, 139,05 Springer-Verlag, December 2005, ISBN 3-540-25994-5 http://geoanalytics.net/eda sample chapter, online demos and tutorials 19

Visual Analytics: Where are we now? Example of visual analytics: evacuation scheduling 20

Outline Introduction Problem analysis and task-oriented design Example work scenario User-controlled schedule modification Conclusion 21

Emergency evacuation problem Time-critical decision problems Require high efficiency Involve much data Complex Problems involving geographical space and time Ill-defined Depend on tacit knowledge and criteria No adequate computer representation for geographic space Computational methods Synergy required! Human expert Background knowledge Understanding of geographical space Experience Intuition 22

Approach produces Draft solution Background knowledge Understanding of geographical space Experience Intuition assesses Input: data, goals, criteria, guides produces Vehicle Vehicle home base Source Source name Destination Destination name Start End Item Item class name Operation ID ID ID class Number ID of items Pick Up 41 Children clinics 75 Children clinics 60 Braun and Co 00:00:00 00:00:04 0 EMPTY 1 Deliver 41 Children clinics 75 Children clinics 60 Braun and Co 00:00:00 00:00:04 0 EMPTY 1 Pick Up 41 Children clinics 60 Braun and Co 68 St. John Hospital 00:00:04 00:00:30 21 invalids who cannot seat 2 Deliver 41 Children clinics 60 Braun and Co 68 St. John Hospital 00:00:04 00:00:30 21 invalids who cannot seat 2 Pick Up 41 Children clinics 68 St. John Hospital 22 St. Peter Hospital 00:00:51 00:00:54 0 EMPTY 1 Deliver 41 Children clinics 68 St. John Hospital 22 St. Peter Hospital 00:00:51 00:00:54 0 EMPTY 1 Pick Up 41 Children clinics 22 St. Peter Hospital 40 Spa healing house 00:00:54 00:01:22 21 invalids who cannot seat 2 Deliver 41 Children clinics 22 St. Peter Hospital 40 Spa healing house 00:00:54 00:01:22 21 invalids who cannot seat 2 Pick Up 102 City coach park 109 City coach park 60 Braun and Co 00:00:00 00:00:05 0 EMPTY 1 Deliver 102 City coach park 109 City coach park 60 Braun? and Co 00:00:00 00:00:05 0 EMPTY 1 Pick Up 102 City coach park 60 Braun and Co 50 Exhibition hall 00:00:05 00:00:18 10 general people or children 50 Deliver 102 City coach park 60 Braun and Co 50 Exhibition hall 00:00:05 00:00:18 10 general people or children 50 Pick Up 102 City coach park 50 Exhibition hall 32 Kindergarten 00:00:28 00:00:34 0 EMPTY 1 Deliver 102 City coach park 50 Exhibition hall 32 Kindergarten 00:00:28 00:00:34 0 EMPTY 1 Pick Up 102 City coach park 32 Kindergarten 41 Descartes School 00:00:34 00:00:51 10 general people or children 50 Deliver 102 City coach park 32 Kindergarten 41 Descartes School 00:00:34 00:00:51 10 general people or children 50 Pick Up 48 City coach park 109 City coach park 60 Braun and Co 00:00:00 00:00:05 0 EMPTY 1 Deliver 48 City coach park 109 City coach park 60 Braun and Co 00:00:00 00:00:05 0 EMPTY 1 Pick Up 48 City coach park 60 Braun and Co 49 Leonardo School 00:00:05 00:00:18 10 general people or children 50 Deliver 48 City coach park 60 Braun and Co 49 Leonardo School 00:00:05 00:00:18 10 general people or children 50 Pick Up 48 City coach park 49 Leonardo School 5 Frings Gymnasium 00:00:28 00:00:33 0 EMPTY 1 Deliver 48 City coach park 49 Leonardo School 5 Frings Gymnasium 00:00:28 00:00:33 0 EMPTY 1 Pick Up 48 City coach park 5 Frings Gymnasium 41 Descartes School 00:00:33 00:00:48 10 general people or children 50 Deliver 48 City coach park 5 Frings Gymnasium 41 Descartes School 00:00:33 00:00:48 10 general people or children 50 Pick Up 78 City Gennady coach park & 109 Natalia City coach park Andrienko 21 Kindergarten 00:00:00 00:00:05 0 EMPTY 1 Deliver 78 City coach park 109 City coach park 21 Kindergarten 00:00:00 00:00:05 0 EMPTY 1 Pick Up 78 City coach park 21 Kindergarten 50 Exhibition hall 00:00:05 00:00:19 10 general people or children 20 Deliver 78 City coach park 21 Kindergarten 50 Exhibition hall 00:00:05 00:00:19 10 general people or children 20 Pick Up 78 City coach park 50 Exhibition hall 18 Albert College 00:00:29 00:00:35 0 EMPTY 1 Deliver 78 City coach park 50 Exhibition hall 18 Albert College 00:00:29 00:00:35 0 EMPTY 1 Pick Up 78 City coach park 18 Albert College 42 Riverside hall 00:00:35 00:00:52 10 general people or children 50 Deliver 78 City coach park 18 Albert College 42 Riverside hall 00:00:35 00:00:52 10 general people or children 50 Pick Up 117 Bus travel company 110 Bus travel company 60 Braun and Co 00:00:00 00:00:06 0 EMPTY 1 Deliver 117 Bus travel company 110 Bus travel company 60 Braun and Co 00:00:00 00:00:06 0 EMPTY 1 Pick Up 117 Bus travel company 60 Braun and Co 42 Riverside hall 00:00:06 00:00:23 10 general people or children 100 Deliver 117 Bus travel company 60 Braun and Co 42 Riverside hall 00:00:06 00:00:23 10 general people or children 100 Improved solution assesses Accepted solution Requires visualisation! 23

Outline Introduction Problem analysis and task-oriented design Example work scenario User-controlled schedule modification Conclusion 24

Emergency evacuation problem Several categories of people - General public; critically sick or injured people; disabled people who can/cannot sit, prisoners, Multiple source locations - Number of people of different categories - Time constraints (e.g. latest allowed departure time) Multiple destinations - Suitability and capacity for different categories Different types of vehicles - E.g. buses, ambulance cars, police vans, - Suitability and capacity for different people categories Task: - divide people into groups fitting in available types of vehicles - assign the groups to suitable destinations - find appropriate vehicles to deliver them - set the times for the trips of the vehicles 25

Scheduling Algorithm For transportation problems, heuristic methods work better than deterministic approaches We apply Breeder Genetic Algorithm (devised by Bartling & Muehlenbein) Extended functionality as compared to typical tools for business applications: - Divides the total number of people in a location into groups fitting in available vehicles - Chooses an appropriate destination for each group Any-time method: - valid solution exists at any moment - while the quality is progressively improved as the algorithm continues its work 26

Algorithm output.g. 14 source locations 4692 people 6 categories 105 vehicles 7 vehicle types 25 destinations No time to inspect all the orders! Cannot be summarized in a few indicators! 400 transportation orders OrderId SourceName DestName ItemClass Number VhclId VhclType VhclHBName StartTime EndTime 12-4 St. John Hospital St. Peter Hospital LEER 0 12 20 University clinics 00:32:40 00:40:4 72-1 Braun and Co Exhibition hall general people or children 50 72 10 City coach park 00:11:00 00:20:4 61-4 Albert College Descartes School general people or children 40 61 10 City coach park 00:31:00 00:50:2 29-1 Braun and Co Rehabilitation Centre critically sick or injured people 1 29 21 Children Clinics 00:05:00 00:11:2 63-0 City coach park ABC mall LEER 0 63 10 City coach park 00:00:00 00:11:0 43-6 Beethoven Gymnasiu Galileo College general people or children 50 43 10 City coach park 00:48:20 01:11:0 53-0 City coach park ABC mall LEER 0 53 10 City coach park 00:00:00 00:11:0 12-1 St. Peter Hospital University clinics invalids who cannot seat 2 12 20 University clinics 00:08:00 00:16:1 56-3 Kindergarten Plato Gympasium general people or children 50 56 10 City coach park 00:33:20 00:50:0 58-4 Albert College City hall general people or children 20 58 10 City coach park 00:37:00 00:43:4 46-0 City coach park Real school LEER 0 46 10 City coach park 00:00:00 00:13:0 43-1 ABC mall Leonardo School general people or children 20 43 10 City coach park 00:12:00 00:21:4 58-5 City hall Albert College LEER 0 58 10 City coach park 00:44:40 00:50:4 91-2 Elder home Children Clinics sabled people using wheelchairs 8 91 13 Bus travel company 00:27:40 00:36:0 86-0 Bus travel company AAlbert College LEER 0 86 10 Bus travel company 00:00:00 00:11:0 61-1 Braun and Co City hall general people or children 40 61 10 City coach park 00:11:00 00:23:4 23-0 Children Clinics Braun and Co LEER 0 23 20 Children Clinics 00:00:00 00:05:0 47-1 Braun and Co Helmholtz Gymnasium general people or children 50 47 10 City coach park 00:11:00 00:24:4 40-5 Beethoven Gymnasiu St. Teresa's school general people or children 20 40 10 City coach park 00:28:40 00:44:4 52-4 St.Joseph's basic sch Helmholtz Gymnasium general people or children 50 52 10 City coach park 00:34:20 00:44:0 20-3 Elder home Children clinics invalids who cannot seat 2 20 20 St. John Hospital 00:26:20 00:33:3 103-0 Jailhouse Prison LEER 0 103 30 Jailhouse 00:00:00 00:46:0 11-0 St. Peter Hospital Braun and Co LEER 0 11 21 St. Peter Hospital 00:00:00 00:08:0 17-1 Braun and Co Children clinics critically sick or injured people 1 17 21 University clinics 00:16:00 00:22:2 6-2 Braun and Co Children clinics invalids who cannot seat 2 6 20 St. Peter Hospital 00:17:20 00:23:3 58-3 Exhibition hall Albert College LEER 0 58 10 City coach park 00:22:00 00:37:0 54-2 City hall Albert College LEER 0 54 10 City coach park 00:25:20 00:31:2 52-2 Braun and Co City hall general people or children 40 52 10 City coach park 00:13:20 00:25:4 44-0 City coach park Braun and Co LEER 0 44 10 City coach park 00:00:00 00:11:0 52-5 Helmholtz GymnasiumSt.Joseph's basic scho LEER 0 52 10 City coach park 00:45:40 00:53:4 61-3 City hall Albert College LEER 0 61 10 City coach park 00:25:00 00:31:0 25-6 University clinics St. Peter Hospital LEER 0 25 20 Children Clinics 00:47:00 00:55:0 61-0 City coach park Braun and Co LEER 0 61 10 City coach park 00:00:00 00:11:0 81-5 Albert College Heighbourhood House general people or children 40 81 10 City coach park 00:56:00 01:08:2 27

Schedule evaluation Questions to be answered: Does the plan achieve the goal? Goal: all people are timely delivered to appropriate destination places by appropriate vehicles Is it feasible? Is it rational? 28

Possible problems Undelivered people Late deliveries w.r.t. time constraints Use of improper vehicles Delivery to improper places Overuse of resources Multiple vehicles in same place Can emerge due to lack or deficiency of resources Require human to find appropriate corrective measures (additional vehicles, additional or intermediate destinations, ) Excluded by the algorithm, but correctness should be demonstrated May be a problem or an advantage; requires human s local knowledge Idle vehicles Choice of distant destinations Lower priority; may be examined when time permits Low use of vehicle capacities Effectiveness problems (i.e. goal not attained) Feasibility problems Rationality problems 29

Requirements The presence or absence of effectiveness and feasibility problems must be immediately visible In case of problems, the reasons must be immediately seen or easy to find out - Undelivered people, use of improper destinations lack of suitable destinations - Late deliveries, use of improper vehicles deficiency of suitable vehicles - Multiple vehicles in same place: examine each place individually It must be possible to spot and explore rationality problems when time permits 30

Visual Analytics Approach Visual Analytics Mantra by D.Keim: Analyze First Show the Important Zoom, Filter and Analyze Further Details on Demand Input data Interactive visual interfaces Scheduling algorithm Schedule Computational analysis module Secondary data User s evaluation feedback Queries 31

Outline Introduction Problem analysis and task-oriented design Example work scenario User-controlled schedule modification Conclusion 32

Analyze First Show the Important ummary display of the transportation progress: Signals of problems: Some people are not delivered will be seen here Reason: lack of destinations Delay in transportation will be seen here Reason: not enough vehicles Not always the reason is evident 33

Zoom, Filter and Analyze Further Find a non-evident reason for a delay But they will be here while we need them here! The return trip takes 47 minutes At this time we shall have 5 free vehicles! This is the earliest possible time for taking the remaining prisoners 34

Details on demand 35

Zoom, Filter and Analyze Further So many trips to Braun and Co! How are they distributed in time? Check the feasibility 23 buses come simultaneously! permutable matrix shows the numbers of trips between pairs of locations but this is a big chemical plant with a large parking 36

Details on demand 37

Zoom, Filter and Analyze Further Assess the rationality But it is hard to see anything when we focus on the general people The choice of the destinations for the critically sick and injured persons seems quite reasonable 38

Zoom, Filter and Analyze Further Assess the rationality (continued) -shaped signs: Number of items Some people will be moved quite far Travel time while the capacities in the closer destinations are not fully used The planner may wish to change this if the time permits! 39

Outline Introduction Problem analysis and task-oriented design Example work scenario User-controlled schedule modification Conclusion 40

Reasons for Schedule Modification Undelivered people - Requires finding additional destinations Unacceptable delays - Requires finding additional vehicles or closer destinations (possibly, intermediate) Multiple vehicles in same place - The planner may shift some orders forward in time Non-rational choice of destinations and use of capacities in destinations - The planner may exclude distant places Situation changes after the evacuation started - New people appear, some destinations become unavailable (e.g. roads blocked), some vehicles get out of use, trips take longer than expected, 41

General Procedure Divide the orders into fixed and modifiable - by people category (e.g. critically sick fixed, prisoners modifiable) - by time: fix all trips starting before t in particular, for adapting to the changing situation - by source location (e.g. from Braun and Co modifiable) - by a combination of these criteria Update the input data - Add data about new sources, people, resources - Remove unavailable resources - Correct the travel times Re-run the scheduling algorithm (it is appropriately designed) 42

Conclusion Solving complex problems related to space & time requires a synergy between computers and human experts Visual analytics methods can support this We did a task-oriented design of VA tools for schedule assessment We verified the adequacy of the tools Trials with professional users are in preparation Details: Gennady Andrienko, Natalia Andrienko, Ulrich Bartling Visual Analytics Approach to User-Controlled Evacuation Scheduling IEEE Visual Analytics Science and Technology (VAST 2007) Proceedings, IEEE Computer Society Press, 2007, pp.43-50 43

Visual Analytics: Where are we now? Example of visual analytics: analysis of movement 44

Data (example) Positions of a private car from 03/12/06 till now N;Time;Lat;Long;Height;Course;Speed;PDOP;State;NSat 8;22/03/07 08:51:52;50.777132;7.205580; 67.6;345.4;21.817;3.8;1808;4 9;22/03/07 08:51:56;50.777352;7.205435; 68.4;35.6;14.223;3.8;1808;4 10;22/03/07 08:51:59;50.777415;7.205543; 68.3;112.7;25.298;3.8;1808;4 11;22/03/07 08:52:03;50.777317;7.205877; 68.8;119.8;32.447;3.8;1808;4 12;22/03/07 08:52:06;50.777185;7.206202; 68.1;124.1;30.058;3.8;1808;4 13;22/03/07 08:52:09;50.777057;7.206522; 67.9;117.7;34.003;3.8;1808;4 14;22/03/07 08:52:12;50.776925;7.206858; 66.9;117.5;37.151;3.8;1808;4 15;22/03/07 08:52:15;50.776813;7.207263; 67.0;99.2;39.188;3.8;1808;4 16;22/03/07 08:52:18;50.776780;7.207745; 68.8;90.6;41.170;3.8;1808;4 17;22/03/07 08:52:21;50.776803;7.208262; 71.1;82.0;35.058;3.8;1808;4 18;22/03/07 08:52:24;50.776832;7.208682; 68.6;117.1;11.371;3.8;1808;4 About 90,000 positions by 15/08/2007 (used in examples); more than 110,000 by now 45

The positions in time 46

Task (example) Investigate the movement behaviour of the car owner: - Detect and interpret typical trips: sources, destinations, routes, intermediate stops, purposes, - Find out the typical times of the trips - If possible, interpret also atypical trips Problems: - no explicit trips in the data but just positions - no meaningful places but latitudes and longitudes Places and trips have to be defined through data analysis! We know that the positions were recorded only while the car moved. Otherwise, the data could be filtered to remove sequences of records made in the same place (using an appropriate distance threshold). 47

Data preprocessing (one-time operation in DB) Original data: <t, x, y> Results of preprocessing: Points are arranged into sequences (NEXTTIME field) DIFFTIME and DISTANCE to the next position, DX, DY Derived speed, course, acceleration and turn in each point Additional temporal components are easy to extract from the database: day of week, day of year, decade of month We separated time-consuming data pre-processing (needs to be performed only once!) from rapid analysis and aggregation procedures (that take just few seconds) 48

A new galaxy? (points are plotted according to DX & DY) An artefact of the straight line filtering (threshold = 20 meters) by the data collection software 49

Defining significant places We know that the positions were recorded only while the car moved. Large time gaps between successive positions indicate stops. We can query the database for the positions where DIFFTIME (distance in time to the next position) exceeds a specified threshold The possible meanings of the extracted places will differ depending on the threshold - Large gaps: places where the person spends much time (home, work) - Medium gaps: shops, doctors, sport facilities, - Small gaps: traffic lights, street crossings, Places separated by medium to large gaps may be destinations and sources of trips Sequences of positions between the gaps represent the trips 50

Extraction of the places of stops (DB query + GUI + Visualisation) 3 hours 30 minutes 5 minutes 30 seconds 51

Interpretation of the places of stops A) Long stops ( 3 hours) 1) Spatial clustering: find repeated stops 2) Look at the days and times of the occurrence Days of the week Hours of the day Red: home, blue: work 52

Interpretation of the places of stops B) Medium stops ( 30 minutes) 1) Spatial clustering: find repeated stops 2) Look at the days and times of the occurrence (clusters 1&2 excluded) Days of the week Hours of the day Green & light blue: probably shopping 53

Interpretation of the clusters 3 and 4 (continued) Saturday and Sunday: the stops mostly occur between 10 and 14 Monday to Friday: the stops mostly occur in the evening hours (max between 18 and 19) 54

The interpreted places Visual Analytics example other town parking other town parking shopping centre 1 shopping centre 1 home home shopping centre 2 shopping centre 2 work work sports place sports place 55

Extraction of trips (DB operation + GUI + Visualisation) 56

A trip in more detail End Route Time Start 57

Problems with data quality Many false starts; the real starts are missing Positioning errors 58

Looking for typical trips 1) From where and to where? The trajectories are very roughly clustered according to the spatial proximity of their starts and ends Visual Analytics example 59

Interactive tool for spatial clustering Joint work with S.Rinzivillo, Univ. Pisa: - spatial clustering algorithm OPTICS (Ankerst, Breunig, Kriegel, and Sander 1999) - our implementation: cluster building is separated from distance and neighbourhood computation - benefit: an analyst may try different distance measures - we have developed several variants of distance measures designed specially for trajectories - some of the measures can tolerate the specific errors in the car driving data 60

Cluster 1: from around the work to home (111 trips) Visual Analytics example he false starts can be seen mall green circles) Observation: the eastern route is chosen more often 61

Visual Analytics example Cluster 2: from around home to around the work (111 trips) Observation: the eastern route is chosen much more often 62

Cluster 3: trips starting and ending around home (52 trips) Visual Analytics example Observation: the distant trips are mostly occasional 63

The summarised view in more detail Major flow Many small moves Variations 64

Summarisation of trajectories: how it is done (1) Original trajectory Simplified trajectory Generalised positions Specifies which positions to keep 65

Summarisation: how it is done (2) All trajectories simplified Generalised positions (circles, for simplicity) are built around spatially close positions of all trajectories 66

Summarisation: how it is done (3) Numbers of moves between the positions are counted and represented by arrows of varying width 67

Summarized representation of clusters A single cluster is selected: the moves from this cluster only are counted and shown Disadvantage: some arrows may cover other arrows 2 or more clusters are selected: the moves from these clusters are counted and shown; dark grey corresponds to common moves In the future, we ll try to put them in a 3D view (e.g. Google Earth) 68

Progressive clustering 1) Select one or a few clusters 2) Apply another clustering to the selection; use a different distance function or different parameter settings 69

Analysing the trips from work to home (111 trips) Visual Analytics example Summarised trajectories excluding the noise Hypothesis: one of the routes (red) is direct; the others are used for visiting certain places like shops. 70

The red route (60 trips) Visual Analytics example The red route takes less time than the others. Sometimes, the trips start quite late in the evening while the latest of the remaining trips starts at 19:22. Here the time was spent Stops are rare and rather short; the longest stops (just 2) were in a place not far from home on route prolongation 71

The other ways home Visual Analytics example Our hypothesis is supported: the red route work-home is direct; on the other routes, the person makes intermediate stops. On the way from the work to home, the person often visits the shopping centres. The route through shopping centre 2 (green) is preferred, but not always the person stops there (see min duration of stay). 72

Visual Analytics example The trips through the shopping centres Is there any regularity in preferring one shopping centre to the other? Weekly distribution of the trips Start times and durations Red: trips through both shopping centres On the basis of the clustering results, we have interactively classified the trips according to the visits of the shopping centres. 1) The trips through the shopping centres occur more often on Wednesdays and Thursdays. 2) Shopping centre 1 is usually visited on Mondays, Thursdays, and Fridays. 3) The starting times and durations of the trips through shopping centres 1 and 2 do not significantly differ. 73

The use of 3D view (Google Earth) to analyse trajectories Two clusters of trips from home to work by the direct route differing in dynamics Yellow: faster trips; cyan blue: slower trips The clusters obtained using the distance function route + dynamics 74

Comparison of speed dynamics in 3D view (Google Earth) 75

What we have learned about the car owner: The places where the person lives, works, and shops The typical routes from home to work, from work to home, to the shopping areas The places where the person frequently stops on the way from work to home The places where the person may stop on the way from home to work The durations of the stops, times spent for visiting the shopping areas The times of the trips and of the stops How the chosen routes are distributed over the days of the week The tools we used: Database processing Interactive clustering Computational generalisation and summarisation Interactive displays Dynamic filtering Visual analytics toolkit 76

What we also have learned but haven t noted explicitly: Visual Analytics example The car owner lives in a small town (found using Google Earth and Google Map) The person has a flexible work schedule (concluded from the variation of the times of the trips to and from the work) The person has no small children (concluded from the times of the trips from the work to home) The person is often away or sick (judging from the distribution of the trips over the time period, especially in the summer) green: home to work red: work to home blue: home home 77

Reconstruction of the analytical process raw data (~90K records <x,y,t>) Database processing enhanced data (~90K records <x,y,t,dt,dx,dy, speed >) Interactive UI + Database processing Locations of stops with variable t-resolution aggregated trips Interactive UI + geocomputations Extracted trips (~500) Interactive UI + Database processing Cluster analysis + visualization + Google Earth (additional data!) Clusters of locations, their interpretation Cluster analysis + visualization and interactive analysis tools + Google Earth Typical trips, their interpretation and analysis Algorithms, other details, one more data set: G.Andrienko, N.Andrienko, S.Wrobel Visual Analytics Tools for Analysis of Movement Data ACM SIGKDD Explorations, v.9(2), December 2007 78

Visual Analytics: Where should we go? Some questions about the last example: Where is the knowledge we have gained? 1) In our heads! - Do we remember everything we have learned and how we have come to it? - Shall we remember it in a few days? In a few weeks? 2) In this presentation! - It took us several days to make it How often can analysts afford this? How could we conveniently review and systematise what we have learned? - It might be useful to compare different portions of knowledge gained independently of each other (e.g. behaviour on work days and on weekends) - Further inferences could be made and new insights gained through combining known facts (putting known facts together is essential for a good detective work!) How could such analyses (with more complex data) be made by several analysts? - What should be the result of their joint effort? - What if they disagree in some points? 80

Visual Analytics: Where should we go? The needs Explicit and easily manageable representation of the knowledge gained from analysis Explicit representation of how the knowledge was obtained (observations, inferences, involvement of background knowledge and additional information from other sources) Explicit representation of assumptions, confidences, conditional judgements, inconsistencies, competing hypotheses, alternative interpretations A framework for knowledge systemisation and establishing relations between knowledge items A software tool that supports - capture of observations, knowledge, and argumentation (analytical artefacts) in the process of analysis but without disrupting the process! - storage of analytical artefacts and relations between them (knowledge base) - browsing and searching in the knowledge base - editing and enrichment of the knowledge base - producing reports about the analysis and explanations of the conclusions made 81

Visual Analytics: Where should we go? Can AI research help? Artificial Intelligence Knowledge representation Knowledge capture Reasoning Uncertainty and confidence Learning Explanation Visual Analytics AI seems to have what VA needs! But is it fully suitable and sufficient? 82

Specifics of knowledge and reasoning in VA (why existing methods and software would be inappropriate) Specific knowledge types: observations, abstractions, patterns, structures, Analytical artefacts are tightly connected to visual representations The visual representations need to be captured and stored together with the analytical artefacts Analytical artefacts may refer to geographical space and/or time These references also need to be captured and stored The knowledge is built dynamically rather than exists in advance Visual Analytics: Where should we go? Knowledge capture needs to be seamlessly integrated into the software for analysis - Analysts should concentrate on the analysis but not on knowledge transfer (could standard knowledge engineering approaches be suitable here?) The knowledge representation needs to be human-oriented rather than machineoriented 83

A great challenge: non-disruptive knowledge capture Visual Analytics: Where should we go? Analysts need an intelligent software assistant that could capture the artefacts emerging in the process of analysis without disrupting this process of analysis - Ideally, the assistant should be invisible and require no attention at all Questions: Too ambitious? Then we should at least try to reduce the assistant s opacity to an acceptable level. - E.g. suggestion of appropriate templates at appropriate time moments Can the machine understand what the analyst is doing? Can the machine see (guess) what the analyst sees? Can the machine foresee what kinds of analytical artefacts - are possible for given data? - can result from using given methods or tools? Let s review what we had in our example analysis of movement 84

Examples of analytical artefacts Visual Analytics: Where should we go? Positions of stops: elementary objects Clusters of positions: aggregates Places as areas: abstractions Trajectory: elementary object Clusters of trajectories: aggregates Typical route: abstraction 85

Examples of analytical artefacts (cont.) Visual Analytics: Where should we go? Distribution over a time period, within a time cycle: behaviour (temporal) The positions are clustered (form spatially close groups ): pattern Blue is mostly in the morning, red is mostly in the evening: patterns Red: home, blue: work Distribution in space: behaviour (spatial) inference 86

Examples of analytical artefacts (cont.) Visual Analytics: Where should we go? Frequency distribution of routes and moves: behaviour outliers: pattern Statistical distribution of characteristics: behaviour The eastern route is chosen much more often: pattern 87

Examples of analytical artefacts (cont.) Visual Analytics: Where should we go? Green & light blue: probably shopping hypothesis basis for the hypothesis evidence evidence 88

Examples of analytical artefacts (cont.) Visual Analytics: Where should we go? other town parking interpretations shopping centre 1 home shopping centre 2 Positioning errors work sports place 89

Examples of analytical artefacts (cont.) Visual Analytics: Where should we go? Route with stops: composite object Places, frequency, durations, temporal sequence of stops: behaviour Differences and similarities direct have intermediate stops On this route stops are not usual: pattern (structural) 6 of 7 trips along this route visited shopping centre 1: pattern 90

Could (some of) these kinds of artefacts be theoretically predicted? Movement data: <time> <positions in space>, where time is an independent component and space is a dependent component Time (set of time moments): continuous; has distances; linearly ordered; may be cyclically ordered Space (set of positions): continuous; has distances; partly ordered w.r.t. selected positions and directions Abstractions: point in space place (area in 2D or volume in 3D); moment in time interval Elementary object: [<specific time moment> <specific position>] Visual Analytics: Where should we go? Space has distances positions can be close or distant in space sets of positions may be spatially dispersed or clustered (possible patterns of spatial behaviour) Positions depend on time an analyst may be interested in the distribution of selected positions (e.g. starts or ends of trips) over the time span Time may be cyclically ordered an analyst may be interested in the distribution of selected positions within the temporal cycles Possible patterns of temporal distribution: even distribution; tendency towards certain moments or intervals within the time span or within a cycle 91

Predicting kinds of artefacts for movement data (cont.) Visual Analytics: Where should we go? Positions depend on time; time is linearly ordered positions form sequences Time is continuous there is a position (possibly unknown) for any time moment positions form a continuous sequence Time divided into intervals a sequence of positions divided into sub-sequences Abstraction: continuous (sub-)sequence of time-referenced positions trajectory (start position + end position + course + time interval) treated as unit, or elementary object Trajectories may be compared regarding one or more of their characteristics (i.e. start, end, course, and time interval) and grouped (clustered) by similarity Abstraction: multiple trajectories with similar courses typical route A typical analysis task: study the statistical (frequency) distribution of characteristics over a set of objects (such as trajectories in our case) Possible patterns of statistical distribution: even distribution; prevalence of certain characteristics; outliers Movement data places and trajectories (independent abstractions) can be put together - Places may be characterised in terms of trajectories passing them - Trajectories may be characterised in terms of places they pass and so on 92

Visual Analytics: Where should we go? Predicting kinds of artefacts for analytical tools and methods Map: positions; courses; patterns of spatial distribution Graph or plot with an axis representing time: time moments or intervals; patterns of temporal distribution Histogram, scatter plot: patterns of statistical distribution Temporal histogram: patterns of temporal distribution Clustering: groups of objects with similar characteristics Spatial clustering: groups of objects close in space and so on 93

Sequel Visual Analytics: Where should we go? Questions: Can the machine understand what the analyst is doing? Can the machine see (guess) what the analyst sees? Can the machine foresee what kinds of reasoning artefacts - are possible for given data? - can result from using given methods or tools? The positive answers seem to be quite attainable! But not without efforts 94

Conclusion Visual Analytics: Where should we go? Visual Analytics is conceived as a field of multidisciplinary research Researchers in Artificial Intelligence have valuable expertise and experience and could greatly contribute to the development of Visual Analytics The major research directions: - Typology of analytical artefacts - Representation methods for analytical artefacts - Interpretation of activities of a human analyst - Recognition of analytical artefacts - Systemisation and management of analytical artefacts Would you be interested to join? 95

Spatial Decision Support Computerised assistance to people in solving problems having geographical or spatial components: - develop, evaluate, and select - proper policies, plans, scenarios, projects, interventions E.g. long-term decision-making: - planning for sustainable places - mitigating hazards - infrastructure management - strategic business planning E.g. short-term decision making - resource logistics - emergency response 97

Actors in decision making processes Decision maker (e.g. administrator or politician) Analyst: knows the problem domain; skilled in (computational) analysis and/or modelling Stakeholder: can profit or lose from the decisions made wants to be informed and involved 98

Operations Research Creates methods to automatically develop, evaluate, and select proper variants of problem solutions when the problem, relevant information, and criteria are represented in a suitable form (numbers, formulas, logical expressions, ) Hmm This does not seem to be always possible! 99

Complexities Most spatial decision problems are inherently ill-defined (the goals and/or criteria are vague, incomplete, unquantifiable ) Geographic space is heterogeneous, anisotropic, continuous; multitude of objects, phenomena, relations; scale-dependency No adequate formal representation exists yet! Tacit knowledge and criteria play especially important role (e.g. knowledge of geographic area or general knowledge about space, objects, and relationships) 100

Approach Synergy: I cannot put my knowledge in it But I can try to evaluate possible solutions and choose the best one. Can it show me the solutions? Vehicle Vehicle Vehicle home base Source Source name Destination Destination name Start End Item Item class name class Operation ID ID ID class Number ID of items 20 Pick Up 41 Children clinics 75 Children clinics 60 Braun and Co 00:00:00 00:00:04 0 EMPTY 1 20 Deliver 41 Children clinics 75 Children clinics 60 Braun and Co 00:00:00 00:00:04 0 EMPTY 1 20 Pick Up 41 Children clinics 60 Braun and Co 68 St. John Hospital 00:00:04 00:00:30 21 invalids who cannot seat 2 20 Deliver 41 Children clinics 60 Braun and Co 68 St. John Hospital 00:00:04 00:00:30 21 invalids who cannot seat 2 20 Pick Up 41 Children clinics 68 St. John Hospital 22 St. Peter Hospital 00:00:51 00:00:54 0 EMPTY 1 20 Deliver 41 Children clinics 68 St. John Hospital 22 St. Peter Hospital 00:00:51 00:00:54 0 EMPTY 1 20 Pick Up 41 Children clinics 22 St. Peter Hospital 40 Spa healing house 00:00:54 00:01:22 21 invalids who cannot seat 2 20 Deliver 41 Children clinics 22 St. Peter Hospital 40 Spa healing house 00:00:54 00:01:22 21 invalids who cannot seat 2 10 Pick Up 102 City coach park 109 City coach park 60 Braun and Co 00:00:00 00:00:05 0 EMPTY 1 10 Deliver 102 City coach park 109 City coach park 60 Braun and Co 00:00:00 00:00:05 0 EMPTY 1 10 Pick Up 102 City coach park 60 Braun and Co 50 Exhibition hall 00:00:05 00:00:18 10 general people or children 50 10 Deliver 102 City coach park 60 Braun and Co 50 Exhibition hall 00:00:05 00:00:18 10 general people or children 50 10 Pick Up 102 City coach park 50 Exhibition hall 32 Kindergarten 00:00:28 00:00:34 0 EMPTY 1 10 Deliver 102 City coach park 50 Exhibition hall 32 Kindergarten 00:00:28 00:00:34 0 EMPTY 1 10 Pick Up 102 City coach park 32 Kindergarten 41 Descartes School 00:00:34 00:00:51 10 general people or children 50 10 Deliver 102 City coach park 32 Kindergarten 41 Descartes School 00:00:34 00:00:51 10 general people or children 50 10 Pick Up 48 City coach park 109 City coach park 60 Braun and Co 00:00:00 00:00:05 0 EMPTY 1 10 Deliver 48 City coach park 109 City coach park 60 Braun and Co 00:00:00 00:00:05 0 EMPTY 1 10 Pick Up 48 City coach park 60 Braun and Co 49 Leonardo School 00:00:05 00:00:18 10 general people or children 50 10 Deliver 48 City coach park 60 Braun and Co 49 Leonardo School 00:00:05 00:00:18 10 general people or children 50 10 Pick Up 48 City coach park 49 Leonardo School 5 Frings Gymnasium 00:00:28 00:00:33 0 EMPTY 1 10 Deliver 48 City coach park 49 Leonardo School 5 Frings Gymnasium 00:00:28 00:00:33 0 EMPTY 1 10 Pick Up 48 City coach park 5 Frings Gymnasium 41 Descartes School 00:00:33 00:00:48 10 general people or children 50 10 Deliver 48 City coach park 5 Frings Gymnasium 41 Descartes School 00:00:33 00:00:48 10 general people or children 50 10 Pick Up 78 City coach park 109 City coach park 21 Kindergarten 00:00:00 00:00:05 0 EMPTY 1 10 Deliver 78 City coach park 109 City coach park 21 Kindergarten 00:00:00 00:00:05 0 EMPTY 1 10 Pick Up 78 City coach park 21 Kindergarten 50 Exhibition hall 00:00:05 00:00:19 10 general people or children 20 10 Deliver 78 City coach park 21 Kindergarten 50 Exhibition hall 00:00:05 00:00:19 10 general people or children 20 10 Pick Up 78 City coach park 50 Exhibition hall 18 Albert College 00:00:29 00:00:35 0 EMPTY 1 10 Deliver 78 City coach park 50 Exhibition hall 18 Albert College 00:00:29 00:00:35 0 EMPTY 1 10 Pick Up 78 City coach park 18 Albert College 42 Riverside hall 00:00:35 00:00:52 10 general people or children 50 10 Deliver 78 City coach park 18 Albert College 42 Riverside hall 00:00:35 00:00:52 10 general people or children 50 12 Pick Up 117 Bus travel company 110 Bus travel company 60 Braun and Co 00:00:00 00:00:06 0 EMPTY 1 12 Deliver 117 Bus travel company 110 Bus travel company 60 Braun and Co 00:00:00 00:00:06 0 EMPTY 1 12 Pick Up 117 Bus travel company 60 Braun and Co 42 Riverside hall 00:00:06 00:00:23 10 general people or children 100 12 Deliver 117 Bus travel company 60 Braun and Co 42 Riverside hall 00:00:06 00:00:23 10 general people or children 100 What? A single solution is a table with hundreds of lines? How can I review and evaluate it? A solution must be put in a visual form! 101

Role of visualization Visualization and interactive visual interfaces can effectively convey material for human s analysis and reasoning It s not my job. A visualization specialist must care about displaying the solutions which my algorithm produces. OK. Let me try But it s not so easy! 102

Visualization of a solution A solution of a spatial decision problem is often a complex spatio-temporal construct - E.g. an action plan where actions refer to different positions, regions, or paths in space and to different moments or intervals in time Retail outlet distribution: where, when, and what outlet should be opened, closed, or modified (size, profile, ) Groundwater management: how much water can be taken from each well in different time periods; when and where to restrict water withdrawals People evacuation (item transportation) scheduling: what people (items), from where, to where, how many, when, by what transport 103

Visualization of a solution (cont.) This can only be done by means of multiple coordinated views and interactive query tools 104

Does it solve the problem? I can try to learn how to use all these displays and interactive tools to analyze a single solution. But Is it realistic to compare all possible solutions for choosing the best one? It seems so hard even with two variants What if they are tens or hundreds? And when I am not fully satisfied, how can I tell the algorithm what must be improved? And would it regard my opinion?? Could we propose something better than just a display of end results of algorithms? 105

Enable a true synergy The human analyst should be actively involved in the process of solution generation or search, i.e. cooperate with the computer - Contribute with intangible knowledge and experience - Actively guide the work of algorithms - Effectively reduce the search scope - Improve the quality of solutions Interactive visual interfaces are essential during the whole process as a means of communication between computer and human 106

Enable a true synergy (cont.) Requires fundamentally new decision support methods: computational analysis modeling visualization interaction knowledge synthesis reasoning organically integrated Such methods can only result from cross-disciplinary research It seems that we ll have to work together 107

Regard the needs of different actors I need to see what each option means for me personally (my family, my company, my town ). And I have quite valuable local knowledge that should be taken into account. I want just a few clear and easily comparable options. But I want to understand where they come from and what tradeoffs involve. Besides, what if some assumptions are wrong or the situation changes? Would it be possible to put selected solutions in a form of simple visuointeractive models these guys could play with? Could such models be customizable? 108

Visual Analytics for Spatial Decision Support Cross-disciplinary research Looks for ways to provide computer support to solving space-related decision problems through enhancing human capabilities to analyze, envision, reason, and deliberate Addresses all phases of the decision making process - Initial exploration, generation of solution options, evaluation, choice Addresses the needs of various actors Bases on achievements in geovisualization, information visualization, human-computer interaction, geographic information science, operations research, data mining and machine learning, decision science, 109

Must deal with Huge amounts of source data, high dimensionality, extensive search spaces Complex, heterogeneous information (geography, time, objects, attributes, actions, relations, effects, ) Uncertain, incomplete, noisy information Time-critical decision making (e.g. emergency evacuation) Adaptive real-time decision making: monitoring of decision implementation in an evolving situation, assessment of effectiveness, revision 110

Conclusion Spatial decision support requires multi-disciplinary approaches for creating adequate tools and making theoretical advances So far, there has been limited synthesis and cooperation between different research communities Visual Analytics for Spatial Decision Support is a new concept designed to attract the attention of scientists with relevant expertise promote the development of cross-disciplinary research capable to meet the challenges of the real-world decision making 111

Visual analytics resources Visual Analytics Digital Library: http://vadl.cc.gatech.edu/ InfoVis Community Platform: http://www.infovis-wiki.net/ Software - Many tools from HCIL lead by B.Shneiderman: http://www.cs.umd.edu/hcil/ (TimeSearcher and others, also extensive archive of publications) - InfoVis toolkit by J.-D.Fekete / INRIA: http://ivtk.sourceforge.net/ - Prefuse toolkit by J.Heer / UC Berkeley: http://prefuse.org/ - InfoVis CyberInfrastructure by K.Boerner / Univ. Indiana: http://iv.slis.indiana.edu/sw/index.html - GeoVista Studio from Penn State lead by A.MacEachren: http://www.geovistastudio.psu.edu/jsp/index.jsp - GAV Framework from Linkoping University lead by M.Jern: http://servus.itn.liu.se/projects/geowizard/ - Geospatial Visual Analytics toolkit based on Iris, Descartes, CommonGIS from Fraunhofer IAIS: http://www.commongis.com 113

http://geoanalytics.net welcome to join our activities! GIScience 2006 workshop outcomes: Special issue on GeoVisual Analytics for Spatial Decision Support, including Setting the Research Agenda paper Int.J.GIScience, 2007, v.21(8) AGILE 2008 workshop (May 2008, Girona, Spain) GeoVisualization of Dynamics, Movement and Change forthcoming special issue of Information Visualization 114