Designing Social Behavior for Humanlike Robots

Transcription

1 Designing Social Behavior for Humanlike Robots Bilge Mutlu Assistant Professor Department of Computer Sciences CS-202 December 7, 2009

2 Humanlike Robots Embody human physical, cognitive, and social features Varying degrees of humanlikeness Aethon TUG Matsushita Hospi Honda ASIMO ATR Geminoid

3 Research Space Human interaction with robots Designing social behavior Verbal and vocal behavior Speech, vocal tone, intonation, prosody, etc. Nonverbal behavior Gaze, gestures, proximity, posture, etc. Verbal and vocal: Speech, vocal tone, intonation, prosody Why design social behavior? Observations of a hospital delivery robot Nonverbal: Gaze, gestures, proximity, posture

4

5 Robot with No Manners Workers found the robot to lack manners Humanlike features need to design appropriate social behavior [the robot] doesn't have the manners that we teach our children and it takes precedence over people most of the time I sort of find it insulting that I stand out of the way for patients or a gurney or a wheelchair coming through, but [the robot] just barrels right on You need get out of the way [for the robot]. I called them nasty names and told them, Would you shut the hell up? Can t you see I m on the phone? I ll get to you. If you say, [robot] has arrived, one more time, I m about to kick you in your camera.

6 Research Question How can we design social behaviors for a humanlike robot? Social scientific Computational Human-computer interaction What social behaviors are important or appropriate? How do we represent and implement these behaviors? How do we get social and cognitive benefits?

7 Research Approach Understand Represent & Implement Evaluate

8 Research Approach S 1 S 2 S 3 Existing theory on social behavior Computational models of social behavior Theoretically based manipulations New understanding of social behavior Understand Implementation of behavioral models on interactive platform Represent & Implement Evaluating the social outcome of human-computer interaction Evaluate

9 Designing Social Gaze Behavior Study I Communicating attention (U.S.) Study II Establishing conversational roles (Japan) Study III Communicating intentions (Japan) Understand Represent & Implement Evaluate

10 Designing Gaze Behavior Communicating attention Humanoids06

11 Research Question Increased teacher gaze leads to improved learning Otteson and Otteson, 1980; Sherwood, 1987 Can a robot increase attention looking at a student more?

12 Understanding Human Behavior People shift their gaze at significant points in speech Kendon, 1967 Gaze away Gaze toward Speech The old woman had made some starch. Look away 70% of the time Look at 73% of the time The old woman had made some starch. Theme Rheme

13 Understanding Human Behavior Human storyteller Storyteller gaze data Addressee 1 Addressee 2 Camera

14 Understanding Human Behavior Addressee 1 Addressee 2

15 Understanding Human Behavior Gaze targets

16 Understanding Human Behavior Environment 4 3 Down Addressee Addressee 2 Target clusters

17 Understanding Human Behavior Time spent 5% Frequency 38% Environment 4 3 Down Time spent 30% Frequency 38% Addressee Addressee 2 Time spent 38% Frequency 13% Time spent 27% Frequency 11% Times spent & frequencies

18 Understanding Human Behavior Gamma k=2.19, θ=0.49 count Environment Addressee count length (ms) Down Gamma k=1.38, θ=1.92 length (ms) 2 1 Gamma k=2.72, θ=0.97 count Gamma k=3.32, θ=0.68 count Addressee 2 length (ms) length (ms) Length distributions

19 Computational Model Down Environment (p = 0.50) (p = 0.50) Look away (p = 0.70) Addressee Addressee 1 2 (p = 0.50) Look at (p = 0.73) (p = 0.50) Down Environment (p = 0.50) (p = 0.50) Look away (p = 1.00) The old woman had made some starch. Theme Rheme

20

21 Evaluation ASIMO told a Japanese fairy tale to two participants ASIMO Between-participants design: 20% of the robot s attention 80% of the robot s attention 20 American participants 12 males, 8 females, aged Addressee 2 Addressee 1 (p = 0.50) (p = 0.50) Addressee 2 Addressee 1 (p = 0.80) (p = 0.20) 20% of total gaze directed at addressees 80% of total gaze directed at addressees Look at Look at Participant 1 Participant 2

22 Hypotheses H.1: Participants who are looked at more will perform better than others Fry and Smith, 1975; Otteson and Otteson, 1980; Sherwood, 1987 H.2: Participants who are looked at more will evaluate ASIMO more positively than others Kleck and Nuessle, 1968; Cook and Smith, 1975 Task performance Positive Evaluation 20% attention 80% attention 20% attention 80% attention

23

24 Results Objective: information recall p=ns Task performance 20% attention Predicted Measured Measured F Measured M F M 80% attention Task Performance % attention p=0.03* 80% attention Task Performance Condition 20% 80% F 20% attention p<0.01* M 80% attention Participant Gender F M F M

25 Results Objective: positive evaluation p=ns Positive Evaluation 20% attention Predicted Measured Measured F Measured M 80% attention Positive Evaluation % attention p=ns 80% attention Positive Evaluation Condition 20% 80% % attention p=0.04* F M F M 80% attention Participant Gender F M

26 Conclusions More robot gaze leads to improved learning

27 Designing Social Gaze Behavior Study I Communicating attention (U.S.) Study II Establishing conversational roles (Japan) Study III Communicating intentions (Japan) Understand Represent & Implement Evaluate

28 Designing Gaze Behavior Regulating conversational roles Speaker Addressee

29 Designing Gaze Behavior Regulating conversational roles Unacknowledged non-participants Acknowledged non-participants Overhearer or Eavesdropper Bystander Active participants All participants Speaker Addressee Side participant Levels of participation Goffman, 1979; Clark, 1996

30 Research Question Gaze cues are instrumental in establishing conversational roles Bales et al., 1951; Schegloff, 1968; Sacks et al., 1974; Goodwin, 1981 How can a robot use these signals? What are these cues?

31 Understanding Human Behavior Conversational mechanisms: Gaze cues that help manage turn-exchanges Gaze cues induced by discourse structure Gaze cues that signal conversational roles Spatial Temporal Turn-exchanges Discourse structure Role-signaling Gaze away Gaze toward A1 Gaze toward A2 Speech S Speech A1 Speech A2

32 Understanding Human Behavior Speaker Speaker Overhearer (behind the screen) Bystander Two-party Conversational structure Addressee Addressee Two-party-with-bystander conversational structure Speaker Addressee Addressee Three-party Conversational structure

33 Understanding Human Behavior Interaction space Speaker Addressee Gaze data for all participants Overhearer/Bystander/Addressee

34 Understanding Human Behavior Conversational mechanisms: Gaze cues that help manage turn-exchanges Gaze cues induced by discourse structure Gaze cues that signal conversational roles Spatial Temporal Turn-exchanges Discourse structure Role-signaling Gaze away Gaze toward A1 Gaze toward A2 Speech S Speech A1 Speech A2

35 Understanding Human Behavior Environment Environment (Floor-holding gaze signal) Milliseconds Environment Addresee 1 s face Addressee 1 s body Addresee 2 s face Addressee 2 s body Speaker gaze behavior Speaker speech Addressee 1 speech Speech Addressee 2 speech Question Answer Minimal response Casual narrative segment Talk Simultaneous (overlapping) talk Backchannel responses

36 Understanding Human Behavior Addressee s face Addressee s face (Turn-yielding gaze signal) Milliseconds Environment Addresee 1 s face Addressee 1 s body Addresee 2 s face Addressee 2 s body Speaker gaze behavior Speaker speech Addressee 1 speech Speech Addressee 2 speech Question Answer Minimal response Casual narrative segment Talk Simultaneous (overlapping) talk Backchannel responses

37 Understanding Human Behavior Addressee s face Addressee s face (Turn-taking gaze signal) Milliseconds Environment Addresee 1 s face Addressee 1 s body Addresee 2 s face Addressee 2 s body Speaker gaze behavior Speaker speech Addressee 1 speech Speech Addressee 2 speech Question Answer Minimal response Casual narrative segment Talk Simultaneous (overlapping) talk Backchannel responses

38 Understanding Human Behavior Environment Environment (Floor-holding gaze signal) Milliseconds Environment Addresee 1 s face Addressee 1 s body Addresee 2 s face Addressee 2 s body Speaker gaze behavior Speaker speech Addressee 1 speech Speech Addressee 2 speech Question Answer Minimal response Casual narrative segment Talk Simultaneous (overlapping) talk Backchannel responses

39 Computational Model Environment Environment (Floor-holding gaze signal) Milliseconds Environment Addresee 1 s face Addressee 1 s body Addresee 2 s face Addressee 2 s body Robot gaze behavior Robot speech Addressee 1 speech Speech Addressee 2 speech Question Answer Minimal response Casual narrative segment

40 Computational Model Addressee s face Addressee s face (Turn-yielding gaze signal) Milliseconds Environment Addresee 1 s face Addressee 1 s body Addresee 2 s face Addressee 2 s body Robot gaze behavior Robot speech Addressee 1 speech Speech Addressee 2 speech Question Answer Minimal response Casual narrative segment

41 Computational Model Addressee s face Addressee s face (Turn-taking gaze signal) Milliseconds Environment Addresee 1 s face Addressee 1 s body Addresee 2 s face Addressee 2 s body Robot gaze behavior Robot speech Addressee 1 speech Speech Addressee 2 speech Question Answer Minimal response Casual narrative segment

42 Computational Model Environment Environment (Floor-holding gaze signal) Milliseconds Environment Addresee 1 s face Addressee 1 s body Addresee 2 s face Addressee 2 s body Robot gaze behavior Robot speech Addressee 1 speech Speech Addressee 2 speech Question Answer Minimal response Casual narrative segment

43 [Addressee s response] Floor-holding signal Turn-yielding signal Turn-taking signal Floor-holding signal Away Toward Robot Addressee Hello. My name is Robovie. What is your name? It is nice to meet you... [Addressee s response] Gaze Speech (Casual) Narrative Question Answer (Casual) Narrative

44 Evaluation Robovie conversed with two participants as a travel agent Robovie Between-participants design: c.1: Signaled the roles of a twoparty conversation c.2: Signaled the roles of a twoparty-with-bystander conversation c.3: Signaled the roles of a threeparty conversation 72 Japanese participants All male pairs, aged Participant roles: c.1: Overhearer c.2: Bystander c.3: Addressee Participant role: c.1,2,3: Addressee (Covariate) Participant 1 Participant 2

45 Hypotheses H.1: Addressees will take more speaking turns and speak longer than bystanders and overhearers do H.2: Addressees will recall the details of the information presented by the robot better than bystanders and overhearers do Number of Turns Taken Task Performance Overhearer Bystander Addressee Overhearer Bystander Addressee Two-party Two-party w/ bystander Three-party Two-party Two-party w/ bystander Three-party

46 Hypotheses H.3: Addressees and bystanders will evaluate the robot more positively than overhearers do H.4: Addressees will express stronger feelings of groupness than bystanders and overhearers do Liking Feelings of Groupness Overhearer Bystander Addressee Overhearer Bystander Addressee Two-party Two-party w/ bystander Three-party Two-party Two-party w/ bystander Three-party

47

48 Results Behavioral: turn-taking Participants conformed to their roles 97% of the time

49 Results Behavioral: turn-taking Number of Turns Taken Overhearer Predicted Measured Bystander Addressee Number of Turns Overhearer (Condition 1) p=ns p<0.01* Bystander (Condition 2) p=0.01* Addressee (Condition 3) Time Spent Speaking (seconds) Overhearer (Condition 1) p=ns p=0.01* Bystander (Condition 2) p=0.03* Addressee (Condition 3) Two-party Two-party w/ bystander Three-party Conversational Role Overhearer Bystander Addressee

50 Results Objective: task performance/attentiveness Predicted Measured p=ns p=ns p=ns p=ns p<0.01* p=0.01* Task Performance Task Performance Task Attentiveness Overhearer Bystander Addressee 0 Overhearer (Condition 1) Bystander (Condition 2) Addressee (Condition 3) 1 Overhearer (Condition 1) Bystander (Condition 2) Addressee (Condition 3) Two-party Two-party w/ bystander Three-party Conversational Role Overhearer Bystander Addressee

51 Results Subjective: liking/rapport p=0.01* p=0.05* p=ns Predicted Measured 7 6 Role Overhearer Liking Liking 5 4 Bystander Addressee 3 2 Overhearer Bystander Addressee 1 Overhearer (Condition 1) Bystander (Condition 2) Addressee (Condition 3) Two-party Two-party w/ bystander Three-party

52 Results Subjective: groupness p=ns p<0.01* p=0.03* Feelings of Groupness Predicted Measured Feelings of Groupness Role Overhearer Bystander Addressee Overhearer Bystander Addressee 1 Overhearer (Condition 1) Bystander (Condition 2) Addressee (Condition 3) Two-party Two-party w/ bystander Three-party

53 Conclusions Two-party conversation Two-party conversation Three-party conversation Bystander Robots can manage the roles of their conversational partners using gaze cues

54 Designing Social Gaze Behavior Study I Communicating attention (U.S.) Study II Establishing conversational roles (Japan) Study III Communicating intentions (Japan) Understand Represent & Implement Evaluate

55 Designing Gaze Behavior Communicating mental states using subtle cues Yes Picker No A game of guessing HRI09 Guesser Is your pick black?

56 Research Question Subtle gaze cues lead to mental state attribution Emery, 2000 Could robots use subtle cues to communicate mental states?

57 Understanding Human Behavior Picker Guesser Picker Participants playing the guessing game Guesser

58

59

60 Understanding Human Behavior Normal (µ=527,σ=127) Shapiro-Wilk W Test, W=0.98, p=.70 count length (ms) Cue length distribution

61 Computational Model Robovie Geminoid Task space (Stationary state) Leakage cue (Robot s pick) Guesser s face (Eye-contact) Milliseconds Guesser s face Robot s pick Task space Robot gaze behavior Robot speech Speech Guesser s speech Guesser s question Gaze travel Leakage cue Gaze travel Robot s answer

62 Geminoid, without the cue

63 Geminoid, with the cue

64 Robovie, without the cue

65 Robovie, with the cue

66 Evaluation Guessing game with the robots Robovie Geminoid Robots, picker; participants, guesser Played 8 rounds of game Between-participants: Geminoid vs. Robovie Within-participants: Glance vs. not glance at the pick (4 of 8 rounds each) 26 Japanese participants 17 males, 9 females; aged Participant

67 Hypotheses H.1: Participants will perform better when the robot leaks information H.2: The effect will be stronger with Geminoid than with Robovie Task Performance (time/number of questions) Lower values indicate better performance Task Performance (time/number of questions) Geminoid Robovie No Leakage Cue Leakage Cue No Leakage Cue Leakage Cue

68

69

70 Results Hypothesis I Lower values indicate better performance p=0.04* p=0.02* Task Performance (time/number of questions) Predicted Measured Number of Questions Time (secs) No Leakage Cue Leakage Cue 0 No Gaze Cue Gaze Cue 0 No Gaze Cue Gaze Cue Population No Gaze Cue Gaze Cue

71 Results Hypothesis II Task Performance (time/number of questions) Geminoid Predicted Robovie Predicted Geminoid Measured Robovie Measured Time (secs) p=0.05* p=ns Reported Gaze Cues p<0.01* Yes Yes No No No Leakage Cue Leakage Cue 0 No Gaze Cue Geminoid Gaze Cue No Gaze Cue Robovie Gaze Cue 0.00 Geminoid Robovie Population No Gaze Cue Gaze Cue Population No Yes

72

73 Conclusions Verbal and vocal: Speech, vocal tone, intonation, prosody Subtle social cues can communicate intentions Nonverbal: Gaze, gestures, proximity, posture

74 High-level Conclusions Computational models of human behavior Oratorial gaze Conversational gaze (turn-taking, role-signaling cues, gaze patterns induced by discourse structure, etc.) Subtle leakage cues Social and cognitive impact Better information recall Higher conversational participation Heightened task attentiveness Stronger rapport and feelings of groupness Better task performance led by stronger interpretations of mental states

75 Current Work Applications Learning and games Diagnosis and therapy Collaborative Interfaces Entertainment

76

77 If You Are Interested Take my courses Undergraduate course Introduction to Human-Computer Interaction Graduate course Human-Computer Interaction Join my lab The Human-Computer Interaction Lab More information