Introduction to Talking Robots. Graham Wilcock Adjunct Professor, Docent Emeritus University of Helsinki

Transcription

1 Introduction to Talking Robots Graham Wilcock Adjunct Professor, Docent Emeritus University of Helsinki

2 Robot Gestures and Emotions Graham Wilcock (Thanks for slides by Prof. Kristiina Jokinen)

3 Gestures and Interaction Natural conversational interaction How to give feedback? How to take the turn and give the turn? Humans use gestures to structure the interaction

4 Multimodal Feedback Most frequent feedback is vocal (by voice) Words or non-verbal sounds Most frequent gestures in feedback: Head movements (nods, shakes, tilts ) Facial expressions (not gestures ) Smile, raising eyebrows, Hand gestures Body posture

5 Nodding and Feedback Down nods: Acknowledge expected information Encourage partner to continue Up nods: Acknowledge unexpected information Express surprise In Japan? そうですか? ほんとうに?

6 Nordic Nodding: Finland, Sweden, Denmark Single nod vs. repeated nods Swedes and Danes use repeated nods more than Finns Finns tend to use slow, single nods Up-nods vs. down-nods Swedes use up-nods more than Finns and Danes Feedback in Nordic First-Encounters: a Comparative Study C. Navarretta, E. Ahlsen, J. Allwood, K. Jokinen and P. Paggio, LREC 2012, Istanbul

7 Hand Gestures in Communication Complement the speech (pointing) That is my old school Illustrate the speech (iconic) The increase is small Give rhythm to the speech (batonic, beats) He had three children: Susanna, and twins Hamnet and Judith Conventional, symbolic (Emblems) V for victory

8 Gesture Families (Kendon 1980-) Open hand palm down Express stopping or halting of an action (own or other) Open hand palm up Express offering and giving of ideas and concepts Open hand vertical movement Express cutting, limiting or structuring information Open hand lateral movement Express withdrawal or ignorance, abandoning the task Index finger extended (pointing gestures) Express ideas that are precise and explicit

9 Communicative Gestures: 1. Standing, 2. Speaking

10 Communicative Gestures: 3. Open hand, palm up

11 Communicative Gestures: 4. Open hand, palm vertical

12 Communicative Gestures: 5. Nod Down, 6. Nod Up

13 Communicative Gestures: 7. Open Arms

14 References for Gestures A. Kendon Gesture: Visible Action as Utterance. Cambridge University Press. C. Navarretta, E. Ahlsén, J. Allwood, K. Jokinen, P. Paggio Feedback in Nordic First-Encounters: a Comparative Study. LREC 2012, Istanbul. R. Meena, K. Jokinen, G. Wilcock Integration of Gestures and Speech in Human-Robot Interaction. CogInfoCom 2012, Kosice

15 Creating New Gestures with Choregraphe Animations Create new gestures as animations Animations examples on Youtube Detailed explanation of one animation (2 mins) How-to showing multiple animations (10 mins)

16 Robots and Emotions Robots do not feel emotions, but Robots can express emotions Gestures, animations, body language Facial expression (Nao eye colours) Robots can recognize emotions Human voice features Other non-verbal cues

17 Expressing Emotions Emotion demo on Youtube Emotion gradients in body posture variation Very happy, happy, sad, very sad Nao animations library Many ready-made emotion animations Surprise, pride, enthusiasm, embarrassment,... Can be used in Dialog Topic scripts

18 Recognizing Emotions Emotion detection from speech LIMSI-CNRS research, Paris (Laurence Devillers) New programming interface on Nao: Voice Emotion Analysis (in NAOqi API) Other non-verbal cues (research topics) Recognizing human faces and facial expressions Recognizing human body language Proximity, engagement,

19 Creating Animations using Timeline in Choregraphe Textbook Chapter 4: Let s Dance /priv/docs/4-LetsDance.pdf Robot raises right arm, raises left arm Turns right palm up, turns left palm up Puts one arm on head, on hip, on back,... (Note: Python parts are optional)

20 Timeline in Choregraphe: Real Robot and Virtual Robot Using a real robot (textbook Chapter 4) Set joints stiffness off Move robot arm to desired position Record position at keyframe on timeline Using a virtual robot Right click on arm to see joints view Move virtual arm to desired position by using slider bars to rotate arm joints Record position at keyframe on timeline