Introduction to Talking Robots Graham Wilcock Adjunct Professor, Docent Emeritus University of Helsinki 25.11.2016 1
Robot Gestures and Emotions Graham Wilcock (Thanks for slides by Prof. Kristiina Jokinen) 25.11.2016 2
Gestures and Interaction Natural conversational interaction How to give feedback? How to take the turn and give the turn? Humans use gestures to structure the interaction 25.11.2016 3
Multimodal Feedback Most frequent feedback is vocal (by voice) Words or non-verbal sounds Most frequent gestures in feedback: Head movements (nods, shakes, tilts ) Facial expressions (not gestures ) Smile, raising eyebrows, Hand gestures Body posture 25.11.2016 4
Nodding and Feedback Down nods: Acknowledge expected information Encourage partner to continue Up nods: Acknowledge unexpected information Express surprise In Japan? そうですか? ほんとうに? 25.11.2016 5
Nordic Nodding: Finland, Sweden, Denmark Single nod vs. repeated nods Swedes and Danes use repeated nods more than Finns Finns tend to use slow, single nods Up-nods vs. down-nods Swedes use up-nods more than Finns and Danes Feedback in Nordic First-Encounters: a Comparative Study C. Navarretta, E. Ahlsen, J. Allwood, K. Jokinen and P. Paggio, LREC 2012, Istanbul 25.11.2016 6
Hand Gestures in Communication Complement the speech (pointing) That is my old school Illustrate the speech (iconic) The increase is small Give rhythm to the speech (batonic, beats) He had three children: Susanna, and twins Hamnet and Judith Conventional, symbolic (Emblems) V for victory 25.11.2016 7
Gesture Families (Kendon 1980-) Open hand palm down Express stopping or halting of an action (own or other) Open hand palm up Express offering and giving of ideas and concepts Open hand vertical movement Express cutting, limiting or structuring information Open hand lateral movement Express withdrawal or ignorance, abandoning the task Index finger extended (pointing gestures) Express ideas that are precise and explicit 25.11.2016 8
Communicative Gestures: 1. Standing, 2. Speaking 25.11.2016 9
Communicative Gestures: 3. Open hand, palm up 25.11.2016 10
Communicative Gestures: 4. Open hand, palm vertical 25.11.2016 11
Communicative Gestures: 5. Nod Down, 6. Nod Up 25.11.2016 12
Communicative Gestures: 7. Open Arms 25.11.2016 13
References for Gestures A. Kendon 2004. Gesture: Visible Action as Utterance. Cambridge University Press. C. Navarretta, E. Ahlsén, J. Allwood, K. Jokinen, P. Paggio 2012. Feedback in Nordic First-Encounters: a Comparative Study. LREC 2012, Istanbul. R. Meena, K. Jokinen, G. Wilcock 2012. Integration of Gestures and Speech in Human-Robot Interaction. CogInfoCom 2012, Kosice. 25.11.2016 14
Creating New Gestures with Choregraphe Animations Create new gestures as animations Animations examples on Youtube Detailed explanation of one animation (2 mins) https://www.youtube.com/watch?v=xuzcp3eaiby How-to showing multiple animations (10 mins) https://www.youtube.com/watch?v=u8uaymkmzty 25.11.2016 15
Robots and Emotions Robots do not feel emotions, but Robots can express emotions Gestures, animations, body language Facial expression (Nao eye colours) Robots can recognize emotions Human voice features Other non-verbal cues 25.11.2016 16
Expressing Emotions Emotion demo on Youtube Emotion gradients in body posture variation https://www.youtube.com/watch?v=gl1avfhdsh4 Very happy, happy, sad, very sad Nao animations library Many ready-made emotion animations Surprise, pride, enthusiasm, embarrassment,... Can be used in Dialog Topic scripts 25.11.2016 17
Recognizing Emotions Emotion detection from speech LIMSI-CNRS research, Paris (Laurence Devillers) https://www.youtube.com/watch?v=p1id-gvunws New programming interface on Nao: Voice Emotion Analysis (in NAOqi API) Other non-verbal cues (research topics) Recognizing human faces and facial expressions Recognizing human body language Proximity, engagement,... 25.11.2016 18
Creating Animations using Timeline in Choregraphe Textbook Chapter 4: Let s Dance http://www.ling.helsinki.fi/~gwilcock/doshisha- 2016/priv/docs/4-LetsDance.pdf Robot raises right arm, raises left arm Turns right palm up, turns left palm up Puts one arm on head, on hip, on back,... (Note: Python parts are optional) 25.11.2016 19
Timeline in Choregraphe: Real Robot and Virtual Robot Using a real robot (textbook Chapter 4) Set joints stiffness off Move robot arm to desired position Record position at keyframe on timeline Using a virtual robot Right click on arm to see joints view Move virtual arm to desired position by using slider bars to rotate arm joints Record position at keyframe on timeline 25.11.2016 20