Improving Conversational Interaction for Intelligent Tutoring Systems Tanner Jackson, Heather Hite Mitchell, Art Graesser, and Max Louwerse University of Memphis
What is AutoTutor? An intelligent tutoring system that incorporates teaching strategies modeled after human tutors. Originally designed to teach computer literacy in three topic areas (hardware, operating systems, and the internet). The redesigned, web-enabled versions include tutoring for computer literacy and conceptual physics.
What does AutoTutor do? Asks questions and presents problems Comprehends student answers (keyboard, not speech) Gives feedback on student answers Agent displays emotions and conversational responses Corrects bugs and misconceptions Adds information that is missed Provides the student with hints Prompts for specific information Answers student questions Manages mixed-initiative tutorial dialog
AutoTutor Language Extraction Topic/Problem Selection Speech act classifier Dialog Management Latent Semantic Analysis Curriculum Script Animated Agent
How does AutoTutor Interact? Makes use of the Dialog Advancer Network (DAN) Used to improve the structure of conversational turntaking during the tutoring sessions. Determines appropriate thresholds for the Discourse Markers and Short Feedback. Uses Discourse Markers to improve the smoothness and naturalness of conversation. Uses an Animated Agent to display selected Discourse Markers and Short Feedback, as well as emphasize important aspects in a topic.
What is a discourse marker? Discourse Markers (Schiffrin, 1987) devices for marking transition points in discourse Cue Words (Grosz & Sidner, 1986; Knott, 1996) devices cueing hearer to a change in discourse structure Discourse Operators (Polanyi & Scha, 1985) Devices marking movement between two discourse units Cohesion Relations (Louwerse, 2001) markers cueing comprehenders how to build a mental representation
DM s: Previous Versions of AutoTutor DM were generated intuitively Transition DM: randomly selected from small bags of words or phrases e.g. and, furthermore, moving on Feedback DM : used to serve a pedagogical function Only 21 markers covering Positive, Neutral, and Negative feedback
Toward a Data-driven Taxonomy: Method Substitution test (Knott, 1996) Synonyms (x and y replace each other) Hypernyms (x replaces y but not vice versa) Hyponyms (y replaced by x but can t replace x) yes oh yeah yeah alright no doubt that s nice yes no way
Toward a Data-driven Taxonomy: Materials Santa Barbara Corpus 14 natural dialogs 7557 turns with 3125 DM tokens & 170 types Additional DM Penn Treebank Project CELEX database Other studies
Toward a Data-driven Taxonomy: Procedure 196 start-of-turn discourse markers Markers start of 3817 turns, yielding 748132 needed substitutions Formula used: (5 substitutions per DM) + (5% of total DM freq) Resulted in 747 tokens and 145665 substitutions Two trained grad students (Alpha =.797)
Initial Results apparently that's right Kind of I know yeah No doubt Of course exactly Oh that's right Oh yes yep yes that's true Oh yeah unhunh mhm basically since hell right That's fine because and man definitely sure cause In fact
4 Categories of Data-driven 1) DIRECTION backward & forward Taxonomy 2) CONTINUITY continuous, neutral & discontinuous 3) ACKNOWLEDGEMENT acknowledge & part. acknowledge 4) EMPHATICS emphatic & non-emphatic
DIRECTION BACKWARD relations - relate to previous utterance but does not require any further communicative act - AutoTutor s feedback - (e.g. okay) FORWARD relations - relate to the previous utterance but requires a further communicative act in addition to the relation. - (e.g. In addition) - AutoTutor s Transition Discourse Markers
CONTINUITY CONTINUOUS relations: follow up the response expected by the speaker (e.g. no doubt) DISCONTINUOUS relations: violate the response expected by the speaker (e.g. Oh no) NEUTRAL relations: don t explicitly follow the implicature of the speech act, but do not violate it either. (e.g. I guess)
Continuous Relations really like Oh yeah definitely exactly talking about n'kay surely You bet I ll bet n'yes I bet There you go that's nice there you have it course Oh good I know mhm That's fine No doubt
Continuous DM in AutoTutor TUTOR: STUDENT: TUTOR: TUTOR: STUDENT: TUTOR: TUTOR: TUTOR: Which container will accelerate less? the one with the larger mass good Now, When the same force is applied to both containers, the larger mass will accelerate? less right accelerate less Once again, Since mass times acceleration is equal to force, the larger mass will accelerate less for the same force.
Discontinuous Relations Oh no oh shit uh-oh Too bad man shit oh great ugh no oh gosh Oh man oh dear jeez ew gee tough oh boy yuck dang hell
Discontinuous DM in AutoTutor TUTOR: TUTOR: STUDENT: TUTOR: TUTOR: TUTOR: What's more, So what about If the man speeds up, will the egg land behind the man a little bit? behind the man a lot? next to the man? or in front of the man? behind the man a little bit not really That is not right. next to the man
Neutral Relations just maybe ooo look possibly on the other hand I guess so then otherwise possibly
ACKNOWLEDGMENT ACKNOWLEDGMENT relations: fully acknowledge the implicature of the utterance expressed by the speaker e.g. alright PART-ACKNOWLEDGMENT relations: acknowledge the general implicature of the speech act expressed by the speaker, but add some comments to it e.g. but then again
EMPHATICS EMPHATIC relations: emphasize the continuity or discontinuity e.g. Bravo, Nonsense NON-EMPHATIC relations: dispassionate about the continuity or discontinuity e.g. hm
Improved Discourse markers in AutoTutor Bags of discourse markers now based on taxonomy continuous - neutral - discontinuous LSA scores determine choice of marker Conversational dialog!
Animated Conversational Agent
Animated Conversational Agent Purpose To aid in conversational interaction To display appropriate feedback Design Created in Poser 4, by Curious Labs Compiled using ACE Displayed using MS Agent Selection Discourse Markers Short Feedback Back Channel Feedback
Animated Conversational Agent Discourse Markers right speaks phrase while nodding head nope speaks phrase while shaking head kind of speaks phrase and tilts head Short Feedback Pumps Hints Prompts Normal Conversational Dialog Discusses the topics Used to explain pictures / images
Animated DM Examples Yeah Big Yeah Hesitant Yeah Yeah with Proclivity
Conversational Behaviors Head Movements Facial Expressions Intonation Gaze Behaviors Hand Gestures Blinking Patterns
Head Movements Nodding/Shaking Behaviors Short feedback dialog moves Back-channel feedback cues Complex dialog moves Scripted in the introduction
Facial Expressions Convey affective states appropriate for tutoring Short feedback dialog moves Other discourse functions: Emphasis Replace utterances Agreement
Intonation Intonation parameters Rate Pitch Pause duration Word Emphasis Some dialog moves require discourse sensitive cues
Gaze Behaviors Primarily used to regulate turn taking Gazes at student: during longer tutor turns when relinquishing the floor during shorter tutor turns when answering questions Looks away from student: at grammatical boundaries when asking a question when graphical displays appear
Gestures Hand Gestures Deictic gestures Attentional gestures Skeptical gestures Blinking Can serve as conversational signals Accentuate a word Denote a pause
AutoTutor By incorporating these animated behaviors into the delivery of the new discourse markers we are making another step toward improving the overall interaction with tutoring systems. http://www.psyc.memphis.edu/autotutor.html
Funding Agencies This research was supported by grants from the National Science Foundation (SBR 9720314 and REC 0106965) and the Department of Defense Multidisciplinary University Research Initiative (MURI) administered by the Office of Naval Research under grant N00014-00-1-0600. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of ONR or NSF.