Instructions for the Preparation of a

Socially Intelligent Learner-Agent Interaction Tactics W. Lewis Johnson, Sander Kole, Erin Shaw1, Helen Pain2
1Center for Advanced Research in Technology for Education (CARTE)
USC / Information Sciences Institute
4676 Admiralty Way, Marina del Rey, CA 90292 USA
{johnson, skole, shaw}@isi.edu 2 School of Informatics, University of Edinburgh
80 South Bridge, Edinburgh EH1 1HN, UK
H.Pain@ed.ac.uk Abstract. The overall goal of this work is to provide pedagogical
agents with social intelligence, so that they can judge when is an
appropriate time to interact with the learner, be sensitive to the
cognitive and emotional state of the learner, and try to develop a
positive social relationship with the learner. This paper reports
on a learner-agent interaction framework designed for use by
socially intelligent agents. Interactions between a tutor and
students working on computer-based exercises were videotaped and
analysed, with particular emphasis on how the tutor sought to
motivate and engage the learner. Interactions were analysed as a
series of interaction tactics, where the speaker seeks to address
one or more informational, motivational, or social goals, and
monitors the listener's response to ensure that these goals are
achieved. Parsing and generation frameworks implementing
interaction tactics were developed, based upon the videotape
transcripts, and evaluated by other tutors.
Topic: intelligent tutoring and scaffolding. Subtopic: agent-based
interaction Introduction Animated pedagogical agents can promote effective learning in computer-
based learning environments [13]. Learning materials incorporating
interactive agents engender a higher degree of interest than similar
materials that lack animated agents [17]. Animated agents can produce a
positive affective response in the viewer, sometimes referred to as the
persona effect [15] . They exploit the human tendency to treat computer
systems as if they are social actors [19], capable of expressing emotions
and attitudes.
Researchers have also come to recognize the importance in tutoring of
recognizing the learner's affective and motivational states [14].
Computational techniques are being developed for inferring and tracking
such states [22][3], and for influencing them, e.g., to try to promote a
positive attitude about learning [21]. If such techniques were combined
with animated agent technologies, it might then be possible to create an
agent that can display emotions and attitudes as appropriate to convey
empathy and solidarity with the learner, and thus further promote learner
motivation.
The Social Intelligence (SI) Project is developing animated
pedagogical agents with well-developed social skills, which they can employ
to promote learning. The goal is to create agents that exhibit
expressiveness (the ability to convey emotions and attitudes), empathy
(sensitivity to learner motivational and emotional states), and politeness
(an understanding of when and how to interact in socially appropriate
ways). Such agents would then be able to tailor and adapt their style of
interaction according to the needs, preferences, and affective and
motivational states of each individual learner.
The current test bed for this work is the Virtual Factory Teaching
System (VFTS), an on-line factory simulation used to teach industrial
engineering concepts and skills [5][20]. An intelligent coach for
scientific experimentation, ALI, has already been integrated with VFTS [6].
Nevertheless, learners who lack extensive industrial engineering
background find the VFTS difficult to understand and use. We anticipate
that by integrating a socially intelligent agent with the VFTS,
particularly one that supports novice learners, a wider range of learners
will benefit from working wih the VFTS.
This paper is concerned with developing an appropriate set of
utterances that a socially intelligent agent can use to communicate with
learners. An agent needs to know what to say and how to say it in order to
convey and exploit its social intelligence. It also needs the ability to
express its comments with an appropriate tone of voice, accompanied with
appropriate gestures; these are also areas of active research at CARTE
(Marsella and Gratch)[1](Johnson et al. 2002)[12], but are beyond the scope
of this paper. Human tutorial dialogs were analyzed for this purpose, and
some of the results of this analysis are presented here. The resulting
dialog exchanges were implemented in an agent interface that is being used
to emulate socially intelligent tutorial interaction. 1. Learner-Tutor Interaction Study To investigate the role that social intelligence plays in learner-tutor
interaction, we videotaped interactions between learners and tutor while
the students were using the VFTS. Students read through an on-line
tutorial in a Web browser, and carried out actions on the VFTS simulation
as indicated by the tutorial. Learners were supposed to analyse the
history of previous factory orders in order to forecast future demand,
develop a production plan, and then schedule the processing of jobs within
the factory in order to meet the demand. The tutor sat next to the
students as they worked, and could interact with them as the student or the
tutor felt appropriate. Completing the entire scenario required
approximately two hours of work, divided into two sessions of approximately
one hour. Three video cameras were used: one focused on the learner's
face, one focused on the computer screen, and one providing a view of the
learner and tutor together. This made it possible to track the learner's
actions and focus of attention, as well as verbal and nonverbal
interactions between the learner and the tutor.
Prior to the first session learners were given a brief questionnaire
to assess their familiarity with manufacturing and business management
techniques, and with computers. The Myers-Briggs Type Inventory (MBTI) was
administered to each learner in order to assess personality
characteristics. After the learners completed the tutorial and all the
exercises the tutor had the learners complete a post-test relating to
factory management. Finally, the student and tutor were each interviewed
separately. The learners were questioned regarding subjective motivational
and affective factors, e.g., how difficult or easy they found the material,
how confident they felt, and whether they felt the work to be enjoyable or
frustrating. They were also asked whether these factors changed over the
course of the sessions. The tutor was asked to make similar assessments of
the student, e.g., whether the student felt confident, frustrated, etc.
The tutor was also asked to explain what led him to draw these conclusions.
The tutor in this preliminary study was an industrial engineering
professor who had won awards for excellence in teaching, and who uses the
VFTS in his courses. We plan follow-up studies with other tutors with
varying skills, as well as with pedagogical agents, in order to assess the
generality of the results of this study. Two students participated in the
study. One was an electrical engineering student who was familiar with
working computers and solving engineering problems, but did not have
expertise in industrial engineering per se. The other student was a
business major who had little experience with engineering problems, but was
ultimately able to apply her knowledge of business to the industrial
engineering problem.
The interactions in this experiment differ in a number of respects
from those observed in most previous studies of tutorial dialog. First,
the tutor was there to coach the learner as needed, rather than act as
tutor per se. The tutor was aware of the instructional objectives of the
exercise, and might interrupt to assist the learner with a particular topic
if the tutor deemed it necessary, but most of the time the tutor would
respond to the learner's questions, or offer advice and hints about the
learner's problem solving activities. This contrasts with tutorial dialog
studies such as those of Graesser, Person, et al. (e.g, [9]) and Chi et al.
[4] in which the tutor leads the dialog via a series of questions.
Furthermore, the learner was interacting with both the VFTS and the tutor.
The VFTS does not critique the learner's actions, but it does respond to
them immediately, which can help the learner to tell whether he or she is
making progress. Other researchers such as Merrill et al. [16] have
studied tutorial dialog in the context of problem solving, but there the
tutor is the sole source of feedback for the student, e.g., because the
student works out the problem on paper. In the VFTS context the tutor must
decide not just how to guide the learner, but also when. This is similar
to the problem that a pedagogical agent faces in deciding when to interrupt
a learner with advice or feedback.
To analyse the interactions, and use them in designing learner-agent
dialog, we transcribed them and annotated them using the DISCOUNT scheme
[18]. DISCOUNT represents the structure of educat