Global Feedback in ACTIVEMATH - erica melis - Universität des ...

Global Feedback in ACTIVEMATH
Erica Melis and Eric Andrès
German Research Center for Artifical Intelligence
and Universität des Saarlandes
Germany
melis@dfki.de
eandres@activemath.org Abstract:
This paper describes the global feedback in ACTIVEMATH, a web-based
adaptive learning environment for mathematics and beyond. It addresses some
of the cognitive foundations and the architecture of the suggestion
mechanism with its components. This architecture separates components for
diagnosis from components for suggestions and different types of
information: the observable facts about the student, implied non-observable
diagnoses about the student's actions, suggestions implied from the
observable and non-observable diagnoses, and the actual rendering of the
suggestions. This mechanism is highly adaptive. 1 Introduction
Human tutors react to a student's learning activities such as reading,
understanding examples or problem solving attempts. Similarly, an
intelligent tutoring system should provide such reactions, non-verbal as
well as verbal ones. As the granularity and type of the student's
activities vary, the tutorial responses do not only depend on the learner's
actions but also what the system delivered in the time interval the
feedback responds to. For instance, responses can be given on problem
solving steps, on an overall exercise performance, or on a lesson.
Obviously, reactions for navigation deviations will have to differ from a
response to a misconception. In addition, responses may depend on the
learning goal, the pedagogical strategy, and on the learner's
characteristics.
Feedback is a reaction to a student's learning behavior. It can consist of
a verbal tutorial reaction and of non-verbal reactions, such as pointing,
attention-drawing, offering of next steps, etc.
Cognitive and pedagogical psychology research has primarily investigated
feedback, i.e., feedback in problem solving exercises. This includes
questions such as: is feedback effective for learning and for which
students? Is feedback indeed used as expected? How should feedback be
designed and when should it be offered to the student to achieve as large
as possible learning gains?
The following summarizes results on feedback types that appeared to be
useful inside exercises (this summary is adopted from Jacobs (2001), which
is based on a myriad of scientific articles). There are several types of
feedback:
. Knowledge of result (KR) means a feedback that states whether a solution
is correct or wrong. This feedback has proved to be the minimal one. For
strong students KR might suffice to stimulate further elaboration.
. Knowledge of correct result (KCR) provides correct solution. This is
recommended for students with little prior knowledge, little ability,
many errors, and relatively simple learning goals.
. Answer until correct (AUC) asks the student to try again until she
answers correctly. It proved valuable, for complicated task and students
with sufficient ability to solve the exercise.
. Instruction-based elaboration (IBE) may include explanation of the
correct solution, correction of errors (local), or the presentation of
the original instruction.
Several investigators (Aleven & Koedinger (2000), Narciss & Huth (2003))
found that delivering KCR immediately - without any effort by the learner -
is counterproductive. Therefore, they suggest to inhibit KCR as long as the
learner did not try to follow KR and hints.
For multi-step problem solving, Farquhar (1995) shows that elaborate
feedback improves learning time and learning result more than the pure KCR.
For Farquhar elaborate feedback may comprise several of the following:
notification of error, reason for incorrectness, brief description of
appropriate subgoal, list of steps to complete the subgoal, indication of
progress, identification of very next step. Certainly, elaborate feedback
could contain other actions as well, e.g. a counter example or a
description of consequences of a mistake.
Extra-instructional elaboration (EIE) in problem-solving may include a
variety of information, e.g.,
. strategic help (Dempsey et al. (1993))
. meta-level feedback such as prompts for self-explanation (Chi et al.
(1994), Chi (2001), Stark (2000), Sandoval et al.(1995)) or prompts for
summarizing and elaborating.
Not only the content of a tutorial reaction matters but also the point in
time when it is delivered to a particular student and how (positive vs.
negative feedback). Some relevant aspects for the actual delivery of
feedback are:
. feedback is particularly useful when an error has occurred. Weak students
benefit more from feedback than strong students. There is also one paper
that reports a beneficial effect of positive feedback (Webb et al.
(1994)).
. feedback may be used inappropriately by learners and may not improve
learning because the user just receives the correct solution without any
thinking effort (Aleven & Koedinger (2000b), Clariana (1999)). The
learner may use this information only for judging his performance (in
competition) rather than for reasoning about mistakes and for correcting
errors.
. as with any learning, the understanding of feedback can be supported by
referring to knowledge familiar to the student, by asking questions, by
prompting the student to deliver explanations, to find new examples, to
make notes, or to draw a concept map ( Jacobs(2001)).
. feedback or help should be sparsely provided, if needed only (Bunt et al.
(2001)), and mainly as a guidance for poor learners. Feedback should be
brief and relevant (Jacobs (2001)). Some researches found questions to be
superior to statements (Chi et al. (2001b))
. often, feedback is better acknowledged by the student when personalized
by pictures of the teacher, a friend and tailored to the student (by name
etc).
. the question whether immediate or delayed feedback is more effective does
not have a clear answer currently. The diverging results may be caused
by the influence of characteristics of the learner and of the learning
situation. See more on immediate vs. delayed feedback in §2.
Most of the mentioned reactions in feedback are conveyed verbally. A (non-
verbal) form of reaction to a student's learning activities is the
selection of new exercises by the tutor. Although most of the results were
obtained for feedback in exercises only, this brief summary shows that
there are many aspects to be investigated for appropriate and useful
tutorial reactions.
This brief introduction already shows that there are many cognitive aspects
to be investigated for appropriate and useful tutorial reactions. The web-
based, adaptive, and interactive learning environment ACTIVEMATH strives to
support learning mathematics, currently at university and college level
(and soon with a school version too). For true learning, interactivity and
feedback are two key ingredients that can be offered by e-Learning
(Schulmeister (2004)). Therefore, we are developing components of
ACTIVEMATH which can deliver intelligent feedback on interaction. One of
these components - the suggestion mechanism - is presented in this paper.
In order to generate feedback that helps learning, the cognitive empirical
results have to be respected in the design of the suggestion mechanism and
the actual suggestions. In addition, the technological problems need to be
solved, such that, e.g., extensibility and modifiability of feedback and
the generation will be guaranteed.
In this article, we shall address some of the cognitive and technological
issues. After an introduction to the differences between two types of
feedback, local and global feedback, we recapitulate the features of the
learning environment ACTIVEMATH. Then, the global feedback in the
ACTIVEMATH system is described, in particular, the architecture and some of
the rules for user-adaptive global learning suggestions implemented so far. 2 Local and Global Feedback
In most intelligent tutoring systems (ITSs) feedback is an immediate
reaction to the actual problem solving and supposed to help students to
accomplish a solution of an exercise. Usually, this local feedback reflects
the experience with typical errors in the domain of the exercise.
Some ITSs also provide feedback targeting meta-cognitive skills of the
student. For instance, the Andes system (Conati & VanLehn (1999)) tries to
support self-explanation of worked-out examples and SciWise (White &
Shimoda (1999)) provides advice for planning and reflection activities in
experimental science and for collaboration. This type of feedback is
delivered outside of exercises.
More generally, two kinds of feedback and guidance can be provided by an
ITS, a local response to student activities which is coaches the correction
of a problem solving attempt of the learner and a global feedback coaching
(several aspects of) the entire learning process. This differentiation
somewhat resembles the distinction of task-level and high-level described
in the process model of Almond, Steinberg, & Mislevy (1999).
Local and global feedback differ with respect to
. which of the student's (sequences of) activities is supported,
scaffolded, or re-assured
. what the feedback is about and whether domain-level or meta-level
reas