Beyond Problem-solving: Student-adaptive Interactive Simulations for Math and Science

hesitantdoubtfulΤεχνίτη Νοημοσύνη και Ρομποτική

29 Οκτ 2013 (πριν από 4 χρόνια και 15 μέρες)

77 εμφανίσεις






Cristina
Conati

Department of Computer Science

University of British
Columbia


















Beyond Problem
-
solving:

Student
-
adaptive Interactive Simulations

for Math and Science



Overview


Motivations


Challenges of devising student
-
adaptive simulations


Two examples of how we target these challenges


ACE: interactive simulation for mathematical functions


CSP Applet:
interactive simulation for AI algorithm


Conclusions and Future work





Intelligent Tutoring Systems (ITS)



Create computer
-
based tools that support
individual
learners

By
autonomously

and
intelligently

adapting to their specific
needs

Student

Model

Tutor

Domain

Model

Adaptive
Interventions

ITS Achievements


In the last 20 years, there have been many successful
initiatives in devising Intelligent Tutoring Systems


(Woolf 2009, Building Intelligent Interactive Tutors, Morgan Kaufman)



Mainly ITS that provide individualized support to problem
solving through tutor
-
lead interaction (
coached problem
solving
)


Well defined problem solutions => guidance on problem solving
steps


Clear definition of correctness => basis for feedback





Beyond Coached Problem Solving


Coached problem solving is a very important component of
learning


Other forms of instruction, however, can help learners
acquire the target skills and abilities


At different stages of the learning process


For learners with specific needs and preferences


Our Goal
: Extend ITS to other learning activities that
support student initiative and engagement:


Interactive Simulations


Educational Games


Overview


Motivations


Challenges of devising student
-
adaptive simulations


Two examples of how we target these challenges


ACE: interactive simulation for mathematical functions


CSP Applet:
interactive simulation for AI algorithm


Conclusions and Future work





Challenges


Activities more open
-
ended and less well
-
defined than
pure problem solving


No clear definition of correct/successful behavior


Different user states to be captured (meta
-
cognitive,
affective) in order to provide good tutorial interventions


difficult to assess unobtrusively from interaction events



How to model what the student is doing?


How to provide feedback that fosters learning while
maintaining student initiative and engagement?


Our
Approach


Student models based on formal methods for
probabilistic reasoning and machine learning



Increase
information available to student model through
innovative input
devices:


e.g
.

eye
-
tracking and physiological
sensors



Iterative model design and evaluation


Overview


Motivations


Challenges of devising student
-
adaptive simulations


Two examples of how we target these challenges


ACE: interactive simulation for mathematical functions


CSP Applet:
interactive simulation for AI algorithm


Conclusions and Future work











ACE: Adaptive Coach for Exploration








Activities organized into units to explore mathematical
functions (e.g. input/ouput, equation/plot)


Probabilistic student model that captures student
exploratory behavior and other relevant traits


Tutoring agent that generates tailored suggestions to
improve student exploration/learning when necessary

(Bunt, Conati, Hugget, Muldner, AIED 2001)

Adaptive Coach for Exploration

EDM 2010

11

12

Adaptive Coach for Exploration

13

Adaptive Coach for Exploration

Before you leave

this exercise, why don’t you
try scaling the function by a large negative
value?

Think about how this will affect

the plot

ACE Student Model

(Bunt and
Conati

2002)

Knowledge

Individual

Exploration Cases

Exploration

of Exercises

Exploration

Categories

Exploration

of Units




Iterative
process of design and evaluation


Probabilistic model of how individual exploration actions
influence exploration and understanding of exercises and concepts


e.g. (in Plot unit)



positive/negative slope


positive/negative intercept



large/small, positive/negative exponents…

Modeling Student Exploration


Our first attempt (Bunt and
Conati
, 2002)


Learning


Student Model

Number and Coverage
of Exploratory Actions, e.g.


Positive/negative Y
-
Intercept


Odd/Even, Positive Negative Exponent....


Interface Actions

Preliminary Evaluation


Quasi
-
experimental design with 13 participants using
ACE

(Bunt and
Conati

2002)


The more exercises were effectively explored according to
the student model, the more the students improved



The more hints students followed, the more they learned



Because the model only considers coverage of student
actions, it can overestimate student exploration



Need to consider whether the student is reasoning about
the effects of his/her actions


Self
-
explanation

meta
-
cognitive skill:



Revised User Model

(Bunt,
Muldner

and Conati, ITS2004;
Merten

and Conati, Knowledge Based Systems 2007)

Learning


Student Model


Number and coverage of student actions


Self
-
explanation of
action outcomes


Time between actions



Gaze Shifts in Plot Unit

Interface Actions

Input from eye
-
tracker

Results on Accuracy


We evaluated the complete model against


The original model with no self
-
explanation


A model that uses only time in between actions as evidence of self
-
explanation


50
60
70
80
Accuracy on SE
Accuracy on
Learning
No SE
SE (Time)
SE (Time + Gaze)
What’s Next (1)


Test adaptive interventions to trigger self
-
explanation
(
Conati

2011)




Discussion


ACE work provided evidence that


It is possible to track more “open ended” students’
behaviors

than structured problem solving


eye
-
tracking can support the process


However, hand
-
coding the relevant behaviors, as we did
for ACE (
knowledge
-
based approach
)


is time consuming


likely to miss other, less intuitive patterns of interaction
related to learning (or lack thereof)

Alternative Approach

(
Amershi

and
Conati

2009,
Kardan

and
Conati

2011)

Behavior Discovery Via Data Mining

Association
Rules

Mining

Clustering

Actions Logs

Other Data

Fe
atu
re


Ve
cto
rs



Vector of Interaction
Features

-

Frequency Of Actions

-

Latency Between


Actions

……………




Extract

rules describing
distinguishing patterns in
each cluster

Groups

together students
that have similar
interaction behaviors

Interpret in terms of

learning




Experts


Performance
Measure(s)



Overview


Motivations


Challenges of devising student
-
adaptive simulations


Two examples of how we target these challenges


ACE: interactive simulation for mathematical functions


CSP Applet:
interactive simulation for AI algorithm


Conclusions and Future work





Tested with AI Space CSP applet


AISpace

(
Amershi

et al., 2007)


set
of
applets implementing interactive simulations of
common
Artificial Intelligence
algorithms


Used regularly in our AI courses


Google “
AISpace
” if you want to try it out


Applet for Constraint Satisfaction problems (CSP),
visualizes the working of the AC3 algorithm

27

AISpace

CSP Applet






Direct Arc
Clicking

User Study
(
Kardan

and
Conati

2011)


65 subjects


Read intro material on the AC
-
3 algorithm


Pre test


Use CSP applet on two problems


Post test


13,078 actions


More than 17

hours of interaction


Dataset


Features:


frequencies of use for each action


pause duration between actions (Mean and SD)


7 actions


㈱⁦e慴ures



Performance measure for validation


Learning Gain from pretest to posttest

Feature
vectors

Clustering

Behavior Discovery

Rule Mining



Found 2 clusters



Statistically significant
difference in Learning
Gains (LG)


High Learners (HL) and
Low Learners (LL) clusters

3
2

Feature
vectors

Clustering

Behavior Discovery

Rule Mining

Clustering

Usefulness:

Sample Rules

HL members:


Use
Direct Arc Click
action very frequently (
R
1
).

HL

cluster
:


R
1
:

Direct

Arc

Click

frequency

=

Highest

(Conf

=
100
%
,

Class

Cov

=

100
%
)

LL

cluster
:

R2
:

Direct Arc Click Pause
Avg

= Lowest (Conf =100%, Class
Cov

= 100%)


R3
:
Direct Arc Click frequency
= Lowest (Conf = 93%, Class
Cov
=93.5%)

3
3

LL members:


Use
Direct Arc Click
sparsely (
R3
)


Leave l
ittle

time between a
Direct Arc Click
and the next action
(
R2
)

Feature
vectors

Clustering

Behavior Discovery

Rule Mining

Great, but what do we do with this?


We can use the learned clusters and rules to classify a new
student based on her behaviors



Use detected behaviours for adaptive support


Promoting the behaviours conducive of learning


Discouraging/preventing detrimental behaviours

3
4

The User Modeling Framework

3
5

Association
Rules

Mining

Clustering

Feature
Vector
Calculation

Online

Classifier

Adaptive
Interventions

Behavior Discovery

User Classification

Actions Logs

Other Data

F
e
at
u
re



New
user’s

Actions



Vector of
Interaction
Features



If

user is
a LL

and
uses
Direct Arc Click

very
infrequently

(
R
3
)


Then

prompt this
action



If

user is
a LL

and
pauses
very briefly after a
Direct Arc Click

(
R2
)


Then

take action to slow
her down


Classifier Evaluation


Leave
-
one
-
out Cross Validation on dataset of 64 users


For each user
u

in dataset

1.
Remove user
u

2.
do
Behaviour

Discovery on the remaining 63

3.
for each of
u
’s

actions:

»
Calculate the feature vector
u
v

»
Classify
u
v


»
Compare with
u
’s

original label


Accuracy as a function of observed actions

Discussion


User modeling framework for open
-
ended and
unstructured interactions


Relevant behaviours are discovered via data mining
techniques instead being hand
-
crafted


Very encouraging results with CSP applet


Detected clusters represent groups with different
learning gains


Online classifier:
good accuracy soon enough to generate
adaptive interventions


These interventions can be derived from the generated
rules

Current Work



Applying the discovered rules to generate the adaptive
version of the CSP applet


Adding eye
-
tracking input to the dataset


Conclusions



Research on devising student
-
adaptive didactic support
for exploratory activities beyond problem solving


Interactive simulations


Challenges in modeling interactions with no clear
structure or definition of correctness


Student modeling approaches based on probabilistic
techniques and unsupervised machine learning



very promising results


Shown how eye
-
tracking can help!


We are also exploring it in relation to assessing engagement and
attention in educational games (Muir and
Conati

2011)