FPSP2004 - Department of Computer Science & Engineering

blabbedharborΤεχνίτη Νοημοσύνη και Ρομποτική

23 Φεβ 2014 (πριν από 3 χρόνια και 8 μήνες)

80 εμφανίσεις

Intelligent
Agents to
Deliver
Learning
Materials

Leen
-
Kiat Soh

Computer Science & Engineering

University of Nebraska

Lincoln, NE 68588
-
0115


lksoh@cse.unl.edu

http://www.cse.unl.edu/agents

Future Problem
Solving Workshop

Hastings




January 21, 2004

Challenges


Customized education


e.g., Modularized courseware to meet specific
requirements, deficiencies, and sequences


Adapting education


e.g., Flexible courseware that adapts in real time
to student behavior and aptitude


Effective distance education


e.g., tools with
anytime
,
anywhere

capabilities


e.g., infrastructures to bring classroom experience
to distance learners

Problem


Corbett
et al.
(1999):


“the arsenal of sophisticated computational modules
inherited from AI produce learning gains of approximately .3
to 1.0 standard deviation units compared with students
learning the same content in a classroom.”



Graesser
et al.
(2001):


“Human tutors produce impressive learning gains (between
.4 and 2.3 standard deviation units over classroom
teachers), even though the vast majority of tutors in a
school’s system have modest domain knowledge, have no
training in pedagogical techniques, and rarely use the
sophisticated tutoring strategies of ITSs.”

ITS = Intelligent Tutoring System

Problem 2


Woolf
et al.

(2002) also lists abilities that are
needed or present in tutors:


Generative:


Generates appropriate instructional material (problems,
hints, help) based on student performance


Student modeling:


Assesses the current state of a student’s knowledge and
does something instructionally useful based on the
assessment


Expert modeling:


Models expert performance and does something
instructionally useful based on domain knowledge

Problem 3


Woolf
et al.

(2002) also lists abilities that are
needed or present in tutors, cont’d:


Instructional modeling:



Changes pedagogical strategies based on the changing
state of the student model, prescriptions of an expert
model, or both


Mixed
-
initiative


Human Computer Interaction (HCI)


Self
-
improving:


Capacity to monitor, evaluate, and improve its own
teaching performance as a function of experience

Problem 4


Graesser
et al.

(2001) criticize the current
state of tutoring systems:


If students merely keep guessing until they find an action
that gets positive feedback, they can learn to do the right
thing for the wrong reasons


shallow learning



The tutor does not ask students to explain their actions


multiple choice questions



The user interface of tutoring systems requires students to
display many of the details of their reasoning


no stepping
back to see the “basic approach”



When students learn quantitative skills (e.g., algebra or
physics problem solving), they are usually not encouraged to
see their work from a qualitative, semantic perspective


Solution Ideas


Examples



PACT

(Koedinger
et al.
1997): algebra, geometry, and
computer languages


ANDES
(Gertner and VanLehn 2000; VanLehn 1996): physics


Used as an adjunct to college and high
-
school physics courses to help
students do their homework problems


Has an immediate feedback to enhance learning


SHERLOCK

(Lesgold
et al.
1992): electronics


ATLAS
(VanLehn
et al.
2000):


Model
-
tracing


Students scored significantly higher than the ANDES students on a
conceptual post
-
test


Solution Ideas 2


Examples



AutoTutor
(Graesser
et al.
2001): introductory
course in computer literacy


Fundamentals of computer hardware, OS, Internet, etc.


Was designed to be a
good conversational partner

that
comprehends, speaks, points, and displays emotions, all
in a coordinated fashion


Simulates a
multi
-
turn
conversation to extract more
information from the student and get the student to
articulate pieces of the answer


Pumps the student for what s/he knows before
drilling
down

to specific pieces of the answer


Uses
Latent Semantic Analysis

(LSA) to compute
matches between the student’s speech acts to the
expectations

Solution Ideas 3


Examples



AutoTutor
(Graesser
et al.
2001): introductory
course in computer literacy, cont’d:


Feeds back to the student at three levels:


(a)
backchannel feedback

that acknowledges the
learner’s input


(b)
evaluative pedagogical feedback

on the learner’s
previous turn based on the LSA values of the learner’s
speech acts (negative, neutral negative, neutral positive,
positive)


(c)
corrective feedback

that repairs bugs and
misconceptions that learners articulate

Solution Ideas 4


Examples



CIRCISM
(Freedman and Evens 1996)


BEE

(Basic Electricity and Electronics) tutor (Ros
é
et al.
1999)


EVELYN Reading Coach

and
EMILY Reading Coach

(Mostow and Aist 2002): Project LISTEN


Help students read by listening to children read aloud


BELVEDERE

(Suthers
et al.
2002):


Supports students in collaboratively solving ill
-
structured
problems in science and other areas (such as public
policy) as they develop critical inquiry skills


Solution Ideas 5


Teachable Agents
(Biswas
et al.
2002)


let students teach the agents to do things;
through teaching, the students learn


Articulate Software
(Forbus 2002); properties
are:


It should be
fluent

(some understanding of the
subject being taught)


It should be
supportive

(scaffolding)


It should be
generative

(pose new questions)


It should be
customizable

(manually)

Solution Ideas: Intelligent Agents


What is an
agent
?


An agent is an entity that
takes sensory input from
its environment
,

makes
autonomous

decisions
,

and

carries out actions

that
affect

the environment


A thermostat is an agent


A calculator is
not

an agent


Environment

sensory

input

output

actions

Agent

think!

Solution Ideas: Intelligent Agents 2


What is an
intelligent

agent
?


An intelligent agent is one that is capable of
flexible

autonomous actions in order to meet its design
objectives, where flexibility means:


Reactivity
: agents are able to perceive their environment, and
respond in a
timely

fashion to changes that occur in order to
satisfy their design objectives


Pro
-
activeness
: agents are able to exhibit
goal
-
directed

behavior

by
taking the initiative

in order to satisfy their design
objectives


Social ability
: agents are capable of
interacting

with other
agents (and possibly humans) in order to satisfy their design
objectives



(Wooldridge and Jennings 1995)

Solution Ideas: Intelligent Agents 3


Machine Learning in AI says






Agents that learn are intelligent


Not all agents are intelligent!




The acquisition of new knowledge and motor and cognitive skills

and the incorporation of the acquired knowledge and skills in

future system activities, provided that this acquisition and

incorporation is conducted by the system itself and leads to an

improvement in its performance.

Solution Ideas: Agent Environment


Inaccessible

vs. accessible



Incomplete vs. complete data


Deterministic vs.
non
-
deterministic


Certainty vs. uncertainty


Episodic vs.
non
-
episodic


Each episode is independent or not


Static vs.
dynamic


Remain unchanged except by the performance
of actions by the agent?


Discrete vs.
continuous


“Chess game” vs. “taxi driving”

Solution Ideas: Why Agents?


If the system
-
to
-
be
-
built has, during the
execution of the system


Incomplete data


Uncertainty in the assessment/interaction of its
environment


Inter
-
dependent episodes of events


No full control over the events in the environment


An “open world”, instead of a “closed world”


In other words, agents are used when you need
to build a system that is adaptive to an uncertain,
dynamic, and at times unexpected environment


So you can make full use of the
autonomous

property
of an agent

Why does a person hire an agent?

Evaluation Criteria


Pre
-
Design


Necessity


Is the agent solution necessary? Or an overkill?


Feasibility


Does the agent solution make sense? Is it impossible to
implement? Do we have the resources? Will it work?


During Design


Modularity


Are there different modules and components? Is the “data”
separated from the “brain”?


Extensibility


What if we want to apply it to another problem?


Scalability


What if we want to apply it to the same but bigger problem?

Some examples, not exhaustive

Evaluation Criteria 2


Post
-
Design


Correctness


Are the algorithms correctly designed? Are the
solutions correct?


Usefulness


Is the solution useful? Does it actually address the
problem? Does it help the user?


Reliability


Does the solution work for all possible problems?
Will the performance deteriorate after a while?


Adaptiveness/Learning


Can the solution evolve by itself to solve new
problems or to become better at solving old
problems?

Some examples, not exhaustive

AI

ILMDA


Intelligent Learning Materials Delivery
Agent


Bibliography


Allen, J. F., D. K. Byron, M. Dzikovska, G. Ferguson, L. Galescu, and A. Stent
(2001). Toward Conversational Human
-
Computer Interaction,
AI Magazine
,
22
(4):27
-
37.


André, E. and T. Rist (2001). Controlling the Behavior of Animated
Presentation Agents in the Interface: Scripting versus Instructing,
AI
Magazine
,
22
(4):53
-
66


Birmingham, W. P.,
et al
(1994. The University of Michigan Digital Library:
This Is Not Your Father's Library,
Proc. Digital Libraries 94, Hypermedia
Research Laboratory
, Texas A&M University, College Station, Tex., pp. 53
-
60.


Biswas, G., D. Schwartz, J. Bransford, and the Teachable Agents Group at
Vanderbilt (2002). Technology Support for Complex Problem Solving: From
SAD Environments to AI, in Forbus, K. D. and P. J. Feltovich (2002). (Eds.)
Smart Machines in Education
, Menlo Park, CA: AAAI Press, pp. 71
-
97.


Cassell, J. (2001). Embodied Conversational Agents: Representation and
Intelligence in User Interfaces,
AI Magazine
,
22
(4):67
-
83.


Cassell, J. and H. Vilhjálmsson (1999). Fully Embodied Conversational
Avatars: Making Communicative Behaviors Autonomous,
Autonomous Agents
and Multi
-
Agent Systems
,
2
:45
-
64.

Bibliography 2


Cohen, P. R. and S. L. Oviatt (1995). The Role of Voice Input for Human
-
Machine Communication, in
Proceedings of the National Academy of
Sciences,
92
(22):9921
-
9927.


Corbett, A., J. Anderson, A. Graesser, K. Koedinger, and K. VanLehn (1999).
Third Generation Computer Tutors: Learn from or Ignore Human Tutors? in
Proceedings of the 1999 Conference of Computer
-
Human Interaction
, 85
-
86.


D’Souza, A., J. Rickel, B. Herreros, and W. L. Johnson (2001). An
Automated Lab Instructor for Simulated Science Experiments, in J. D.
Moore, C. L. Radfield, and W. L. Johnson (eds.)
Artificial Intelligence in
Education: AI
-
ED in the Wired and WirelessFuture
, 65
-
76, Amsterdam: IOS.


Forbus, K. D. (2002). Articulate Software for Science and Engineering
Education, in Forbus, K. D. and P. J. Feltovich (2002). (Eds.)
Smart
Machines in Education
, Menlo Park, CA: AAAI Press, pp. 235
-
267.


Forbus,

K
.

D
.

and

P
.

J
.

Feltovich

(
2002
)
.

(Eds
.
)

Smart

Machines

in

Education
,

Menlo

Park,

CA
:

AAAI

Press
.


Freedman,

R
.

and

M
.

W
.

Evens

(
1996
)
.

Generating

and

Revising

Hierarchical

MultiTurn

Text

Plans

in

an

ITS,

in

Intelligent

Tutoring

Systems
:

Proceedings

of

the

1996

Conference
,

632
-
640
.

Bibliography 3


Gertner, A. S. and K. VanLehn (2000). ANDES: A Coached Problem
-
Solving
Environment for Physics, in
Intelligent Tutoring Systems: Fifth International
Conference, ITS 2000
, eds. G. Gautheier, C. Frasson, and K. VanLehn, 133
-
142.


Graesser, A. C., K. Wiemer
-
Hastings, P. Wiemer
-
Hastings, R. Kreuz, and the
Tutoring Research Group (1999). AutoTutor: A Simulation of a Human
Tutor,
Journal of Cognitive Systems Research
,
1
(1):35
-
51.


Graesser, A. C., K. VanLehn, C. P. Rosé, P. W. Jordan, and D. Harter (2001).
Intelligent Tutoring Systems with Conversational Dialogue,
AI Magazine
,
22
(4):39
-
51.


Herzog, M. and P. Petta (1995). Multimedia Information System in Open
-
World Domains, in
Working Notes of 1
st

Int. Workshop on Multimedia
Information Systems, Arlington
, VA, September 28
-
30, 13
-
16.


Herzog, M. and P. Petta (1996). Knowledge Representation in WERKL, an
Architecture for Intelligent Multimedia Information Systems, in
Workshop
Notes of the ECAI'96 Workshop on Knowledge Representation for Interactive
Multimedia Systems: Research and Experience (KRIMS'96), ECAI'96
,
Budapest, Hungary, August 12, 17
-
25.


Johnson, W. L. (2001). Pedagogical Agent Research at CARTE,
AI Magazine
,
22
(4):85
-
94.

Bibliography 4


Johnson, W. L., J. W. Rickel, and J. C. Lester (2000). Animated Pedagogical
Agents: Face
-
to
-
Face Interaction in Interactive Learning Environments,
International Journal of Artificial Intelligence in Education
,
11
:47
-
78.


Koedinger, K. R., J. R. Anderson, W. H. Hadley, and M. A. Mark (1997).
Intelligent Tutoring Goes to School in the Big City,
Journal of the Artificial
Intelligence in Education
,
8
(1):30
-
43.


Lesgold, A., S. Lajoie, M. Bunzo, and G. Eggan (1992). SHERLOCK: A
Coached Practice Environment for an Electronics Troubleshooting Job, in J. H.
Larkin and R. W. Chabay (eds.)
Computer Assisted Instruction and Intelligent
Tutoring Systems
, 201
-
238, Hillsdale, NJ: Lawrence Erlbaum.


Lester, J. C., C. B. Callaway, J. P. Grégoire, G. D. Stelling, S. G. Towns, and
L. S. Zettlemoyer (2002). Animated Pedagogical Agents in Knowledge
-
Based
Learning Environments, in Forbus, K. D. and P. J. Feltovich (2002). (Eds.)
Smart Machines in Education
, Menlo Park, CA: AAAI Press, pp. 269
-
298.


Marsella, S. and W. L. Johnson (1998). An Intelligent Assistant for Team
Training in Dynamic Multi
-
Agent Virtual Worlds, in
Proceedings of Intelligent
Tutoring Systems, 4
th

International Conference
, 464
-
473.


Mostow, J. and G. Aist (2002). Evaluating Tutors that Listen: An Overview of
Project LISTEN, in Forbus, K. D. and P. J. Feltovich (2002). (Eds.)
Smart
Machines in Education
, Menlo Park, CA: AAAI Press, pp. 169
-
234.

Bibliography 5


Raines, T., M. Tambe, and S. Marsella (2000). Automated Assistants to Aid
Humans in Understanding Team Behaviors, in
Proceedings of the Fourth
International Conference on Autonomous Agents
, 419
-
426.


Rickel, J. and W. L. Johnson (1999). Animated Agents for Procedural
Training in Virtual Reality: Perception, Cognition, and Motor Control,
Applied
Artificial Intelligence
,
13
(4):343
-
382.


Rickel, J., R. Ganeshan, C. Rich, C. L. Sidner, and N. Lesh (2000). Task
-
Oriented Tutorial Dialogue: Issues and Agents, paper presented at the
AAAI
Fall Symposium on Building Dialogue Systems for Tutorial Applications
,
November 3
-
5, North Falmouth, MA.


Rosé, C. P., B. Di Eugenio, and J. Moore (1999). A Dialogue
-
Based Tutoring
System for Basic Electricity and Electronics, in S. P. Lajoie and M. Vivet
(eds.)
Artificial Intelligence in Education
, 759
-
761, Amsterdam: IOS.


Rusell, S. and P. Norvig (1995).
Artificial Intelligence: A Modern Approach
,
Prentice
-
Hall.


Schank, R. and A. Neaman (2002). Motivation and Failure in Educational
Simulation Design, in Forbus, K. D. and P. J. Feltovich (2002). (Eds.)
Smart
Machines in Education
, Menlo Park, CA: AAAI Press, pp. 37
-
69.

Bibliography 7


Shaw, E., W. L. Johnson, and R. Ganeshan (1999). Pedagogical Agents on
the Web, in
Proceedings of the Third International Conference on
Autonomous Agents
, 283
-
290.


Smith, J. (2000). GrandChair: Conversational Collection of Family Stories, M.
S. Thesis, Media Lab, Massachusetts Institute of Technology, Cambridge, MA.


VanLehn, K. (1996). Conceptual and Metalearning during Coached Problem
Solving, in
Proceedings of the Third Intelligent Tutoring Systems Conference
,
29
-
47, Berlin: Springer
-
Verlag.


Suthers, D., J. Connelly, A. Lesgold, M. Paolucci, E. E Toth, J. Toth, and A.
Weiner (2002). Representational and Advisory Guidance for Students
Learning Scientific Inquiry, in Forbus, K. D. and P. J. Feltovich (2002). (Eds.)
Smart Machines in Education
, Menlo Park, CA: AAAI Press, pp. 7
-
35.


Weiss, G. (ed.) (1999).
Multiagent Systems: A Modern Approach to
Distributed Artificial Intelligence
, MIT Press.


Wooldridge, M. and N. R. Jennings (1995). Intelligent Agents: Theory and
Practice,
The Knowledge Engineering Review
,
10
(2):115
-
152.


Woolf, B. P., J. Beck, C.Eliot, and M. Stern (2002). Growth and Maturity of
Intelligent Tutoring Systems: A Status Report, in Forbus, K. D. and P. J.
Feltovich (2002). (Eds.)
Smart Machines in Education
, Menlo Park, CA: AAAI
Press, pp. 99
-
144.



Latent Semantic Analysis


Latent Semantic Analysis (LSA) is a theory and method for extracting
and representing the contextual
-
usage meaning of words by
statistical computations applied to a large corpus of text (Landauer
and Dumais, 1997). The underlying idea is that the aggregate of all
the word contexts in which a given word does and does not appear
provides a set of mutual constraints that largely determines the
similarity of meaning of words and sets of words to each other.


Based on Landauer, T. K., P. W., and D. Laham (1998) Introduction to
Latent Semantic Analysis,
Discourse Processes
,
25
:259
-
284.