Personalised E-Commerce Agents for Virtual Webpages

closebunkieAI and Robotics

Nov 15, 2013 (3 years and 9 months ago)

169 views

© W. Wahlster, DFKI

Mexican International Conference on Artificial Intelligence 2000


Wednesday, April 12

German Research Center for Artificial
Intelligence, DFKI GmbH


Stuhlsatzenhausweg 3

66123 Saarbruecken, Germany

phone: (+49 681) 302
-
5252/4162

fax: (+49 681) 302
-
5341

e
-
mail: wahlster@dfki.de

WWW:http://www.dfki.de/~wahlster

Wolfgang Wahlster

Personalised E
-
Commerce
Agents for Virtual Webpages


© W. Wahlster, DFKI

First Generation

Second Generation

Third Generation

Static Web Sites

Fossils cast in HTML

Interactive Web
Sites

JavaScripts and Applets

Database Access and

Template
-
based Generation

Dynamic Web Sites

Virtual Webpages

Netbots,

Information Extraction,

Presentation Planners

Adaptive Web Sites

User Modeling,

Machine Learning,

Online Layout

Three Generations of Web Sites

© W. Wahlster, DFKI

A Virtual Web Page


l

is generated on the fly as a combination of various media objects from


multiple web sites or as a transformation of a real web page.



l


looks like a real web page, but is not persistently stored
.


l


integrates generated and retrieved material in a coordinated way.


l


can be tailored to a particular user profile and adapted to a particular


interaction context.


l


has an underlying representation of the presentation context so that


an Interface Agent can comment, point to and explain its components.

Virtual Memory, Virtual Relation, Virtual Reality...

What is a Virtual Web Page?

© W. Wahlster, DFKI

Yahoo
Weather
Server

PAN Travel Agent Andi

Car Route Planner

Yahoo
News
Server

Gault Millau
Restaurant
Guide

Hotel
Guide

AiA: Information Integration for Virtual Webpages

© W. Wahlster, DFKI

Virtual Webpage Retrieved from 5 Different Servers

© W. Wahlster, DFKI

Multi
-
Domain

Problem Specs

NETBOT

Retrieved

Results

Information Structures

l

Relations, Lists

l

KR Terms

Media Objects

l


Texts, Sounds, Videos

l


Pictures, Maps,


Animations

Distributed

Information

Multiple

Data Sources

The Combination of Retrieved and Generated Media
Objects for Virtual Webpages

© W. Wahlster, DFKI

Retrieved

Results

Select & Design

Select Canned

Media Objects

Design New

Media Objects

Information Structures

l

Relations, Lists

l

KR Terms

l


Graphics,


Animation

l


Text,


Speech,


Mimic

l

Icons,


Clip Art

l

Frames,


Sounds

Reuse & Transform

Coordinate

Media Objects

Transform

Media Objects

l

Temporal


Synchroni
-


zation

l


Spatial


Layout

l


Clip, Convert,


Abstract

l


Zoom, Pan,


Transition


Effects

Media Objects

l


Texts, Sounds, Videos

l


Pictures, Maps,


Animations

The Combination of Retrieved and Generated Media
Objects for Virtual Webpages

© W. Wahlster, DFKI

Operational Models of Referential Semantics for
Robots and Netbots

Robot

Netbot

“Screw”

Physical

Objects

Screw 1

Screw N

...

Set of Recognizers

Set of Subsumption

Relations in an Ontology

“Departure Time”

Set of Subsumption

Relations in an Ontology

WWW

Objects

DT 1

DT N

Set of Wrappers

...

© W. Wahlster, DFKI

System is able to flexibly
tailor presentations to the
individual user and the
current situation.

Enhancement of User Interfaces through Personalization

An animated character

serves as “Alter Ego”

of the presentation system.

Personalized Presenters at DFKI

© W. Wahlster, DFKI

The Role of Ontological Annotations for the
Generation and Analysis of virtual Webpage

Information

Extraction Agent

Presentation

Planner

Webpages

with Ontological

Annotations

Webpages

without Ontological

Annotations

Virtual Webpage

Presentation

Agent

Persona

Information

Extraction

Agents

TriAS

With Ontological Annotations in:

SHOE, OML,















XOL, OIL

and Persona Annotation in



PML



© W. Wahlster, DFKI

Video Dresdner Bank

© W. Wahlster, DFKI

Netbot

PAN

Trip Data

Pictures and

Graphics

Pieces of Text

Coordinates for

Pointing Gestures

Input for Speech

Synthesis

Icons for

Hyperlinks

Hotel

Agent

Map

Agent

Address

Weather

Agent

Train & Flight

Scheduling

Agent

Major Event

Agent

Virtual

Web

Presentation

Constraint
-

based

Online

Layout

Presentation

Planner

Persona

Server

Components

of virtual

Webpages

AiA

The Generation of Virtual Webpages with PAN and AiA

© W. Wahlster, DFKI

Persona as a Personal Travel Consultant

© W. Wahlster, DFKI

WML
-
Browser


MS
-
Agent
Controller


WML

SMIL

Agent
Script

PET
-

PML

PET
Persona
Player


SMIL
Player


Presentation Planner


AIA’s presentation planner has been extended to
accommodate for various target platforms through the
introduction of a mark
-
up language layer

© W. Wahlster, DFKI



Persona

Server

Behaviors

Presentation Gestures

Reactive Behaviors

Idle
-
time actions

Navigation actions

Auditory Characteristics

Sound effects, auditory icons

Voice: male, female

Visual Appearances

Hand
-
drawn

Cartoon

Bitmaps

Generated

Bitmaps



from



3D
-
Models

Video Bitmaps

l

l

l

l

l

l

PPP’s Persona Server implements a generic Presentation Agent
that can be easily adapted to various applications

© W. Wahlster, DFKI

Classification of Persona Gestures

Talking Posture 1


cautious, hesitant


appeal for compliance


avoids body
-
gestures

Talking Posture 2


active, attentive


self
-
confident


uses body
-
gestures


Gesture Catalogue

© W. Wahlster, DFKI

take
-
position (t

1


t

2

)

point
-
to (t

3


t

4

)

move
-
to (t

1


t

2

)

r
-
stick
-
pointing (t

3


t

4

)

High
-
Level

Persona Actions

Context
-
Sensitive

Expansion

(including Navigation Actions)

Decomposition



into



Uninterruptable



Basic Postures

r
-
turn (t

1


t

21

)

r
-
step (t

21


t

22

)

f
-
turn (t

22


t

2

)

r
-
hand
-
lift (t

3


t

31

)

r
-
stick
-
expose (t

31

t

4

)

Bitmaps

...

...

...

...

Context
-
Sensitive Decomposition of Persona Actions

© W. Wahlster, DFKI

Production Act

Presentation Act

Introduce

Create
-

Graphics

S
-
Show

S
-
Wait

S
-
Position

Elaborate
-
Parts

S
-
Create
-

Window

S
-
Depict

Label

Label

S
-
Point

S
-
Speak

S
-
Speak

S
-
Point

Qualitative constraints:

Create
-
Graphics meets S
-
Show, ...

Metric constraints:

1 <= Duration S
-
Wait <= 1, ...

Distinction between production and presentation acts

(i.e. Persona
-

or display acts)

Explicit representation of qualitative and quantitative constraints

Extensions of the Representation Formalism

© W. Wahlster, DFKI

Persona Presents an Automatically Designed

Business Chart

© W. Wahlster, DFKI

Virtual Webpage with Animation Effects Based on a
Single GIF Image

© W. Wahlster, DFKI

Transition Effects in a Series of Retrieved Pictures

© W. Wahlster, DFKI

Objective:



l

Enable non
-
professional computer users to populate their





web pages with lifelike characters

PET comes with:


l


a set of characters and basic gestures


l


an easy
-
to
-
learn Persona markup language

Developer’s PET will include:



l

a character design tool which enables users to build their





own characters

Technical Realization:



l


Based on XML and Java

PET: Persona
-
Enabling Toolkit

© W. Wahlster, DFKI

Specification of the
character to be used

Specification of
Persona actions

<html>

<head>

<title> Persona Test </title>

</head>

<body>

<persona bitmap=“cartoon” ...>

<uselib url= .../>

<do name=“greet”/>

<do name =“speak” args=“hello”/>

</persona>

</body>

</html>

Features:


XML
-
based


easy to learn

The Persona Markup Language

© W. Wahlster, DFKI

PET
-
Parser



l

Preprocessing (required for non
-
standard HTML pages)



l


Identification of Persona Tags

PET
-
Generator

l

Replacement of Persona Tag with Java Applet

l

Determination of parameters for Java Applet

l

Realization of Persona active elements as JavaScript

l

Generation of DHTML layers (transparency of Applet box)

l

Generation of data, e.g. audio files for speech output

l

Generation of scripts


Tasks of PET

© W. Wahlster, DFKI

URL of Webpage
with Persona Tag

Persona Engine

Behavior Monitor

Character
Composer

Event
Handler

<html>

<head>

<title> Persona Test </title>

</head>

<body>

<persona bitmap=“cartoon” ...>

<uselib url= .../>

<do name=“greet”/>

<do name=“standard”/>

<do name =“speak”
args=“hello”/>

</persona>

</body>

</html>

Persona Scripts

waitscreen 4

gesture greet 0 0 null

gesture laugh 0 0 null

...

Audio Data

Bitmaps

PET Application
Server

PET
Parser

PET
Generator

Webpage with Reference to
Java Applet

<html> ...

<APPLET
archive=“personaplayer.jar”...
</APPLET>

...</html>

Functional View of PET

© W. Wahlster, DFKI

l

Text Input

l

Speech Input

l

Menu Input

l

Direct Manipulation


Input

Web Persona

Triggers

actions of the Persona

Triggers

operations on elements

of the webpage

l

Mouse Clicks

l

Mouse Movements

The Bidirectional Control Flow on

Persona
-
Enabled Webpages

© W. Wahlster, DFKI

Video 911 & Boxter

© W. Wahlster, DFKI

<PDIV DIVNAME=“name of the element” DVFRAME=“frame name”
TOP=“anchor
-
y” LEFT=“anchor
-
x”>some HTML elements</PDIV>

<ACTIVEIMAGE SRC=“image” HREF=“url” NAME=“image name”
STATUS=“status message” ALT=“tooltip” CACTION=“persona action onClick”
OACTION=“persona action on MouseOver” ...>

l
Active Images



An active image starts a persona action when clicked.








l
Addressable Objects


An addressable object is an object which can be addressed and


manipulated by Persona via its name and its position
.





Persona Active Elements (PAE)

© W. Wahlster, DFKI

Spatial and Temporal Coordination of
Multimedia Presentations


Multimedia coordination in previous systems:


no declarative representation of spatial and temporal
layout


no synchronization of Persona actions with other
dynamic multimedia objects


Solution: Use PrePlan for automated generation of
SMIL expressions




Synchronized Multimedia Integration Language


officially recommended by W3 consortium

© W. Wahlster, DFKI

Approach


High
-
level specification of
temporal
and
spatial
constraints

within the operators of the Presentation
Planner PrePlan


Use of the
incremental constraint solving

toolkit
Cassowary


Uniform treatment of spatial and temporal constraints


Transformation of qualitative constraints into metric
constraints


Integration of a backtracking mechanism to handle
disjunctions


© W. Wahlster, DFKI

Processing Steps

Decomposition of presentation goals into elementary acts
and collection of spatial and temporal constraints

Constraint propagation using extended Cassowary constraint
solver

Representation of spatial and temporal layout as SMIL
constructs

© W. Wahlster, DFKI

Example of a Plan Operator

(define
-
plan
-
operator

:header

(A0 (ShowPresentation ?topic))

:constraints


(*and*

(BELP (
Illustrates

?video ?topic)) (BELP (Video ?video))



(BELP (Sets
-
to
-
Music ?audio ?topic)) (BELP (Audio ?audio))



(BELP (
Summarizes
?title ?topic)))

:inferiors


(A1 (SAddSmilCode (?
video
)))


(A2 (SAddSmilCode (?
audio
)))


(A3 (SAddSmilCode (?
title
)))

:
temporal


((A1 (d) A3) (2 <= begin A1
-

begin A2))

:
spatial






(

(aligntop A1) (centerh A1) (centerh A3)





(20 <= top A1
-

bottom A3 <= 20))

© W. Wahlster, DFKI

Representation of Spatial and Temporal
Constraints


Spatial constraints


Qualitative constraints


(Image1 LeftOf Image2), (Image1 TopAlign Image2)


Quantitative constraints


(4 < Top Image1
-

Top Image2)


Temporal constraints


Qualitative constraints


Allen constraints: (meets, before, overlaps, ...)


Quantitative constraints


Linear inequalities: (Begin Audio1
-

End Audio2 < 3),


(2 <= Duration Audio1 <= 6)

© W. Wahlster, DFKI

Resulting SMIL
-
Document

<smil>

<head>

<meta content="Smiling Saarland" name="title"/>

<layout>


<
root
-
layout height="50" width="300"/
>


<
region height="30" id="reg43463" left="0" top="0" width="300"
/>

</layout>

</head>

<body>


<par>



<
audio begin
="1.0s" end="21.807s" src="snowman.wav"/>





<
textstream begin
="0.0s" end="22.807s" region="reg43463"
src="snowman.rt"/>



<
video begin
="3.0s" end="21.807s" src="snowman.avi"/>


</par>

</body>

</smil>

© W. Wahlster, DFKI

Interactive Web Demo

© W. Wahlster, DFKI

Synchronization of Persona Actions with other
Media Objects


Some Examples:


point to an object when it appears in a video


comment on a video 3 seconds after it has started


Smile when video appears on the screen


repeat lip movements until audio stops

© W. Wahlster, DFKI

Synchronization of Persona Actions

(define
-
plan
-
operator

:header

(A0 (ShowPresentation ?topic))

:constraints


(*and*

(BELP (Describes ?audio ?topic)) (BELP (Audio ?audio))



(BELP (
TalkingGesture
?video)) (BELP (
RepeatGesture
?video))

:inferiors



(A1 (PresentPictureSequence (?topic)))



(A2 (SAddSmileCode (?audio)))



(A3 (SAddSmilCode (?video)))

:temporal

((A2 (d) A1) (2 <= begin A2
-

begin A1) (A2 (e) A3))

:spatial





(

(aligntop A1) (alignleft A1)



(1 <= bottom A0
-

bottom A3 <= 1)



(1 <= right A0
-

right A3 <= 1))


© W. Wahlster, DFKI

SMIL Specification for Persona Presentation

<smil> <head>



<meta name="title" content="Cyberella in RealPix Demo" />

<layout>



<
root
-
layout width="650" height="385" background
-
color="#9898F8
"/>



<
region id=“reg471101" top="0" left="12" width="385" height="284"


background
-
color="#9898F8"/
>



<
region id=“reg081591" top="95" left=“354" width="295" height="385"


background
-
color="#9898F8"/
>

</layout> </head> <body>

<par>


<
img region
= " reg471101" begin= " 0.0" src= "schwarzenacker.rp"
fill="freeze"/>


<
animation begin
= "2.0s" end="15.1" region="reg471101"



src= "talking
-
gst.rp"/>


<
audio begin
= "2.0s" end="15.1" src="schwarzenacker.rm" />

</par> </body> </smil>

© W. Wahlster, DFKI

Resulting Timeline Diagram

At archeological finds in Schwarzenacker, ...

...

...

...

...

...

© W. Wahlster, DFKI

Using SMIL to Synchronize Persona Actions

© W. Wahlster, DFKI

Video Otto 1

© W. Wahlster, DFKI

Information Extraction Agents


Information Filtering


Information Retrieval


Information Integration

identify relevant

documents

wrappers



...



identify and extract relevant


pieces of information



transform them into canonical form

wrappers



operational

descriptions of a target concept



abstract

from concrete occurrence within document



robust

against modifications

© W. Wahlster, DFKI

The Trainable Information Agents Framework

Browser

Application

InfoBroker

Info Extraction

Trainer

planning knowledge

user preferences

domain ontology

Web site annotations

User

requests

training

specifications

results

info requests

info

info requests

info or script

PBD dialog

preferences/heuristics

combination of "classical"

problem
-
solving methods

and information agents

query planning, optimization,

and execution

improved dialog guidance

© W. Wahlster, DFKI

High Degree of Parallelism of Queries

© W. Wahlster, DFKI

Knowledge about a Webpage Shared by
User and Agent

structural

visual/semantic

procedural

Naive

User

Learning Annotation

Agent

common part (usable

for communication)

© W. Wahlster, DFKI

Video PAN

© W. Wahlster, DFKI

Example
-

Ontology

Train_Connection [ from =>> Location;






to =>> Location;






travel_date =>> Date;






time =>> Time;






depart_time =>> Time;






arrive_time =>> Time;







cost =>> Price;






travel_duration =>> Duration;






info_url =>> URL;





... ]

© W. Wahlster, DFKI

Query Planning
-

I


states: information states


concepts / attributes and instantiations


operators: querying schemes


preconditions (
´
+
´
) and effects (
´
-
´
)





to time arrive_time travel_duration




< +, +, +, +,
-
,
-
,
-
, u,
-

>





from travel_date depart_time cost info_url

© W. Wahlster, DFKI

Query Planning
-

II


merging of alternative plans with identical
intermediate states















(a)

(a,b)

(a,c)

(a,b,c,d)

(a,b,c,d,e)

(a,b,c,d)

op1

op2

op3

op4

op5

(a,b,c,d,f)

op6



(a,b,c,d,f)









(a)

op1

op2

(a,b)

(a,c)

(a,b,c,d)

op3

op4

op5

merging



(a,b,c,d,e)

op6

© W. Wahlster, DFKI

Query Planning
-

III

City.value = München

City.language = German

...

State

CityName1

City

value

language

Language

Top

String

...

Ontology

< String, Language, String, Language >

+

+





babelfish

.

.

.

Operators

(

)

op

pre

c



"

c

c







=

$

c

i

S

i

::

:

0

)

(

0

op

Int

S

=



© W. Wahlster, DFKI

Query Plan Visualization


Features


alternative queries


past states


future states


state descriptions


PBD requests


accept / reject PBD
request


assessment of plans


expected completion time

© W. Wahlster, DFKI

Using the presentation planner to serve
mobile users


accommodate for device
-
specific display and
interaction limitations (e.g. 60*90 pixel displays)

The new challenge

-


develop new designs for presenting information

on mobile devices

-


identify elementary design elements

-


define new presentation strategies for the


automated composition of designs

Approach

© W. Wahlster, DFKI

Planner

“runway view”

“isometric
view”

“vertical
bar view”

a selected path gets
translated by the planner
into a sequence of WML
pages to be displayed on a
mobile phone

Application example: Route Descriptions over
the Mobile Phone

© W. Wahlster, DFKI

Video Otto 2

© W. Wahlster, DFKI

Use of a Life
-
like Character for Electronic Commerce

Digital Assistant Selector

© W. Wahlster, DFKI

Simulated Dialogues as a Novel Presentation
Technique


Presentation teams convey certain
rhetorical
relationships

in a more canonical way


Provide pros and cons


The single presenters can serve as
indices

which help
the user to classify information.


Provide information from different points of view, e.g.
businessman versus tourist


Presentation teams can serve as rhetorical devices that
allow for a continuous
reinforcement of beliefs


involve pseudo
-
experts to increase evidence




© W. Wahlster, DFKI

Presentation Teams

I recommend you this SLX
limousine.

© W. Wahlster, DFKI

Underlying Knowledge Base


Representation of domain


FACT attribute car_1 consumption_car_1



Value dimensions for cars adopted from a study of
the German car market


safety, economy, comfort, sportiness, prestige, family
and environmental friendliness


FACT polarity consumption_car_1 economy negative



Difficulty to infer implication of dimension on attribute


FACT difficulty consumption_car_1 economy low

© W. Wahlster, DFKI

Example of a Dialogue Strategy

Question:


How much gas does it consume?


Answer:


It consumes 8l per 100 km.


Negative Response:


I’m worrying about the running
costs.


Dampening Counter:


Forget about the costs.

Think of the prestige!


Header
:

(dampening_counter ?agent ?prop
?dim)


Constraints
:


(*and*


(positive ?agent)


(pol ?prop ?other_dim

positive))


Inferiors:



(Speak ?agent


(“Forget about the ” ?dim “!”))

(Speak ?agent


(“Think of the ” ?other_dim “!”))




© W. Wahlster, DFKI

But, it’s
fast!

Multiple Decision Support Agents

...

...

weighted propositions

User
-
Adaptive Search Planning

Spare parts for this car
are rather expensive!

Current and Future Work: Multiple Interface Agents for
User
-
adaptive Decision Support

© W. Wahlster, DFKI

l

The generation of virtual webpages is heavily based on






ontological annotation


l

The plan
-
based approach to generating virtual webpages





guarantees high
-
level ontological annotations


l

Ontological annotations are needed not only for information




extraction agents but also for presentation agents


l

Realization procedures and wrappers form an important part of the

referential semantics of objects on the web


l

Using presentation planning with temporal and spatial constraint


processing the low
-
level media synchronization can be done




automatically using SMIL annotations

Conclusion

© W. Wahlster, DFKI

Computer Science:


Elisabeth André

Mathias Bauer

Dietmar Dengler

Alexander Kröner

Marcus Meyer

Jochen Müller

Stephan Neurohr

Gaby Paul

Thomas Rist

Wolfgang Wahlster


Graphics Design:


Bernhard Kirsch

Renato Orsini

Peter Rist



Cognitive Psychology
:


Susanne van Mulken

The WebPersona and Virtual Webpages

Team at DFKI


© W. Wahlster, DFKI

URL of this Presentation:

http://www.dfki.de/~wahlster/micai_talk/