Extraordinary User Interfaces

quartercircleAI and Robotics

Nov 14, 2013 (3 years and 7 months ago)

73 views

Université catholique de Louvain (UCL)

Belgian Laboratory of Computer
-
Human Interaction (BCHI)

Place des Doyens, 1

B
-
1348 Louvain
-
la
-
Neuve (Belgium)

Presented by

University of Castilla
-
La Mancha
-

Albacete, November 15, 2004

1

Extraordinary User Interfaces

Jean Vanderdonckt


vanderdonckt@isys.ucl.ac.be

Head of BCHI Lab,
http://www.isys.ucl.ac.be/bchi

University of Castilla
-
La Mancha
-

Albacete, November 15, 2004

2

What is an extraordinary user
interface?


An extraordinary User Interface (UI)


may exhibit untraditional interaction


Techniques: drag & drop, fold & drop, pick & drop


Styles: mixed reality


Devices: head mounted display


but should not necessarily be limited to
sophisticated applications


You can do that too on existing UIs


You can also create extraordinary things on ordinary UIs

University of Castilla
-
La Mancha
-

Albacete, November 15, 2004

3

Target Applications,
Domains

Notations and tools

User Interface Interaction
Techniques

2004

Adapted from

[Palanque,2002]

No Interaction Technique

Automated, batch systems

Nothing

Character UIs

Business applications

Screen definitions

Graphical User

Interfaces

Information systems

Entity
-
relationship

Attribute model

State
-
transition diagrams

University of Castilla
-
La Mancha
-

Albacete, November 15, 2004

4

Graphical User Interfaces


Design once, use many


UsiXML = USer Interface eXtensible
Markup Language


See
http://www.usixml.org


FormiXML: forms editor for Java
applications


FlashiXML: rendering engine in SWF

University of Castilla
-
La Mancha
-

Albacete, November 15, 2004

5

Graphical User Interfaces


Usual interaction technique


Drag & drop


New interaction techniques


Fold & drop


Pick & drop

[Dragicevic, 2004]

[Rekimoto, 2001]

http://liihs.irit.fr/dragice/foldndrop/index.html

University of Castilla
-
La Mancha
-

Albacete, November 15, 2004

6

Target Applications,
Domains

Notations and tools

User Interface Interaction
Technique

2004

Adapted from

[Palanque,2002]

2005

No Interaction Technique

Automated, batch systems

Nothing

Character UIs

Business applications

Screen definitions

Graphical User

Interfaces

Information systems

Entity
-
relationship

Attribute model

State
-
transition diagrams

Multi
-
platform
User Interfaces

Web, Java
applications

Task model, context

model, UML
,…

University of Castilla
-
La Mancha
-

Albacete, November 15, 2004

7

The situation


Multiple computing platforms


Until 2005, the average user will be confronted with
at least 5 computing platforms


[Forrester,1999]

University of Castilla
-
La Mancha
-

Albacete, November 15, 2004

8

The situation



Home



Bookmarks



Yes



No



Sports



O

p

tions



Back



----

Home Page

-----



University of Castilla
-
La Mancha
-

Albacete, November 15, 2004

9

The problem


To develop user interfaces (UIs) simulta
-
neously for multiple contexts of use


A context of use = triple


User



Computing platform


Surrounding environment


Organisation


Socio
-
psychological factors


University of Castilla
-
La Mancha
-

Albacete, November 15, 2004

10

The problem


Why is it difficult?


For the designer:


Multiplicity of contexts of use


No systematic design method


For the user:


Poor usability of resulting UI


UI not adapted to the genuine context of use


For the developer:


Increase of development and maintenance costs


Increasing development complexity


Little of no factoring out of common elements

University of Castilla
-
La Mancha
-

Albacete, November 15, 2004

11

The problem


Why should it be systematic?


Some consider it noble to have a method


Other consider it noble not to have a method


Not to have a method is bad


But to stop entirely at any method is worse still


One should at first observe rules severely, then
change them in an intelligent way


The aim of possessing method is to seem finally as
if one had no method

[Lao Tch’ai Tche, many many years ago]

University of Castilla
-
La Mancha
-

Albacete, November 15, 2004

12

Mono
-
platform UI


Traditional approach


Visual

development

University of Castilla
-
La Mancha
-

Albacete, November 15, 2004

13

Mono
-
platform UI


Advantages of traditional approach


UI is graphical by nature



A UI prototype can be


Rapidly produced


Easily modified


Visually demonstrated

University of Castilla
-
La Mancha
-

Albacete, November 15, 2004

14

Mono
-
platform UI


Shortcomings of traditional approach


No structured method for developing a UI


All the knowledge remain implicit


Modification can lead to unstructured design


Little or no tool support for iterative design


Selection of widgets can be inappropriate


UI Layout can be tedious, repetitive


Problem of the spaghetti of callbacks


Early prototyping is hard to achieve and time
consuming


Limited reusability (throw away)


University of Castilla
-
La Mancha
-

Albacete, November 15, 2004

15

Mono
-
platform UI


Shortcomings of traditional approach


Limited use of UI builder



Table with dynamic data


Gantt chart


Direct manipulation

Desired interface

Obtained interface


Menu bar and pull
-
down menus


Toolbar


Scrollbars

[Szekely,1994]

University of Castilla
-
La Mancha
-

Albacete, November 15, 2004

16

Mono
-
platform UI

Interface builders

cannot build

their own UI

[Szekely,1994]

University of Castilla
-
La Mancha
-

Albacete, November 15, 2004

17

Mono
-
platform UI


Model
-
based approach


Models
: explicitly capture knowledge about
UI and Interactive Applications with
appropriate abstractions


Method
: structures the definition and use of
underlying models in a stage
-
wise approach


Supporting

tools
: support the use of the
method by providing tools for models and
their related operations. Ideally, one model
should be supported by at least one tool

University of Castilla
-
La Mancha
-

Albacete, November 15, 2004

18

Mono
-
platform UI


Goal: to integrate all three facets

Method

Model 1

Model …

Model
n

Models

Model

Model

Model

Interface

1

Tools

University of Castilla
-
La Mancha
-

Albacete, November 15, 2004

19

Multi
-
platform


Plastic UI: Virtual keyboard

[Grolaux, 2001]

University of Castilla
-
La Mancha
-

Albacete, November 15, 2004

20

Multi
-
platform


Plastic UI: FlexClock

[Grolaux, Van Roy, Vanderdonckt, 2002]

See
http://www.info.ucl.ac.be/people/ned/flexclock/

University of Castilla
-
La Mancha
-

Albacete, November 15, 2004

21

Multi
-
platform


Plastic UI: adaptable bounded value

See
http://www.isys.ucl.ac.be/bchi/research/adaptable/sizedemo.htm

University of Castilla
-
La Mancha
-

Albacete, November 15, 2004

22

Multi
-
platform for Emergency

[Amouh et al.,2004]

University of Castilla
-
La Mancha
-

Albacete, November 15, 2004

23

Multi
-
platform for Emergency


Three platforms


Pocket PC


Desktop PC


Wall Screens




See
http://www.tele.ucl.ac.be/PROJECTS/ARTHUR/

University of Castilla
-
La Mancha
-

Albacete, November 15, 2004

24

Multi
-
platform for Emergency


Model and method


Design the reference screen first


Refine the others screens later


By applying graceful degradation


By applying transformation techniques

See http://www.polymedis.com

University of Castilla
-
La Mancha
-

Albacete, November 15, 2004

25

Graceful degradation








[Florins & Vanderdonckt,2004]

University of Castilla
-
La Mancha
-

Albacete, November 15, 2004

26

Transformation rules






University of Castilla
-
La Mancha
-

Albacete, November 15, 2004

27

Transformation rules






Add all >>
Add >
<< Remove all
< Remove
>>
>
<<
<
>
<
Item 1
Item 2
Item 3
Item 4
Item 5
Item 6
Item 7
Item 1
Group box
Item 1
Item 2
Item 3
Item 4
Item 5
Item 6
Item 7
Item 8
Item 1
Item 2
Item 3
Item 4
Item 5
Item 6
Item 7
See http://www.isys.ucl.ac.be/bchi/publications/2004/Florins
-
IUI2004.pdf

University of Castilla
-
La Mancha
-

Albacete, November 15, 2004

28

Transformation rules






University of Castilla
-
La Mancha
-

Albacete, November 15, 2004

29

Target Applications,
Domains

Notations and tools

User Interface Interaction
Technique

2004

Adapted from

[Palanque,2002]

2005

No Interaction Technique

Automated, batch systems

Nothing

Character UIs

Business applications

Screen definitions

Graphical User

Interfaces

Information systems

Entity
-
relationship

Attribute model

State
-
transition diagrams

Multi
-
platform
User Interfaces

Web, Java
applications

Task model, context

model, UML
,…

Virtual Reality
User Interfaces

3D Applications

Scene model

University of Castilla
-
La Mancha
-

Albacete, November 15, 2004

30

Virtualisation of UIs


Going from 2D to 3D


Mapping widgets in 2D to 3D

[Vanderdonckt, Bouillon, Chow, 2004]

See http://www.isys.ucl.ac.be/bchi/publications/2004/Vanderdonckt
-
W3D2004.pdf

University of Castilla
-
La Mancha
-

Albacete, November 15, 2004

31

Virtualisation of UIs


Graphical rendering in 3D of UsiXML


Virtual world


[Molina, 2004]

University of Castilla
-
La Mancha
-

Albacete, November 15, 2004

32

Target Applications,
Domains

Notations and tools

User Interface Interaction
Technique

2004

Adapted from

[Palanque,2002]

2005

No Interaction Technique

Automated, batch systems

Nothing

Character UIs

Business applications

Screen definitions

Graphical User

Interfaces

Information systems

Entity
-
relationship

Attribute model

State
-
transition diagrams

Multi
-
platform
User Interfaces

Web, Java
applications

Task model, context

model, UML
,…

Virtual Reality
User Interfaces

3D Applications

Scene model

Mixed Reality

User Interfaces

Command and

control systems,
games

World models

University of Castilla
-
La Mancha
-

Albacete, November 15, 2004

33

Mixed reality


Mixed reality =
real world + digital
world

See http://urbicande.tele.ucl.ac.be

University of Castilla
-
La Mancha
-

Albacete, November 15, 2004

34

Mixed reality for games

See http://www.alterface.com

University of Castilla
-
La Mancha
-

Albacete, November 15, 2004

35

No representation for



tasks in real or virtual world


tasks independent of user interaction e.g.
tracking object of interest


various contexts of use in

Mixed Reality Systems

i.e. changing point
-
of
-
view

Adaptive IS composition

based on focus of interest

Mixed reality in surgery

[Trevisan, Gemo, Vanderdonckt, Macq, 2004]

University of Castilla
-
La Mancha
-

Albacete, November 15, 2004

36

Composing the MIS

Real CIO

Patient head

Virtual CIOs
Anatomic organs


Virtual CIOs
Annotations


Primary object

of interest


University of Castilla
-
La Mancha
-

Albacete, November 15, 2004

37



No overlapping annotations for readability



Highest priority of virtual object (3)



Medium priority of real object (2)



Low priority of background (1)



Annotations placed from peripheral to central zones



Other objects as part of Background

Annotation placement strategy

University of Castilla
-
La Mancha
-

Albacete, November 15, 2004

38

Context
-
aware Multimodal

Annotation Rendering

(a) Background context

(b) Background/Real

Object context

(c) Virtual Object context

-

what information to display

-

annotation positions

-

modality switch


User focus controls adaptive

presentation management

University of Castilla
-
La Mancha
-

Albacete, November 15, 2004

39

A
context model

describes all the entities that may
influence carrying out the interactive task of user with
the intended UI




Environment model

-

s
uch attributes may be physical (e.g.,
lighting conditions), psychological (e.g., level of stress), and
organization (e.g., location and role definition in the
organization chart).




Platform model


couple of software/hardware, such
atributes may be screen resolution/size



User model


stereotypes, sub
-
stereotypes (profiles)

Context model in mixed reality

[Trevisan, Gemo, Vanderdonckt, Macq, 2004]

University of Castilla
-
La Mancha
-

Albacete, November 15, 2004

40

Plasticity Domains

in the MIS

Plasticity thresholds and context

boundary values

Inter
-
context continuity

Perceptive, cognitive, functional
intra
-
context continuity and inter
-
context continuity

Intra
-
context / inter
-
context
frequency and frequency switch

Refined Metrics for Plastic MR systems

Plasticity analysis and usability
evaluation

[Trevisan, Gemo, Vanderdonckt, Macq, 2004]

Target Applications,
Domains

Notations and tools

User Interface Interaction
Technique

The Invisible UI

No more programming? No more bugs?

All Applications

2020

2004

2006

Adapted from

[Palanque,2002]

2005

No Interaction Technique

Automated, batch systems

Nothing

Character UIs

Business applications

Screen definitions

Graphical User

Interfaces

Information systems

Entity
-
relationship

Attribute model

State
-
transition diagrams

Multi
-
platform
User Interfaces

Web, Java
applications

Task model, context

model, UML
,…

Virtual Reality
User Interfaces

3D Applications

Scene model

Mixed Reality

User Interfaces

Command and

control systems,
games

World models

Tangible UIs

Embodied UIs

Physical
models

Embarked syst

University of Castilla
-
La Mancha
-

Albacete, November 15, 2004

42

Thank you very muchfor your
attention and see you in 2020

For more information and free downloading,

visit our web site at:

http://www.isys.ucl.ac.be/bchi


University of Castilla
-
La Mancha
-

Albacete, November 15, 2004

43

Special thanks to


Pascual González López


María Dolores Lozano Pérez



Víctor Manuel López Jaquero


José Pascual Molina Massó


Francisco Montero Simarro